chatgpt-md-converter 0.3.7__tar.gz → 0.3.9__tar.gz

{chatgpt_md_converter-0.3.7 → chatgpt_md_converter-0.3.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: chatgpt_md_converter
-Version: 0.3.7
+Version: 0.3.9
 Summary: A package for converting markdown to HTML for chat Telegram bots
 Home-page: https://github.com/botfather-dev/formatter-chatgpt-telegram
 Author: Kostiantyn Kriuchkov
@@ -114,6 +114,24 @@ Hidden by default
 Multiple lines</blockquote>
 ```
 
+
+## Performance
+
+Benchmarks were recorded on Linux 6.16.6 (Python 3.11.10) using 1,000 iterations per sample.
+
+| Sample       | Direction     | Avg ms/call | Ops/sec |
+|--------------|---------------|-------------|---------|
+| short_inline | Markdown→HTML | 0.043       | 23,476  |
+| short_inline | HTML→Markdown | 0.078       | 12,824  |
+| medium_block | Markdown→HTML | 0.108       |  9,270  |
+| medium_block | HTML→Markdown | 0.155       |  6,437  |
+| long_mixed   | Markdown→HTML | 0.446       |  2,242  |
+| long_mixed   | HTML→Markdown | 0.730       |  1,370  |
+
+These numbers provide a baseline; real-world throughput depends on text length and interpreter speed.
+
+Reproduce the measurements with `python scripts/benchmark.py --iterations 1000 --json benchmarks.json --summary BENCHMARKS.md`.
+
 ## Requirements
 
 - Python 3.x

chatgpt_md_converter-0.3.9/README.md

@@ -0,0 +1,165 @@
+# ChatGPT Markdown to Telegram HTML Parser
+
+## Overview
+
+This project provides a solution for converting Telegram-style Markdown formatted text into HTML markup supported by the Telegram Bot API, specifically tailored for use in ChatGPT bots developed with the OpenAI API. It includes features for handling various Markdown elements and ensures proper tag closure, making it suitable for streaming mode applications.
+
+## Features
+
+- Converts Telegram-style Markdown syntax to Telegram-compatible HTML
+- Supports text styling:
+  - Bold: `**text**` → `<b>text</b>`
+  - Italic: `*text*` or `_text_` → `<i>text</i>`
+  - Underline: `__text__` → `<u>text</u>`
+  - Strikethrough: `~~text~~` → `<s>text</s>`
+  - Spoiler: `||text||` → `<span class="tg-spoiler">text</span>`
+  - Inline code: `` `code` `` → `<code>code</code>`
+- Handles nested text styling
+- Converts links: `[text](URL)` → `<a href="URL">text</a>`
+- Processes code blocks with language specification
+- Supports blockquotes:
+  - Regular blockquotes: `> text` → `<blockquote>text</blockquote>`
+  - Expandable blockquotes: `**> text` → `<blockquote expandable>text</blockquote>`
+- Automatically appends missing closing delimiters for code blocks
+- Escapes HTML special characters to prevent unwanted HTML rendering
+
+## Usage
+
+To use the Markdown to Telegram HTML Parser in your ChatGPT bot, integrate the provided Python functions into your bot's processing pipeline. Here is a brief overview of how to incorporate the parser:
+
+1. **Ensure Closing Delimiters**: Automatically appends missing closing delimiters for backticks to ensure proper parsing.
+
+2. **Extract and Convert Code Blocks**: Extracts Markdown code blocks, converts them to HTML `<pre><code>` format, and replaces them with placeholders to prevent formatting within code blocks.
+
+3. **Markdown to HTML Conversion**: Applies various regex substitutions and custom logic to convert supported Markdown formatting to Telegram-compatible HTML tags.
+
+4. **Reinsert Code Blocks**: Reinserts the previously extracted and converted code blocks back into the main text, replacing placeholders with the appropriate HTML content.
+
+Simply call the `telegram_format(text: str) -> str` function with your Markdown-formatted text as input to receive the converted HTML output ready for use with the Telegram Bot API.
+
+## Installation
+
+```sh
+pip install chatgpt-md-converter
+```
+
+## Example
+
+```python
+from chatgpt_md_converter import telegram_format
+
+# Basic formatting example
+text = """
+Here is some **bold**, __underline__, and `inline code`.
+This is a ||spoiler text|| and *italic*.
+
+Code example:
+print('Hello, world!')
+"""
+
+# Blockquotes example
+blockquote_text = """
+> Regular blockquote
+> Multiple lines
+
+**> Expandable blockquote
+> Hidden by default
+> Multiple lines
+"""
+
+formatted_text = telegram_format(text)
+formatted_blockquote = telegram_format(blockquote_text)
+
+print(formatted_text)
+print(formatted_blockquote)
+```
+
+### Output:
+
+```
+Here is some <b>bold</b>, <u>underline</u>, and <code>inline code</code>.
+This is a <span class="tg-spoiler">spoiler text</span> and <i>italic</i>.
+
+Code example:
+print('Hello, world!')
+
+<blockquote>Regular blockquote
+Multiple lines</blockquote>
+
+<blockquote expandable>Expandable blockquote
+Hidden by default
+Multiple lines</blockquote>
+```
+
+
+## Performance
+
+Benchmarks were recorded on Linux 6.16.6 (Python 3.11.10) using 1,000 iterations per sample.
+
+| Sample       | Direction     | Avg ms/call | Ops/sec |
+|--------------|---------------|-------------|---------|
+| short_inline | Markdown→HTML | 0.043       | 23,476  |
+| short_inline | HTML→Markdown | 0.078       | 12,824  |
+| medium_block | Markdown→HTML | 0.108       |  9,270  |
+| medium_block | HTML→Markdown | 0.155       |  6,437  |
+| long_mixed   | Markdown→HTML | 0.446       |  2,242  |
+| long_mixed   | HTML→Markdown | 0.730       |  1,370  |
+
+These numbers provide a baseline; real-world throughput depends on text length and interpreter speed.
+
+Reproduce the measurements with `python scripts/benchmark.py --iterations 1000 --json benchmarks.json --summary BENCHMARKS.md`.
+
+## Requirements
+
+- Python 3.x
+- No external libraries required (uses built-in `re` module for regex operations)
+
+## Contribution
+
+Feel free to contribute to this project by submitting pull requests or opening issues for bugs, feature requests, or improvements.
+
+## Prompting LLMs for Telegram-Specific Formatting
+
+> **Note**:
+> Since standard Markdown doesn't include Telegram-specific features like spoilers (`||text||`) and expandable blockquotes (`**> text`), you'll need to explicitly instruct LLMs to use these formats. Here's a suggested prompt addition to include in your system message or initial instructions:
+
+````
+When formatting your responses for Telegram, please use these special formatting conventions:
+
+1. For content that should be hidden as a spoiler (revealed only when users click):
+   Use: ||spoiler content here||
+   Example: This is visible, but ||this is hidden until clicked||.
+
+2. For lengthy explanations or optional content that should be collapsed:
+   Use: **> Expandable section title
+
+   > Content line 1
+   > Content line 2
+   > (Each line of the expandable blockquote should start with ">")
+
+3. Continue using standard markdown for other formatting:
+   - **bold text**
+   - *italic text*
+   - __underlined text__
+   - ~~strikethrough~~
+   - `inline code`
+   - ```code blocks```
+   - [link text](URL)
+
+Apply spoilers for:
+
+- Solution reveals
+- Potential plot spoilers
+- Sensitive information
+- Surprising facts
+
+Use expandable blockquotes for:
+
+- Detailed explanations
+- Long examples
+- Optional reading
+- Technical details
+- Additional context not needed by all users
+````
+
+You can add this prompt to your system message when initializing your ChatGPT interactions to ensure the model properly formats content for optimal display in Telegram.

chatgpt_md_converter-0.3.9/chatgpt_md_converter/__init__.py

@@ -0,0 +1,5 @@
+from .html_splitter import split_html_for_telegram
+from .html_to_markdown import html_to_telegram_markdown
+from .telegram_formatter import telegram_format
+
+__all__ = ["telegram_format", "split_html_for_telegram", "html_to_telegram_markdown"]

chatgpt_md_converter-0.3.9/chatgpt_md_converter/html_markdown/escaping.py

@@ -0,0 +1,68 @@
+"""Shared escaping utilities for Telegram Markdown conversion."""
+
+from __future__ import annotations
+
+import html
+import re
+
+from .tree import Node
+
+_SIMPLE_STAR_ITALIC = re.compile(
+    r"(?<!\\)(?<!\*)\*(?=[^\s])([^\*\n]+?)(?<!\s)\*(?![A-Za-z0-9\*])",
+)
+
+
+def _canonicalize_star_italics(text: str) -> str:
+    def _replace(match: re.Match[str]) -> str:
+        inner = match.group(1)
+        if "*" in inner or "_" in inner or "`" in inner:
+            return match.group(0)
+        return f"_{inner}_"
+
+    return _SIMPLE_STAR_ITALIC.sub(_replace, text)
+
+
+def normalise_text(text: str) -> str:
+    if not text:
+        return ""
+    unescaped = html.unescape(text)
+    return unescaped.replace("\u00a0", " ")
+
+
+def collect_text(node: Node) -> str:
+    if node.kind == "text":
+        return html.unescape(node.text)
+    parts: list[str] = []
+    for child in node.children:
+        if child.kind == "text":
+            parts.append(html.unescape(child.text))
+        elif child.kind == "element":
+            if child.tag.lower() == "br":
+                parts.append("\n")
+            else:
+                parts.append(collect_text(child))
+    return "".join(parts)
+
+
+def escape_inline_code(text: str) -> str:
+    return text.replace("`", "\\`")
+
+
+def escape_link_label(label: str) -> str:
+    escaped = label
+    for ch in "[]()":
+        escaped = escaped.replace(ch, f"\\{ch}")
+    return escaped
+
+
+def escape_link_url(url: str) -> str:
+    return url.replace("\\", "\\\\").replace(")", "\\)")
+
+
+def post_process(markdown: str) -> str:
+    text = re.sub(r"(^|\n)•\s", r"\1- ", markdown)
+    text = re.sub(r"\n{3,}", "\n\n", text)
+    text = text.replace("\r", "")
+    text = "\n".join(line.rstrip() for line in text.split("\n"))
+    text = _canonicalize_star_italics(text)
+    return text.strip()

chatgpt_md_converter-0.3.9/chatgpt_md_converter/html_markdown/handlers.py

@@ -0,0 +1,155 @@
+"""Tag-specific renderers for Telegram Markdown."""
+
+from __future__ import annotations
+
+from typing import Callable, Dict
+
+from .escaping import (collect_text, escape_inline_code, escape_link_label,
+                       escape_link_url, normalise_text)
+from .state import RenderState
+from .tree import Node
+
+InlineHandler = Callable[[Node, RenderState], str]
+
+
+_INLINE_MARKERS: Dict[str, tuple[str, str]] = {
+    "u": ("__", "__"),
+    "ins": ("__", "__"),
+    "s": ("~~", "~~"),
+    "strike": ("~~", "~~"),
+    "del": ("~~", "~~"),
+}
+
+
+def render_nodes(nodes: list[Node], state: RenderState) -> str:
+    return "".join(render_node(node, state) for node in nodes)
+
+
+def render_node(node: Node, state: RenderState) -> str:
+    if node.kind == "text":
+        return normalise_text(node.text)
+
+    handler = TAG_DISPATCH.get(node.tag.lower())
+    if handler:
+        return handler(node, state)
+    return render_nodes(node.children, state)
+
+
+def _handle_bold(node: Node, state: RenderState) -> str:
+    inner_state = state.child(bold_depth=state.bold_depth + 1)
+    inner = render_nodes(node.children, inner_state)
+    return f"**{inner}**"
+
+
+def _handle_italic(node: Node, state: RenderState) -> str:
+    depth = state.italic_depth
+    in_bold = state.bold_depth > 0 and depth == 0
+    marker = "_" if in_bold else ("*" if depth % 2 == 0 else "_")
+    inner_state = state.child(italic_depth=depth + 1)
+    inner = render_nodes(node.children, inner_state)
+    return f"{marker}{inner}{marker}"
+
+
+def _handle_inline_marker(node: Node, state: RenderState) -> str:
+    marker_open, marker_close = _INLINE_MARKERS[node.tag.lower()]
+    inner = render_nodes(node.children, state)
+    return f"{marker_open}{inner}{marker_close}"
+
+
+def _handle_spoiler(node: Node, state: RenderState) -> str:
+    inner = render_nodes(node.children, state)
+    return f"||{inner}||"
+
+
+def _handle_code(node: Node, state: RenderState) -> str:
+    inner = collect_text(node)
+    return f"`{escape_inline_code(inner)}`"
+
+
+def _handle_pre(node: Node, state: RenderState) -> str:
+    children = node.children
+    language: str | None = None
+    content_node: Node
+
+    if len(children) == 1 and children[0].kind == "element" and children[0].tag.lower() == "code":
+        content_node = children[0]
+        class_attr = content_node.attrs.get("class") or ""
+        for part in class_attr.split():
+            if part.startswith("language-"):
+                language = part.split("-", 1)[1]
+                break
+    else:
+        content_node = Node(kind="element", tag="__virtual__", children=children)
+
+    inner_text = collect_text(content_node)
+    fence = f"```{language}" if language else "```"
+    if language or "\n" in inner_text:
+        return f"{fence}\n{inner_text}```"
+    return f"{fence}{inner_text}```"
+
+
+def _handle_link(node: Node, state: RenderState) -> str:
+    href = node.attrs.get("href", "") or ""
+    label = render_nodes(node.children, state)
+    if not label:
+        label = href
+
+    escaped_label = escape_link_label(label)
+    escaped_url = escape_link_url(href)
+
+    if href.startswith("tg://emoji?"):
+        return f"![{escaped_label}]({escaped_url})"
+    return f"[{escaped_label}]({escaped_url})"
+
+
+def _handle_blockquote(node: Node, state: RenderState) -> str:
+    inner = render_nodes(node.children, state)
+    lines = inner.split("\n")
+    expandable = "expandable" in node.attrs
+    rendered: list[str] = []
+    for index, line in enumerate(lines):
+        prefix = "**>" if expandable and index == 0 else ">"
+        stripped = line.rstrip("\r")
+        if expandable:
+            rendered.append(prefix + stripped)
+        else:
+            rendered.append(f"{prefix} {stripped}" if stripped else prefix)
+    return "\n".join(rendered)
+
+
+def _handle_tg_emoji(node: Node, state: RenderState) -> str:
+    emoji_id = node.attrs.get("emoji-id")
+    label = render_nodes(node.children, state)
+    if emoji_id:
+        href = f"tg://emoji?id={emoji_id}"
+        return f"![{escape_link_label(label)}]({href})"
+    return label
+
+
+def _handle_span(node: Node, state: RenderState) -> str:
+    classes = (node.attrs.get("class") or "").split()
+    if any(cls == "tg-spoiler" for cls in classes):
+        return _handle_spoiler(node, state)
+    if any(cls == "tg-emoji" for cls in classes):
+        return render_nodes(node.children, state)
+    return render_nodes(node.children, state)
+
+
+TAG_DISPATCH: Dict[str, Callable[[Node, RenderState], str]] = {
+    "b": _handle_bold,
+    "strong": _handle_bold,
+    "i": _handle_italic,
+    "em": _handle_italic,
+    "u": _handle_inline_marker,
+    "ins": _handle_inline_marker,
+    "s": _handle_inline_marker,
+    "strike": _handle_inline_marker,
+    "del": _handle_inline_marker,
+    "span": _handle_span,
+    "tg-spoiler": _handle_spoiler,
+    "code": _handle_code,
+    "pre": _handle_pre,
+    "a": _handle_link,
+    "blockquote": _handle_blockquote,
+    "tg-emoji": _handle_tg_emoji,
+}

chatgpt_md_converter-0.3.9/chatgpt_md_converter/html_markdown/renderer.py

@@ -0,0 +1,16 @@
+"""High-level HTML → Telegram Markdown renderer."""
+
+from __future__ import annotations
+
+from typing import List
+
+from .escaping import post_process
+from .handlers import render_nodes
+from .state import RenderState
+from .tree import Node, build_tree
+
+
+def html_to_telegram_markdown(html_text: str) -> str:
+    nodes: List[Node] = build_tree(html_text)
+    markdown = render_nodes(nodes, RenderState())
+    return post_process(markdown)

chatgpt_md_converter-0.3.9/chatgpt_md_converter/html_markdown/state.py

@@ -0,0 +1,16 @@
+"""Rendering state for HTML → Telegram Markdown conversion."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+
+@dataclass(frozen=True)
+class RenderState:
+    bold_depth: int = 0
+    italic_depth: int = 0
+
+    def child(self, **updates: int) -> "RenderState":
+        data = {"bold_depth": self.bold_depth, "italic_depth": self.italic_depth}
+        data.update(updates)
+        return RenderState(**data)

chatgpt_md_converter-0.3.9/chatgpt_md_converter/html_markdown/tree.py

@@ -0,0 +1,65 @@
+"""DOM-like tree construction for Telegram HTML fragments."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+from html.parser import HTMLParser
+from typing import Dict, List, Optional
+
+
+@dataclass
+class Node:
+    kind: str  # "text" or "element"
+    text: str = ""
+    tag: str = ""
+    attrs: Dict[str, Optional[str]] = field(default_factory=dict)
+    children: List["Node"] = field(default_factory=list)
+
+
+class _HTMLTreeBuilder(HTMLParser):
+    SELF_CLOSING_TAGS = {"br"}
+
+    def __init__(self) -> None:
+        super().__init__(convert_charrefs=False)
+        self.root = Node(kind="element", tag="__root__")
+        self._stack: List[Node] = [self.root]
+
+    def handle_starttag(self, tag: str, attrs: List[tuple[str, Optional[str]]]) -> None:
+        if tag in self.SELF_CLOSING_TAGS:
+            if tag == "br":
+                self._stack[-1].children.append(Node(kind="text", text="\n"))
+            return
+        node = Node(kind="element", tag=tag, attrs=dict(attrs))
+        self._stack[-1].children.append(node)
+        self._stack.append(node)
+
+    def handle_endtag(self, tag: str) -> None:
+        for index in range(len(self._stack) - 1, 0, -1):
+            if self._stack[index].tag == tag:
+                del self._stack[index:]
+                return
+
+    def handle_startendtag(self, tag: str, attrs: List[tuple[str, Optional[str]]]) -> None:
+        if tag in self.SELF_CLOSING_TAGS:
+            self.handle_starttag(tag, attrs)
+            return
+        node = Node(kind="element", tag=tag, attrs=dict(attrs))
+        self._stack[-1].children.append(node)
+
+    def handle_data(self, data: str) -> None:
+        if data:
+            self._stack[-1].children.append(Node(kind="text", text=data))
+
+    def handle_entityref(self, name: str) -> None:
+        self.handle_data(f"&{name};")
+
+    def handle_charref(self, name: str) -> None:
+        self.handle_data(f"&#{name};")
+
+
+def build_tree(html_text: str) -> List[Node]:
+    """Parse HTML and return the list of top-level nodes."""
+    builder = _HTMLTreeBuilder()
+    builder.feed(html_text)
+    builder.close()
+    return builder.root.children