chatgpt-md-converter 0.3.6__tar.gz → 0.3.8__tar.gz

{chatgpt_md_converter-0.3.6 → chatgpt_md_converter-0.3.8}/PKG-INFO

@@ -1,8 +1,8 @@
 Metadata-Version: 2.4
 Name: chatgpt_md_converter
-Version: 0.3.6
+Version: 0.3.8
 Summary: A package for converting markdown to HTML for chat Telegram bots
-Home-page: https://github.com/Latand/formatter-chatgpt-telegram
+Home-page: https://github.com/botfather-dev/formatter-chatgpt-telegram
 Author: Kostiantyn Kriuchkov
 Author-email: latand666@gmail.com
 Classifier: Programming Language :: Python :: 3

chatgpt_md_converter-0.3.8/README.md

@@ -0,0 +1,147 @@
+# ChatGPT Markdown to Telegram HTML Parser
+
+## Overview
+
+This project provides a solution for converting Telegram-style Markdown formatted text into HTML markup supported by the Telegram Bot API, specifically tailored for use in ChatGPT bots developed with the OpenAI API. It includes features for handling various Markdown elements and ensures proper tag closure, making it suitable for streaming mode applications.
+
+## Features
+
+- Converts Telegram-style Markdown syntax to Telegram-compatible HTML
+- Supports text styling:
+  - Bold: `**text**` → `<b>text</b>`
+  - Italic: `*text*` or `_text_` → `<i>text</i>`
+  - Underline: `__text__` → `<u>text</u>`
+  - Strikethrough: `~~text~~` → `<s>text</s>`
+  - Spoiler: `||text||` → `<span class="tg-spoiler">text</span>`
+  - Inline code: `` `code` `` → `<code>code</code>`
+- Handles nested text styling
+- Converts links: `[text](URL)` → `<a href="URL">text</a>`
+- Processes code blocks with language specification
+- Supports blockquotes:
+  - Regular blockquotes: `> text` → `<blockquote>text</blockquote>`
+  - Expandable blockquotes: `**> text` → `<blockquote expandable>text</blockquote>`
+- Automatically appends missing closing delimiters for code blocks
+- Escapes HTML special characters to prevent unwanted HTML rendering
+
+## Usage
+
+To use the Markdown to Telegram HTML Parser in your ChatGPT bot, integrate the provided Python functions into your bot's processing pipeline. Here is a brief overview of how to incorporate the parser:
+
+1. **Ensure Closing Delimiters**: Automatically appends missing closing delimiters for backticks to ensure proper parsing.
+
+2. **Extract and Convert Code Blocks**: Extracts Markdown code blocks, converts them to HTML `<pre><code>` format, and replaces them with placeholders to prevent formatting within code blocks.
+
+3. **Markdown to HTML Conversion**: Applies various regex substitutions and custom logic to convert supported Markdown formatting to Telegram-compatible HTML tags.
+
+4. **Reinsert Code Blocks**: Reinserts the previously extracted and converted code blocks back into the main text, replacing placeholders with the appropriate HTML content.
+
+Simply call the `telegram_format(text: str) -> str` function with your Markdown-formatted text as input to receive the converted HTML output ready for use with the Telegram Bot API.
+
+## Installation
+
+```sh
+pip install chatgpt-md-converter
+```
+
+## Example
+
+```python
+from chatgpt_md_converter import telegram_format
+
+# Basic formatting example
+text = """
+Here is some **bold**, __underline__, and `inline code`.
+This is a ||spoiler text|| and *italic*.
+
+Code example:
+print('Hello, world!')
+"""
+
+# Blockquotes example
+blockquote_text = """
+> Regular blockquote
+> Multiple lines
+
+**> Expandable blockquote
+> Hidden by default
+> Multiple lines
+"""
+
+formatted_text = telegram_format(text)
+formatted_blockquote = telegram_format(blockquote_text)
+
+print(formatted_text)
+print(formatted_blockquote)
+```
+
+### Output:
+
+```
+Here is some <b>bold</b>, <u>underline</u>, and <code>inline code</code>.
+This is a <span class="tg-spoiler">spoiler text</span> and <i>italic</i>.
+
+Code example:
+print('Hello, world!')
+
+<blockquote>Regular blockquote
+Multiple lines</blockquote>
+
+<blockquote expandable>Expandable blockquote
+Hidden by default
+Multiple lines</blockquote>
+```
+
+## Requirements
+
+- Python 3.x
+- No external libraries required (uses built-in `re` module for regex operations)
+
+## Contribution
+
+Feel free to contribute to this project by submitting pull requests or opening issues for bugs, feature requests, or improvements.
+
+## Prompting LLMs for Telegram-Specific Formatting
+
+> **Note**:
+> Since standard Markdown doesn't include Telegram-specific features like spoilers (`||text||`) and expandable blockquotes (`**> text`), you'll need to explicitly instruct LLMs to use these formats. Here's a suggested prompt addition to include in your system message or initial instructions:
+
+````
+When formatting your responses for Telegram, please use these special formatting conventions:
+
+1. For content that should be hidden as a spoiler (revealed only when users click):
+   Use: ||spoiler content here||
+   Example: This is visible, but ||this is hidden until clicked||.
+
+2. For lengthy explanations or optional content that should be collapsed:
+   Use: **> Expandable section title
+
+   > Content line 1
+   > Content line 2
+   > (Each line of the expandable blockquote should start with ">")
+
+3. Continue using standard markdown for other formatting:
+   - **bold text**
+   - *italic text*
+   - __underlined text__
+   - ~~strikethrough~~
+   - `inline code`
+   - ```code blocks```
+   - [link text](URL)
+
+Apply spoilers for:
+
+- Solution reveals
+- Potential plot spoilers
+- Sensitive information
+- Surprising facts
+
+Use expandable blockquotes for:
+
+- Detailed explanations
+- Long examples
+- Optional reading
+- Technical details
+- Additional context not needed by all users
+````
+
+You can add this prompt to your system message when initializing your ChatGPT interactions to ensure the model properly formats content for optimal display in Telegram.

chatgpt_md_converter-0.3.8/chatgpt_md_converter/__init__.py

@@ -0,0 +1,4 @@
+from .telegram_formatter import telegram_format
+from .html_splitter import split_html_for_telegram
+
+__all__ = ["telegram_format", "split_html_for_telegram"]

chatgpt_md_converter-0.3.8/chatgpt_md_converter/extractors.py

@@ -0,0 +1,95 @@
+import re
+
+
+def ensure_closing_delimiters(text: str) -> str:
+    # Append missing closing backtick delimiters.
+
+    code_block_re = re.compile(
+        r"(?P<fence>`{3,})(?P<lang>\w+)?\n?[\s\S]*?(?<=\n)?(?P=fence)",
+        flags=re.DOTALL,
+    )
+
+    # Track an open fence.  Once a fence is opened, everything until the same
+    # fence is encountered again is treated as plain text.  This mimics how
+    # Markdown handles fences and allows fence-like strings inside code blocks.
+    open_fence = None
+    for line in text.splitlines():
+        stripped = line.strip()
+        if open_fence is None:
+            m = re.match(r"^(?P<fence>`{3,})(?P<lang>\w+)?$", stripped)
+            if m:
+                open_fence = m.group("fence")
+        else:
+            if stripped.endswith(open_fence):
+                open_fence = None
+
+    # If a fence was left open, append a matching closing fence.
+    if open_fence is not None:
+        if not text.endswith("\n"):
+            text += "\n"
+        text += open_fence
+
+    cleaned_inline = code_block_re.sub("", text)
+
+    # Balance triple backticks that are not part of a complete fence.
+    if cleaned_inline.count("```") % 2 != 0:
+        text += "```"
+
+    # Balance single backticks outside fenced blocks.
+    cleaned_inline = code_block_re.sub("", text)
+    if cleaned_inline.count("`") % 2 != 0:
+        text += "`"
+
+    return text
+
+
+def extract_and_convert_code_blocks(text: str):
+    """
+    Extracts code blocks from the text, converting them to HTML <pre><code> format,
+    and replaces them with placeholders. Also ensures closing delimiters for unmatched blocks.
+    """
+    text = ensure_closing_delimiters(text)
+    placeholders = []
+    code_blocks = {}
+
+    def replacer(match):
+        language = match.group("lang") if match.group("lang") else ""
+        code_content = match.group("code")
+
+        # Properly escape HTML entities in code content
+        escaped_content = (
+            code_content.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
+        )
+
+        placeholder = f"CODEBLOCKPLACEHOLDER{len(placeholders)}"
+        placeholders.append(placeholder)
+        if not language:
+            html_code_block = f"<pre><code>{escaped_content}</code></pre>"
+        else:
+            html_code_block = (
+                f'<pre><code class="language-{language}">{escaped_content}</code></pre>'
+            )
+        return (placeholder, html_code_block)
+
+    modified_text = text
+    code_block_pattern = re.compile(
+        r"(?P<fence>`{3,})(?P<lang>\w+)?\n?(?P<code>[\s\S]*?)(?<=\n)?(?P=fence)",
+        flags=re.DOTALL,
+    )
+    for match in code_block_pattern.finditer(text):
+        placeholder, html_code_block = replacer(
+            match
+        )
+        code_blocks[placeholder] = html_code_block
+        modified_text = modified_text.replace(match.group(0), placeholder, 1)
+
+    return modified_text, code_blocks
+
+
+def reinsert_code_blocks(text: str, code_blocks: dict) -> str:
+    """
+    Reinserts HTML code blocks into the text, replacing their placeholders.
+    """
+    for placeholder, html_code_block in code_blocks.items():
+        text = text.replace(placeholder, html_code_block, 1)
+    return text

chatgpt_md_converter-0.3.8/chatgpt_md_converter/html_splitter.py

@@ -0,0 +1,239 @@
+import re
+from html.parser import HTMLParser
+
+MAX_LENGTH = 4096
+MIN_LENGTH = 500
+
+
+class HTMLTagTracker(HTMLParser):
+    def __init__(self):
+        super().__init__()
+        self.open_tags = []
+
+    def handle_starttag(self, tag, attrs):
+        # saving tags
+        if tag in (
+                  "b", "i", "u", "s", "code", "pre", "a", "span", "blockquote",
+                  "strong", "em", "ins", "strike", "del", "tg-spoiler", "tg-emoji"
+                  ):
+            self.open_tags.append((tag, attrs))
+
+    def handle_endtag(self, tag):
+        for i in range(len(self.open_tags) - 1, -1, -1):
+            if self.open_tags[i][0] == tag:
+                del self.open_tags[i]
+                break
+
+    def get_open_tags_html(self):
+        parts = []
+        for tag, attrs in self.open_tags:
+            attr_str = ""
+            if attrs:
+                attr_str = " " + " ".join(f'{k}="{v}"' for k, v in attrs)
+            parts.append(f"<{tag}{attr_str}>")
+        return "".join(parts)
+
+    def get_closing_tags_html(self):
+        return "".join(f"</{tag}>" for tag, _ in reversed(self.open_tags))
+
+
+def split_pre_block(pre_block: str, max_length) -> list[str]:
+    """
+    Splits long HTML-formatted text into chunks suitable for sending via Telegram,
+    preserving valid HTML tag nesting and handling <pre>/<code> blocks separately.
+
+    Args:
+        text (str): The input HTML-formatted string.
+        trim_leading_newlines (bool): If True, removes leading newline characters (`\\n`)
+            from each resulting chunk before sending. This is useful to avoid
+            unnecessary blank space at the beginning of messages in Telegram.
+
+    Returns:
+        list[str]: A list of HTML-formatted message chunks, each within Telegram's length limit.
+    """
+
+    # language-aware: <pre><code class="language-python">...</code></pre>
+    match = re.match(r"<pre><code(.*?)>(.*)</code></pre>", pre_block, re.DOTALL)
+    if match:
+        attr, content = match.groups()
+        lines = content.splitlines(keepends=True)
+        chunks, buf = [], ""
+        overhead = len(f"<pre><code{attr}></code></pre>")
+        for line in lines:
+            if len(buf) + len(line) + overhead > max_length:
+                chunks.append(f"<pre><code{attr}>{buf}</code></pre>")
+                buf = ""
+            buf += line
+        if buf:
+            chunks.append(f"<pre><code{attr}>{buf}</code></pre>")
+        return chunks
+    else:
+        # regular <pre>...</pre>
+        inner = pre_block[5:-6]
+        lines = inner.splitlines(keepends=True)
+        chunks, buf = [], ""
+        overhead = len('<pre></pre>')
+        for line in lines:
+            if len(buf) + len(line) + overhead > max_length:
+                chunks.append(f"<pre>{buf}</pre>")
+                buf = ""
+            buf += line
+        if buf:
+            chunks.append(f"<pre>{buf}</pre>")
+        return chunks
+
+
+def _is_only_tags(block: str) -> bool:
+    return bool(re.fullmatch(r'(?:\s*<[^>]+>\s*)+', block))
+
+
+def _effective_length(content: str) -> int:
+    tracker = HTMLTagTracker()
+    tracker.feed(content)
+    return len(tracker.get_open_tags_html()) + len(content) + len(tracker.get_closing_tags_html())
+
+
+def split_html_for_telegram(text: str, trim_empty_leading_lines: bool = False, max_length: int = MAX_LENGTH) -> list[str]:
+    """Split long HTML-formatted text into Telegram-compatible chunks.
+
+    Parameters
+    ----------
+    text: str
+        Input HTML text.
+    trim_empty_leading_lines: bool, optional
+        If True, removes `\n` sybmols from start of chunks.
+    max_length: int, optional
+        Maximum allowed length for a single chunk (must be >= ``MIN_LENGTH = 500``).
+        Default = 4096 (symbols)
+
+    Returns
+    -------
+    list[str]
+        List of HTML chunks.
+    """
+
+    if max_length < MIN_LENGTH:
+        raise ValueError("max_length should be at least %d" % MIN_LENGTH)
+
+    pattern = re.compile(r"(<pre>.*?</pre>|<pre><code.*?</code></pre>)", re.DOTALL)
+    parts = pattern.split(text)
+
+    chunks: list[str] = []
+    prefix = ""
+    current = ""
+    whitespace_re = re.compile(r"(\\s+)")
+    tag_re = re.compile(r"(<[^>]+>)")
+
+    def finalize():
+        nonlocal current, prefix
+        tracker = HTMLTagTracker()
+        tracker.feed(prefix + current)
+        chunk = prefix + current + tracker.get_closing_tags_html()
+        chunks.append(chunk)
+        prefix = tracker.get_open_tags_html()
+        current = ""
+
+    def append_piece(piece: str):
+        nonlocal current, prefix
+
+        def split_on_whitespace(chunk: str) -> list[str] | None:
+            parts = [part for part in whitespace_re.split(chunk) if part]
+            if len(parts) <= 1:
+                return None
+            return parts
+
+        def split_on_tags(chunk: str) -> list[str] | None:
+            parts = [part for part in tag_re.split(chunk) if part]
+            if len(parts) <= 1:
+                return None
+            return parts
+
+        def fittable_prefix_length(chunk: str) -> int:
+            low, high = 1, len(chunk)
+            best = 0
+            while low <= high:
+                mid = (low + high) // 2
+                candidate = chunk[:mid]
+                if _effective_length(prefix + current + candidate) <= max_length:
+                    best = mid
+                    low = mid + 1
+                else:
+                    high = mid - 1
+            return best
+
+        while piece:
+            if _effective_length(prefix + current + piece) <= max_length:
+                current += piece
+                return
+
+            if len(piece) > max_length:
+                if _is_only_tags(piece):
+                    raise ValueError("block contains only html tags")
+                splitted = split_on_whitespace(piece)
+                if splitted:
+                    for part in splitted:
+                        append_piece(part)
+                    return
+                tag_split = split_on_tags(piece)
+                if tag_split:
+                    for part in tag_split:
+                        append_piece(part)
+                    return
+            elif current:
+                finalize()
+                continue
+            else:
+                splitted = split_on_whitespace(piece)
+                if splitted:
+                    for part in splitted:
+                        append_piece(part)
+                    return
+                tag_split = split_on_tags(piece)
+                if tag_split:
+                    for part in tag_split:
+                        append_piece(part)
+                    return
+
+            fitted = fittable_prefix_length(piece)
+            if fitted == 0:
+                if current:
+                    finalize()
+                    continue
+                raise ValueError("unable to split content within max_length")
+
+            current += piece[:fitted]
+            piece = piece[fitted:]
+
+            if piece:
+                finalize()
+
+
+    for part in parts:
+        if not part:
+            continue
+        if part.startswith("<pre>") or part.startswith("<pre><code"):
+            pre_chunks = split_pre_block(part, max_length=max_length)
+            for pc in pre_chunks:
+                append_piece(pc)
+            continue
+        blocks = re.split(r"(\n\s*\n|<br\s*/?>|\n)", part)
+        for block in blocks:
+            if block:
+                append_piece(block)
+
+    if current:
+        finalize()
+
+    merged: list[str] = []
+    buf = ""
+    for chunk in chunks:
+        if len(buf) + len(chunk) <= max_length:
+            buf += chunk
+        else:
+            if buf:
+                merged.append(buf)
+            buf = chunk.lstrip("\n") if trim_empty_leading_lines and merged else chunk
+    if buf:
+        merged.append(buf.lstrip("\n") if trim_empty_leading_lines and merged else buf)
+
+    return merged

{chatgpt_md_converter-0.3.6 → chatgpt_md_converter-0.3.8}/chatgpt_md_converter.egg-info/PKG-INFO

@@ -1,8 +1,8 @@
 Metadata-Version: 2.4
 Name: chatgpt_md_converter
-Version: 0.3.6
+Version: 0.3.8
 Summary: A package for converting markdown to HTML for chat Telegram bots
-Home-page: https://github.com/Latand/formatter-chatgpt-telegram
+Home-page: https://github.com/botfather-dev/formatter-chatgpt-telegram
 Author: Kostiantyn Kriuchkov
 Author-email: latand666@gmail.com
 Classifier: Programming Language :: Python :: 3

{chatgpt_md_converter-0.3.6 → chatgpt_md_converter-0.3.8}/chatgpt_md_converter.egg-info/SOURCES.txt

@@ -1,13 +1,16 @@
 LICENSE
+README.md
 setup.py
 chatgpt_md_converter/__init__.py
 chatgpt_md_converter/converters.py
 chatgpt_md_converter/extractors.py
 chatgpt_md_converter/formatters.py
 chatgpt_md_converter/helpers.py
+chatgpt_md_converter/html_splitter.py
 chatgpt_md_converter/telegram_formatter.py
 chatgpt_md_converter.egg-info/PKG-INFO
 chatgpt_md_converter.egg-info/SOURCES.txt
 chatgpt_md_converter.egg-info/dependency_links.txt
 chatgpt_md_converter.egg-info/top_level.txt
-tests/test_parser.py
+tests/test_parser.py
+tests/test_splitter.py

{chatgpt_md_converter-0.3.6 → chatgpt_md_converter-0.3.8}/setup.py

@@ -2,13 +2,13 @@ from setuptools import setup
 
 setup(
     name="chatgpt_md_converter",
-    version="0.3.6",
+    version="0.3.8",
     author="Kostiantyn Kriuchkov",
     author_email="latand666@gmail.com",
     description="A package for converting markdown to HTML for chat Telegram bots",
-    long_description=open("README.MD").read(),
+    long_description=open("README.md").read(),
     long_description_content_type="text/markdown",
-    url="https://github.com/Latand/formatter-chatgpt-telegram",
+    url="https://github.com/botfather-dev/formatter-chatgpt-telegram",
     classifiers=[
         "Programming Language :: Python :: 3",
         "License :: OSI Approved :: MIT License",

{chatgpt_md_converter-0.3.6 → chatgpt_md_converter-0.3.8}/tests/test_parser.py

@@ -730,3 +730,208 @@ def test_ukrainian_text_with_inline_code():
     expected_output = """звісно, майстре тестування. ой та зрозуміло <code>&lt;LAUGH&gt;</code> що ти тут тестуєш."""
     output = telegram_format(input_text)
     assert output == expected_output, f"Output was: {output}"
+
+
+def test_nested_code_fence_quadruple():
+    input_text = """````markdown
+```python
+def hello_world():
+    print("Hello, World!")
+```
+````"""
+    expected_output = (
+        "<pre><code class=\"language-markdown\">```python\n"
+        "def hello_world():\n    print(\"Hello, World!\")\n```\n</code></pre>"
+    )
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+
+def test_nested_code_fence_quadruple_no_lang():
+    input_text = """````
+```python
+print('hi')
+```
+````"""
+    expected_output = (
+        "<pre><code>```python\nprint('hi')\n```\n</code></pre>"
+    )
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+
+def test_nested_code_fence_five_backticks():
+    input_text = """`````markdown
+````python
+print(1)
+````
+`````"""
+    expected_output = (
+        "<pre><code class=\"language-markdown\">````python\nprint(1)\n````\n</code></pre>"
+    )
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+
+def test_nested_code_fence_five_backticks_with_inner_triple():
+    input_text = """`````markdown
+````python
+print("hello world ```")
+````
+`````"""
+    expected_output = (
+        "<pre><code class=\"language-markdown\">````python\n"
+        "print(\"hello world ```\")\n````\n</code></pre>"
+    )
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+
+def test_nested_code_fence_six_backticks():
+    input_text = """``````markdown
+`````python
+print('hi')
+`````
+``````"""
+    expected_output = """<pre><code class=\"language-markdown\">`````python
+print('hi')
+`````
+</code></pre>"""
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+
+def test_nested_code_fence_plain_text():
+    input_text = """
+````markdown
+```
+hello
+```
+````"""
+    expected_output = """<pre><code class=\"language-markdown\">```
+hello
+```
+</code></pre>"""
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+
+
+
+
+def test_expensive_nested_code_five_fence_plain_text():
+    input_text = """
+`````markdown
+````
+```python
+print("hello world ```")
+```
+`````"""
+
+    expected_output = """<pre><code class=\"language-markdown\">````
+```python
+print("hello world ```")
+```
+</code></pre>"""
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+def test_another_expensive_nested_code_five_fence_plain_text():
+    input_text = """`````markdown
+````python
+print("hello world ```"')
+```
+`````"""
+
+    expected_output = """<pre><code class=\"language-markdown\">````python
+print("hello world ```"')
+```
+</code></pre>"""
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+def test_hard_level_nested_code_five_fence_plain_text():
+    input_text = """`````markdown
+````python
+print("hello world ```"')
+````
+`````
+```python
+print("Some another text")""" # That's where closing the second block of python code is missing.
+
+    expected_output = """<pre><code class="language-markdown">````python
+print("hello world ```"')
+````
+</code></pre>
+<pre><code class="language-python">print("Some another text")
+</code></pre>""" # But the code block is still closed correctly.
+    
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+def test_hard_level_nested_code_five_fence_plain_text_2():
+    input_text = """`````markdown
+````python
+print("hello world ```"')
+`````
+```python
+print("Some another text")""" # That's where closing the second block of python code is missing.
+
+    expected_output = """<pre><code class="language-markdown">````python
+print("hello world ```"')
+</code></pre>
+<pre><code class="language-python">print("Some another text")
+</code></pre>""" # But the code block is still closed correctly.
+    
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()
+
+def test_some_new():
+    input_text = """
+``````markdown
+`````
+````python
+print("hello world ```")
+```
+""" # Markdown code wasn't closed
+
+    expected_output = """<pre><code class=\"language-markdown\">`````
+````python
+print("hello world ```")
+```
+</code></pre>""" # But after closed correctly
+    output = telegram_format(input_text)
+    def show_output():
+      print(f"Expected was: \n\n{expected_output}\n\n")
+      print(f"output was: \n\n{output}")
+    assert output == expected_output, show_output()

chatgpt_md_converter-0.3.8/tests/test_splitter.py

@@ -0,0 +1,103 @@
+import re
+
+import pytest
+
+from chatgpt_md_converter.html_splitter import (MIN_LENGTH,
+                                                split_html_for_telegram)
+
+from . import html_examples
+
+
+def test_html_splitter():
+    chunks = split_html_for_telegram(html_examples.input_text)
+    valid_chunks = [
+        html_examples.valid_chunk_1,
+        html_examples.valid_chunk_2,
+        html_examples.valid_chunk_3,
+    ]
+    for index, chunk in enumerate(chunks):
+        assert chunk == valid_chunks[index], (
+            f"expected: \n\n{valid_chunks[index]} \n\n got: \n\n{chunk}"
+        )
+
+def test_html_splitter__remove_leading_brakes():
+    chunks = split_html_for_telegram(html_examples.input_text, trim_empty_leading_lines=True)
+    valid_chunks = [
+        html_examples.valid_chunk_1,
+        html_examples.valid_chunk_2,
+        html_examples.valid_chunk_3_remove_leading_brakes,
+    ]
+    for index, chunk in enumerate(chunks):
+        assert chunk == valid_chunks[index], (
+            f"expected: \n\n{valid_chunks[index]} \n\n got: \n\n{chunk}"
+        )
+
+def test_html_splitter_max_length_550():
+    chunks = split_html_for_telegram(
+        html_examples.long_code_input, max_length=550, trim_empty_leading_lines=True
+    )
+
+    def load_expected_chunks_550():
+        raw = re.split(r"END\n?", html_examples.expected_550)
+        chunks = []
+        for part in raw:
+            if not part.strip():
+                continue
+            lines = part.splitlines()
+            chunks.append("\n".join(lines[1:]))
+        return chunks
+
+    valid_chunks = load_expected_chunks_550()
+    for index, chunk in enumerate(chunks):
+        assert chunk == valid_chunks[index], (
+            f"expected: \n\n{valid_chunks[index]} \n\n got: \n\n{chunk}"
+        )
+        assert len(chunk) <= 550
+
+def test_split_html_respects_max_length_by_words():
+    text = "<b>" + "<i>word</i> " * 100 + "</b>"
+    chunks = split_html_for_telegram(text, max_length=550)
+    assert len(chunks) > 1
+    for chunk in chunks:
+        assert len(chunk) <= 550
+        assert chunk.startswith("<b>")
+        assert chunk.endswith("</b>")
+        assert chunk.count("<i>") == chunk.count("</i>")
+
+
+def test_split_html_only_tags_raises():
+    text = "<b></b>" * 200
+    with pytest.raises(ValueError):
+        split_html_for_telegram(text, max_length=600)
+
+
+def test_split_html_min_length_enforced():
+    with pytest.raises(ValueError):
+        split_html_for_telegram("hello", max_length=MIN_LENGTH - 1)
+
+
+def test_split_html_long_word_exceeds_limit():
+    text = "a" * 600
+    chunks = split_html_for_telegram(text, max_length=550)
+    assert chunks == ["a" * 550, "a" * 50]
+
+
+LONG_TEXT = "<b><i>" + "word " * 96 + "word!" + "</i></b>"
+
+SHORT_TEXT = "<u>" + "another " * 9 + "another" + "</u>"
+
+
+def test_split_html_keeps_newline_without_trim():
+    text = LONG_TEXT + "\n\n" + SHORT_TEXT
+    chunks = split_html_for_telegram(text, max_length=500, trim_empty_leading_lines=False)
+    assert chunks[0] == LONG_TEXT
+    assert chunks[1].startswith("\n")
+    assert chunks[1].endswith(SHORT_TEXT)
+    assert chunks[1].lstrip("\n").startswith("<u>")
+    assert chunks[1].lstrip("\n").endswith("</u>")
+
+
+def test_split_html_trims_leading_newline_on_new_chunk():
+    text = LONG_TEXT + "\n\n" + SHORT_TEXT
+    chunks = split_html_for_telegram(text, max_length=500, trim_empty_leading_lines=True)
+    assert chunks == [LONG_TEXT, SHORT_TEXT]

chatgpt_md_converter-0.3.6/chatgpt_md_converter/__init__.py

@@ -1,3 +0,0 @@
-from .telegram_formatter import telegram_format
-
-__all__ = ["telegram_format"]

chatgpt_md_converter-0.3.6/chatgpt_md_converter/extractors.py

@@ -1,61 +0,0 @@
-import re
-
-
-def ensure_closing_delimiters(text: str) -> str:
-    """
-    Ensures that if an opening ` or ``` is found without a matching closing delimiter,
-    the missing delimiter is appended to the end of the text.
-    """
-    # For triple backticks
-    if text.count("```") % 2 != 0:
-        text += "```"
-    # For single backticks
-    if text.count("`") % 2 != 0:
-        text += "`"
-    return text
-
-
-def extract_and_convert_code_blocks(text: str):
-    """
-    Extracts code blocks from the text, converting them to HTML <pre><code> format,
-    and replaces them with placeholders. Also ensures closing delimiters for unmatched blocks.
-    """
-    text = ensure_closing_delimiters(text)
-    placeholders = []
-    code_blocks = {}
-
-    def replacer(match):
-        language = match.group(1) if match.group(1) else ""
-        code_content = match.group(3)
-
-        # Properly escape HTML entities in code content
-        escaped_content = (
-            code_content.replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
-        )
-
-        placeholder = f"CODEBLOCKPLACEHOLDER{len(placeholders)}"
-        placeholders.append(placeholder)
-        if not language:
-            html_code_block = f"<pre><code>{escaped_content}</code></pre>"
-        else:
-            html_code_block = (
-                f'<pre><code class="language-{language}">{escaped_content}</code></pre>'
-            )
-        return (placeholder, html_code_block)
-
-    modified_text = text
-    for match in re.finditer(r"```(\w*)?(\n)?(.*?)```", text, flags=re.DOTALL):
-        placeholder, html_code_block = replacer(match)
-        code_blocks[placeholder] = html_code_block
-        modified_text = modified_text.replace(match.group(0), placeholder, 1)
-
-    return modified_text, code_blocks
-
-
-def reinsert_code_blocks(text: str, code_blocks: dict) -> str:
-    """
-    Reinserts HTML code blocks into the text, replacing their placeholders.
-    """
-    for placeholder, html_code_block in code_blocks.items():
-        text = text.replace(placeholder, html_code_block, 1)
-    return text