lhtml-markup 2.3.0__tar.gz → 2.4.0__tar.gz

{lhtml_markup-2.3.0/src/lhtml_markup.egg-info → lhtml_markup-2.4.0}/PKG-INFO

@@ -1,11 +1,12 @@
 Metadata-Version: 2.4
 Name: lhtml-markup
-Version: 2.3.0
+Version: 2.4.0
 Summary: Lightweight HTML markup language — simplifies HTML authoring with shorthand syntax
 License: MIT
 Project-URL: Homepage, https://github.com/drohmer/lhtml
 Project-URL: Repository, https://github.com/drohmer/lhtml
 Project-URL: Issues, https://github.com/drohmer/lhtml/issues
+Project-URL: Changelog, https://github.com/drohmer/lhtml/blob/main/CHANGELOG.md
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE.md
@@ -14,12 +15,11 @@ Requires-Dist: pygments>=2.15
 Requires-Dist: pyyaml>=6.0
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0; extra == "dev"
-Requires-Dist: ansicolors; extra == "dev"
 Dynamic: license-file
 
 # LHTML — Lightweight HTML
 
-[![Tests](https://github.com/drohmer/lhtml/actions/workflows/tests.yml/badge.svg?branch=feature/lhtml-v2)](https://github.com/drohmer/lhtml/actions/workflows/tests.yml)
+[![Tests](https://github.com/drohmer/lhtml/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/drohmer/lhtml/actions/workflows/tests.yml)
 [![PyPI](https://img.shields.io/pypi/v/lhtml-markup)](https://pypi.org/project/lhtml-markup/)
 
 LHTML is a markup language that simplifies HTML authoring with embedded CSS styling. It is designed to be **HTML-first**: raw HTML passes through untouched, and only a few shorthand symbols (`::`, `*`, `=`, `**`, `__`) trigger conversions.
@@ -65,7 +65,7 @@ lhtml a.l.html b.l.html -o build/     # Several files into a directory
 python -m lhtml input.l.html          # Alternative invocation
 ```
 
-Files are read and written as UTF-8 (a BOM is accepted). Errors and warnings are reported on stderr with the file name, the remaining files are still processed, and the exit code is non-zero if any file failed. An input file is never overwritten (e.g. `lhtml page.html` without `-o`). Includes are looked up in the input file's directory first, then in the current directory.
+Files are read and written as UTF-8 (a BOM is accepted). Errors and warnings are reported on stderr with the file name, the remaining files are still processed, and the exit code is non-zero if any file failed. No input file in the batch is overwritten (including through symbolic or hard links) (e.g. `lhtml page.html` without `-o`). Includes are looked up in the input file's directory first, then in the current directory.
 
 ### Python API
 
@@ -222,7 +222,7 @@ Output:
 
 ### Links
 
-The URL of `link::`, `img::`, `video::` and `videoplay::` is never modified (`__`, `**`, `$` are kept). Parentheses that are part of the URL are kept (`Mercury_(planet)`, `fig(1).png`), while a group starting with `.` or `#` is a class/id group.
+The URL of `link::`, `img::`, `video::` and `videoplay::` is never modified (`__`, `**`, `$` are kept). Parentheses that are part of the URL are kept (`Mercury_(planet)`, `fig(1).png`), while a group starting with `.` or `#` is a class/id group. The URL may contain Jinja expressions (`img::{{ base }}/photo.jpg`, `link::{{ url_for('page') }}[Home]`), which are kept unchanged.
 
 ```
 link::https://example.com[Click here]
@@ -429,7 +429,9 @@ All keys for the `meta` dict passed to `lhtml.run()`:
 
 ## Design Principles
 
-- **HTML-first**: Raw HTML is never modified. Only LHTML syntax triggers conversions. In particular, the following are never transformed (not even by `include::` or `::#`): HTML tags and their attributes (URLs containing `__`, quoted values containing `>`, ...), `<script>` and `<style>` blocks (CSS `::before`, ...), HTML comments, and math (`$...$`, `$$...$$`, `\(...\)`, `\[...\]`, so that `$x**2$` reaches MathJax/KaTeX intact). Text between HTML tags is still processed.
+- **HTML-first**: Raw HTML is never modified. Only LHTML syntax triggers conversions. In particular, the following are never transformed (not even by `include::` or `::#`): HTML tags and their attributes, including tags spanning multiple lines (URLs containing `__`, quoted values containing `>`, ...), `<script>` and `<style>` blocks (CSS `::before`, ...), HTML comments, and math (`$...$`, `$$...$$`, `\(...\)`, `\[...\]`, so that `$x**2$` reaches MathJax/KaTeX intact). Text between HTML tags is still processed.
+- Styles, classes/IDs and HTML attributes in LHTML tag groups are preserved without inline formatting. Link labels still support formatting.
+- Jinja2 expressions (`{{ ... }}`), statements (`{% ... %}`) and comments (`{# ... #}`) pass through unchanged.
 - **Island grammar**: LHTML syntax "islands" float in a sea of opaque content (HTML, Jinja2 templates, LaTeX, etc.) that passes through untouched.
 - **Minimal**: A few symbols (`::`, `=`, `*`, `**`, `__`, `` ` ``) cover most needs. No complex configuration required.
 - **Composable**: LHTML works seamlessly with Jinja2 templates, making it suitable for static site generators.

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0}/README.md

@@ -1,6 +1,6 @@
 # LHTML — Lightweight HTML
 
-[![Tests](https://github.com/drohmer/lhtml/actions/workflows/tests.yml/badge.svg?branch=feature/lhtml-v2)](https://github.com/drohmer/lhtml/actions/workflows/tests.yml)
+[![Tests](https://github.com/drohmer/lhtml/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/drohmer/lhtml/actions/workflows/tests.yml)
 [![PyPI](https://img.shields.io/pypi/v/lhtml-markup)](https://pypi.org/project/lhtml-markup/)
 
 LHTML is a markup language that simplifies HTML authoring with embedded CSS styling. It is designed to be **HTML-first**: raw HTML passes through untouched, and only a few shorthand symbols (`::`, `*`, `=`, `**`, `__`) trigger conversions.
@@ -46,7 +46,7 @@ lhtml a.l.html b.l.html -o build/     # Several files into a directory
 python -m lhtml input.l.html          # Alternative invocation
 ```
 
-Files are read and written as UTF-8 (a BOM is accepted). Errors and warnings are reported on stderr with the file name, the remaining files are still processed, and the exit code is non-zero if any file failed. An input file is never overwritten (e.g. `lhtml page.html` without `-o`). Includes are looked up in the input file's directory first, then in the current directory.
+Files are read and written as UTF-8 (a BOM is accepted). Errors and warnings are reported on stderr with the file name, the remaining files are still processed, and the exit code is non-zero if any file failed. No input file in the batch is overwritten (including through symbolic or hard links) (e.g. `lhtml page.html` without `-o`). Includes are looked up in the input file's directory first, then in the current directory.
 
 ### Python API
 
@@ -203,7 +203,7 @@ Output:
 
 ### Links
 
-The URL of `link::`, `img::`, `video::` and `videoplay::` is never modified (`__`, `**`, `$` are kept). Parentheses that are part of the URL are kept (`Mercury_(planet)`, `fig(1).png`), while a group starting with `.` or `#` is a class/id group.
+The URL of `link::`, `img::`, `video::` and `videoplay::` is never modified (`__`, `**`, `$` are kept). Parentheses that are part of the URL are kept (`Mercury_(planet)`, `fig(1).png`), while a group starting with `.` or `#` is a class/id group. The URL may contain Jinja expressions (`img::{{ base }}/photo.jpg`, `link::{{ url_for('page') }}[Home]`), which are kept unchanged.
 
 ```
 link::https://example.com[Click here]
@@ -410,7 +410,9 @@ All keys for the `meta` dict passed to `lhtml.run()`:
 
 ## Design Principles
 
-- **HTML-first**: Raw HTML is never modified. Only LHTML syntax triggers conversions. In particular, the following are never transformed (not even by `include::` or `::#`): HTML tags and their attributes (URLs containing `__`, quoted values containing `>`, ...), `<script>` and `<style>` blocks (CSS `::before`, ...), HTML comments, and math (`$...$`, `$$...$$`, `\(...\)`, `\[...\]`, so that `$x**2$` reaches MathJax/KaTeX intact). Text between HTML tags is still processed.
+- **HTML-first**: Raw HTML is never modified. Only LHTML syntax triggers conversions. In particular, the following are never transformed (not even by `include::` or `::#`): HTML tags and their attributes, including tags spanning multiple lines (URLs containing `__`, quoted values containing `>`, ...), `<script>` and `<style>` blocks (CSS `::before`, ...), HTML comments, and math (`$...$`, `$$...$$`, `\(...\)`, `\[...\]`, so that `$x**2$` reaches MathJax/KaTeX intact). Text between HTML tags is still processed.
+- Styles, classes/IDs and HTML attributes in LHTML tag groups are preserved without inline formatting. Link labels still support formatting.
+- Jinja2 expressions (`{{ ... }}`), statements (`{% ... %}`) and comments (`{# ... #}`) pass through unchanged.
 - **Island grammar**: LHTML syntax "islands" float in a sea of opaque content (HTML, Jinja2 templates, LaTeX, etc.) that passes through untouched.
 - **Minimal**: A few symbols (`::`, `=`, `*`, `**`, `__`, `` ` ``) cover most needs. No complex configuration required.
 - **Composable**: LHTML works seamlessly with Jinja2 templates, making it suitable for static site generators.

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "lhtml-markup"
-version = "2.3.0"
+version = "2.4.0"
 description = "Lightweight HTML markup language — simplifies HTML authoring with shorthand syntax"
 readme = "README.md"
 license = {text = "MIT"}
@@ -19,11 +19,11 @@ dependencies = [
 Homepage = "https://github.com/drohmer/lhtml"
 Repository = "https://github.com/drohmer/lhtml"
 Issues = "https://github.com/drohmer/lhtml/issues"
+Changelog = "https://github.com/drohmer/lhtml/blob/main/CHANGELOG.md"
 
 [project.optional-dependencies]
 dev = [
     "pytest>=7.0",
-    "ansicolors",
 ]
 
 [project.scripts]

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0}/src/lhtml/cli.py

@@ -118,7 +118,10 @@ def main():
                 continue
 
             out_path = _output_path_for(f_in, args.output)
-            if os.path.abspath(out_path) == os.path.abspath(f_in):
+            if any(os.path.realpath(out_path) == os.path.realpath(source)
+                   or (os.path.exists(out_path) and os.path.exists(source)
+                       and os.path.samefile(out_path, source))
+                   for source in args.inputFiles):
                 raise ValueError(f'output file would overwrite the input file [{out_path}]')
             out_dir = os.path.dirname(out_path)
             if out_dir:

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0}/src/lhtml/export_html.py

@@ -9,6 +9,8 @@ import html
 import os
 import re
 
+from .patterns import JINJA_RE
+
 
 # ---------------------------------------------------------------------------
 # Attribute helpers
@@ -24,8 +26,15 @@ def _build_attrs(elements):
 
 
 def _attr(value):
-    """Escape the double quotes of a value placed inside a double-quoted HTML attribute."""
-    return value.replace('"', '"')
+    """Escape the double quotes of a value placed inside a double-quoted HTML
+    attribute (Jinja zones are kept as is, they are rendered before HTML)."""
+    parts, prev = [], 0
+    for m in JINJA_RE.finditer(value):
+        parts.append(value[prev:m.start()].replace('"', '"'))
+        parts.append(m.group(0))
+        prev = m.end()
+    parts.append(value[prev:].replace('"', '"'))
+    return ''.join(parts)
 
 
 def export_html_element_style(text):
@@ -115,7 +124,7 @@ def export_html_video(elements, default_inline='', current_directory=''):
     parts = ['<video']
     parts.append(export_html_element_inline(elements['{}']))
     if default_inline:
-        parts.append(f' {default_inline} ')
+        parts.append(f' {default_inline}')
     parts.append(export_html_element_class_and_id(elements['()']))
     parts.append(export_html_element_style(elements['[]']))
     if os.path.isfile(os.path.join(current_directory, poster_candidate)):

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0}/src/lhtml/patterns.py

@@ -49,16 +49,29 @@ MATH             = re.compile(
     r'|(?<![\\$\w])\$(?![\s$])[^$\n]*?[^\s\\$]\$(?![\w$])'  # $inline$ (pandoc-like rule)
     r'|(?<![\\$\w])\$[^\s\\$]\$(?![\w$])',                    # $x$ (single char)
     re.DOTALL)
-# An HTML tag: a name followed by attributes on the same line; quoted
-# values may contain '<' or '>'.
-HTML_TAG         = re.compile(r'</?[A-Za-z][\w:-]*(?=[\s/>])(?:[^<>"\'\n]|"[^"\n]*"|\'[^\'\n]*\')*>')
+# HTML attributes may span lines; quoted values may contain < or >.
+_HTML_ATTRIBUTE = r"[A-Za-z_:][\w:.-]*(?:\s*=\s*(?:\"[^\"]*\"|'[^']*'|[^\s<>\"'=\x60]+))?"
+HTML_TAG = re.compile(r'</?[A-Za-z][\w:-]*(?:\s+' + _HTML_ATTRIBUTE + r')*\s*/?>')
 RAW_HTML_BLOCK   = re.compile(r'<!--.*?-->|<(?i:(script|style))\b[^>]*>.*?</(?i:\1)\s*>', re.DOTALL)
 # URL of link::, img::, video::, videoplay:: (parentheses are kept when they
 # are not a (.class #id) group, e.g. Mercury_(planet) or fig(1).png)
+# Quoted strings can contain the Jinja closing delimiter itself. A quote can
+# only start a string (otherwise an unclosed {{ backtracks exponentially),
+# and a new opening delimiter ends the search (an unclosed {{ does not scan
+# the rest of the document).
+_JINJA_STRING = r"(?:\"(?:\\.|[^\"\\])*\"|'(?:\\.|[^'\\])*')"
+_JINJA_EXPRESSION = r'\{\{(?:' + _JINJA_STRING + r'|(?!\}\}|\{\{)[^"\'])*?\}\}'
+JINJA = (r'\{#.*?#\}'
+         r'|' + _JINJA_EXPRESSION +
+         r'|\{%(?:' + _JINJA_STRING + r'|(?!%\}|\{%)[^"\'])*?%\}')
+JINJA_RE = re.compile(JINJA, re.DOTALL)
 URL_TAGS         = ('link', 'img', 'video', 'videoplay')
-URL_TOKEN        = r'(?:[^\s\[\](){}<>"`\x00]|\((?![.#])[^\s()\[\]{}<>"`\x00]*\))+'
+# A URL may contain Jinja expressions (img::{{ base }}/photo.jpg)
+URL_TOKEN        = (r'(?:' + _JINJA_EXPRESSION +
+                    r'|[^\s\[\](){}<>"`\x00]|\((?![.#])[^\s()\[\]{}<>"`\x00]*\))+')
 PROTECTED        = re.compile(
-    r'(?P<comment><!--.*?-->)'
+    r'(?P<jinja>' + JINJA + r')'
+    r'|(?P<comment><!--.*?-->)'
     r'|(?P<raw><(?i:(?P<rawtag>script|style))\b[^>]*>.*?</(?i:(?P=rawtag))\s*>)'
     r'|`(?P<icode>[^`\n]*)`'
     r'|(?P<urltag>(?<![\w-])(?:' + '|'.join(URL_TAGS) + r')::)(?P<url>' + URL_TOKEN + r')'
@@ -66,6 +79,9 @@ PROTECTED        = re.compile(
     r'|(?P<tag>' + HTML_TAG.pattern + r')',
     re.DOTALL)
 
+# One left-to-right pass: an outer protected zone owns its contents.
+PROTECTED_BLOCKS = re.compile(BLOCKS.pattern + '|' + PROTECTED.pattern, re.DOTALL)
+
 # Line breaks option: elements that make a line part of the structure
 BLOCK_TAGS       = frozenset((
     'div p h1 h2 h3 h4 h5 h6 ul ol li dl dt dd table thead tbody tfoot tr td th '
@@ -75,9 +91,9 @@ LEADING_TAG      = re.compile(r'^\s*</?([A-Za-z][\w-]*)')
 TRAILING_TAG     = re.compile(r'</?([A-Za-z][\w-]*)(?:[^<>"\']|"[^"]*"|\'[^\']*\')*/?>\s*$')
 LEADING_PLACEHOLDER  = re.compile(r'^\s*\x00([A-Z])(\d+)\x00')
 TRAILING_PLACEHOLDER = re.compile(r'\x00([A-Z])(\d+)\x00\s*$')
-JINJA_LINE       = re.compile(r'\s*(\{%.*%\}|\{#.*#\})\s*')
+JINJA_LINE       = re.compile(r'\s*(\{%.*%\}|\{#.*#\})\s*', re.DOTALL)
 # <pre> and <textarea> keep their line breaks: no <br> inside
-PREFORMATTED_TAG = re.compile(r'<(/?)(?i:pre|textarea)\b')
+PREFORMATTED_TAG = re.compile(r'<(/?)(?i:pre|textarea)(?=[\s/>])')
 BLOCK_RAW        = re.compile(r'\s*(<!--|<(?i:script|style)\b|\$\$|\\\[)')
 
 # Placeholders for protected content. They contain no LHTML syntax

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0}/src/lhtml/pipeline.py

@@ -19,12 +19,9 @@ from typing import Callable
 
 META_DEFAULTS = {
     'wrap-auto': False,
-    'add_title_id': False,
     'title': 'Webpage',
     'css': [],
     'js': [],
-    'wrap-custom-pre': '',
-    'wrap-custom-post': '',
     'line-breaks': False,
 }
 
@@ -187,7 +184,7 @@ class ProcessingPipeline:
         ctx.text = process_include_recursive(ctx.text, ctx.meta['directory_include'], stores)
 
         # Phase 3: Block-level elements
-        ctx.text = process_title(ctx.text)
+        ctx.text = process_title(ctx.text, stores)
         ctx.text = process_listing(ctx.text)
 
         # Phase 4: Inline elements
@@ -196,11 +193,11 @@ class ProcessingPipeline:
 
         # Phase 5: Tag elements (uses tag_registry). Inside inline code,
         # only named tags (e.g. link::) are processed.
-        def resolve_urls(s):
-            return stores.restore(s, 'U')
+        def resolve_attributes(s):
+            return stores.restore(stores.restore(s, 'A'), 'U')
 
         ctx.text = process_tag(ctx.text, ctx.current_directory, self.tag_registry,
-                               resolve=resolve_urls)
+                               resolve=resolve_attributes)
         ctx.text = stores.restore(ctx.text, 'I', lambda content: render_inline_code(
             process_tag(content, ctx.current_directory, self.tag_registry, inline=True)))
 

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0}/src/lhtml/process.py

@@ -28,7 +28,7 @@ from .errors import (
     LHTMLParseError, LHTMLWarning,
 )
 from .patterns import (
-    YAML_FRONTMATTER, VERBATIM_BLOCK, CODE_BLOCK, BLOCKS, PROTECTED,
+    YAML_FRONTMATTER, VERBATIM_BLOCK, CODE_BLOCK, PROTECTED, PROTECTED_BLOCKS,
     HEADING, BOLD, ITALIC, INLINE_CODE,
     COMMENT, INCLUDE, TAG_MARKER, SPACER,
     BLOCK_TAGS, LEADING_TAG, TRAILING_TAG, LEADING_PLACEHOLDER, TRAILING_PLACEHOLDER,
@@ -46,12 +46,14 @@ class ProtectionStores:
     R: raw zones: HTML tags, comments, <script>/<style>, math
     I: inline code (content with < and > escaped)
     U: URL of link::/img::/video::/videoplay:: tags
+    A: LHTML styles, classes/IDs and HTML attributes
     """
     V: list = field(default_factory=list)
     C: list = field(default_factory=list)
     R: list = field(default_factory=list)
     I: list = field(default_factory=list)
     U: list = field(default_factory=list)
+    A: list = field(default_factory=list)  # LHTML attribute group contents
 
     def add(self, kind, entry):
         store = getattr(self, kind)
@@ -109,24 +111,6 @@ def process_yaml(text):
 # Verbatim and code blocks (protect / restore)
 # ---------------------------------------------------------------------------
 
-def process_blocks_to_index(text, stores, directories=None, _stack=()):
-    """Replace verbatim:: and code:: blocks with placeholders.
-
-    The leftmost block wins: verbatim markers inside a code block are
-    displayed as code, and a code block inside verbatim stays raw.
-    If `directories` is given, include:: directives inside code blocks
-    are expanded (the included files are inserted as raw code).
-    """
-    def _store(m):
-        if m.group('verbatim') is not None:
-            return stores.add('V', m.group('vbody'))
-        body = m.group('body')
-        if directories is not None:
-            body = _expand_includes_raw(body, directories, _stack)
-        return stores.add('C', (m.group('header'), body))
-    return regex_transform(text, BLOCKS, _store)
-
-
 def process_verbatim_to_index(text, verbatim_index_store):
     """Replace verbatim blocks with placeholders."""
     return store_to_index(text, VERBATIM_BLOCK, 'V', verbatim_index_store,
@@ -165,26 +149,82 @@ def render_inline_code(content):
     return f'<code class="code-inline">{content}</code>'
 
 
-def process_protect(text, stores):
+def process_protect(text, stores, directories=None, _stack=()):
     """Replace zones that LHTML must not transform with placeholders.
 
     Protected (the leftmost zone wins): HTML comments, <script>/<style>
     blocks, inline code, URLs of link::/img::/video:: tags, math
     ($...$, $$...$$, \\(...\\), \\[...\\]) and HTML tags themselves (so
     attribute values are never modified). Text between HTML tags is
-    still processed.
+    still processed. Jinja expressions, statements and comments are opaque.
+    With directories supplied, code/verbatim blocks participate in the same
+    left-to-right scan and code includes use those directories.
     """
     def _protect(m):
+        if directories is not None:
+            if m.group('verbatim') is not None:
+                return stores.add('V', m.group('vbody'))
+            if m.group('code') is not None:
+                body = _expand_includes_raw(m.group('body'), directories, _stack)
+                return stores.add('C', (m.group('header'), body))
         if m.group('icode') is not None:
             return stores.add('I', escape_inline_code(m.group('icode')))
         if m.group('url') is not None:
             return m.group('urltag') + stores.add('U', m.group('url'))
         return stores.add('R', m.group(0))
-    return regex_transform(text, PROTECTED, _protect)
+    pattern = PROTECTED if directories is None else PROTECTED_BLOCKS
+    return regex_transform(text, pattern, _protect)
+
+
+def process_protect_attributes(text, stores):
+    """Hide attribute groups from formatting, keeping link labels active."""
+    def _heading(m):
+        if not m.group(2):
+            return m.group(0)
+        return f'{m.group(1)}({stores.add("A", m.group(2))}) {m.group(3)}'
+    text = regex_transform(text, HEADING, _heading)
+
+    parts, prev = [], 0
+    for m in TAG_MARKER.finditer(text):
+        if m.start() < prev:
+            continue
+        try:
+            element = extract_bracket_elements(text, m.end())
+        except LHTMLParseError:
+            continue
+        tag = element['tag']
+        if tag in ('include', 'code', 'verbatim') or not _is_tag_boundary(
+                text, m.start() - len(tag), tag):
+            continue
+        end = element['index_end']
+        pos = m.end()
+        while pos < end:
+            opening = text[pos]
+            if opening not in '([{':
+                pos += 1
+                continue
+            closing = {'(': ')', '[': ']', '{': '}'}[opening]
+            start = pos
+            depth = 1
+            pos += 1
+            while pos < end and depth:
+                if text[pos] == opening:
+                    depth += 1
+                elif text[pos] == closing:
+                    depth -= 1
+                pos += 1
+            if depth or (tag == 'link' and opening == '['):
+                continue
+            parts.append(text[prev:start + 1])
+            parts.append(stores.add('A', text[start + 1:pos - 1]))
+            prev = pos - 1
+    parts.append(text[prev:])
+    return ''.join(parts)
 
 
 def process_unprotect(text, stores):
     """Restore the raw zones and URLs protected by process_protect."""
+    text = stores.restore(text, 'A')
     text = stores.restore(text, 'R')
     return stores.restore(text, 'U')
 
@@ -195,7 +235,7 @@ def process_unprotect(text, stores):
 
 def process_remove_comment(text):
     """Remove ::# comments (to the end of the line)."""
-    return regex_transform(text, COMMENT, lambda m: '\n')
+    return regex_transform(text, COMMENT, lambda m: '')
 
 
 # ---------------------------------------------------------------------------
@@ -270,8 +310,8 @@ def process_include_recursive(text, directories, stores, _stack=()):
     its own directory, then in `directories`.
     Raises LHTMLIncludeLoopError on circular or too deep inclusion.
     """
-    text = process_blocks_to_index(text, stores, directories, _stack)
-    text = process_protect(text, stores)
+    text = process_protect(text, stores, directories, _stack)
+    text = process_protect_attributes(text, stores)
     text = process_remove_comment(text)
 
     parts = []
@@ -318,14 +358,24 @@ def process_code_inline(text):
 # Headings
 # ---------------------------------------------------------------------------
 
-def process_title(text):
-    """Convert = Title or =(.class) Title to <h1>Title</h1>, etc."""
+def process_title(text, stores=None):
+    """Convert = Title or =(.class) Title to <h1>Title</h1>, etc.
+
+    With stores, the (.class #id) group protected by process_protect_attributes
+    is resolved, and the generated attributes stay protected from inline
+    formatting until process_unprotect.
+    """
     def _heading(m):
         level = str(len(m.group(1)))
-        class_id = (m.group(2) or '').strip()
+        class_id = m.group(2) or ''
+        if stores is not None:
+            class_id = stores.restore(class_id, 'A')
+        class_id = class_id.strip()
         title = m.group(3)
         if class_id:
             attrs = export_html_element_class_and_id(class_id)
+            if stores is not None and attrs:
+                attrs = stores.add('A', attrs)
             return f'<h{level}{attrs}>{title}</h{level}>\n'
         return f'<h{level}>{title}</h{level}>\n'
     return regex_transform(text, HEADING, _heading)
@@ -483,7 +533,10 @@ def process_tag(text, current_directory='', registry=None, inline=False, resolve
             continue
         if not tag and (inline or not _adjust_bare_tag(text, m, element)):
             continue
-        element['context'] = _context(text, tag_start, element['index_end'])
+        context = text[tag_start:element['index_end']]
+        if resolve is not None:
+            context = resolve(context)
+        element['context'] = _context(context, 0, len(context))
         if resolve is not None:
             for key in ('text', '[]', '()', '{}'):
                 element[key] = resolve(element[key])
@@ -520,7 +573,7 @@ def _is_block_placeholder(kind, idx, stores):
     if kind != 'R':
         return False
     entry = stores.R[idx]
-    if BLOCK_RAW.match(entry):
+    if BLOCK_RAW.match(entry) or JINJA_LINE.fullmatch(entry):
         return True
     m = LEADING_TAG.match(entry)
     return bool(m) and m.group(1).lower() in BLOCK_TAGS
@@ -541,11 +594,20 @@ def _is_structure_line(line, stores):
 
 
 def _preformatted_depth_change(line, stores):
-    """Net number of <pre>/<textarea> elements opened by the line (raw HTML
-    tags are still placeholders, so they are resolved)."""
-    resolved = PLACEHOLDER.sub(
-        lambda m: stores.R[int(m.group(2))] if m.group(1) == 'R' else '', line)
-    return sum(-1 if m.group(1) else 1 for m in PREFORMATTED_TAG.finditer(resolved))
+    """Count actual preformatted tags, never tag-like text inside raw zones."""
+    tags = []
+    for placeholder in PLACEHOLDER.finditer(line):
+        if placeholder.group(1) == 'R':
+            entry = stores.R[int(placeholder.group(2))]
+            # An actual tag starts at the beginning of its stored zone.
+            # Do not scan inside comments, scripts, attributes or Jinja.
+            tags.append(PREFORMATTED_TAG.match(entry))
+    # Tags generated by handlers (and direct calls without stores) are
+    # not placeholders. Tokenize them so quoted attributes stay opaque.
+    for zone in PROTECTED.finditer(line):
+        if zone.group('tag') is not None:
+            tags.append(PREFORMATTED_TAG.match(zone.group('tag')))
+    return sum(-1 if tag.group(1) else 1 for tag in tags if tag is not None)
 
 
 def process_line_breaks(text, stores=None):

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0/src/lhtml_markup.egg-info}/PKG-INFO

@@ -1,11 +1,12 @@
 Metadata-Version: 2.4
 Name: lhtml-markup
-Version: 2.3.0
+Version: 2.4.0
 Summary: Lightweight HTML markup language — simplifies HTML authoring with shorthand syntax
 License: MIT
 Project-URL: Homepage, https://github.com/drohmer/lhtml
 Project-URL: Repository, https://github.com/drohmer/lhtml
 Project-URL: Issues, https://github.com/drohmer/lhtml/issues
+Project-URL: Changelog, https://github.com/drohmer/lhtml/blob/main/CHANGELOG.md
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE.md
@@ -14,12 +15,11 @@ Requires-Dist: pygments>=2.15
 Requires-Dist: pyyaml>=6.0
 Provides-Extra: dev
 Requires-Dist: pytest>=7.0; extra == "dev"
-Requires-Dist: ansicolors; extra == "dev"
 Dynamic: license-file
 
 # LHTML — Lightweight HTML
 
-[![Tests](https://github.com/drohmer/lhtml/actions/workflows/tests.yml/badge.svg?branch=feature/lhtml-v2)](https://github.com/drohmer/lhtml/actions/workflows/tests.yml)
+[![Tests](https://github.com/drohmer/lhtml/actions/workflows/tests.yml/badge.svg?branch=main)](https://github.com/drohmer/lhtml/actions/workflows/tests.yml)
 [![PyPI](https://img.shields.io/pypi/v/lhtml-markup)](https://pypi.org/project/lhtml-markup/)
 
 LHTML is a markup language that simplifies HTML authoring with embedded CSS styling. It is designed to be **HTML-first**: raw HTML passes through untouched, and only a few shorthand symbols (`::`, `*`, `=`, `**`, `__`) trigger conversions.
@@ -65,7 +65,7 @@ lhtml a.l.html b.l.html -o build/     # Several files into a directory
 python -m lhtml input.l.html          # Alternative invocation
 ```
 
-Files are read and written as UTF-8 (a BOM is accepted). Errors and warnings are reported on stderr with the file name, the remaining files are still processed, and the exit code is non-zero if any file failed. An input file is never overwritten (e.g. `lhtml page.html` without `-o`). Includes are looked up in the input file's directory first, then in the current directory.
+Files are read and written as UTF-8 (a BOM is accepted). Errors and warnings are reported on stderr with the file name, the remaining files are still processed, and the exit code is non-zero if any file failed. No input file in the batch is overwritten (including through symbolic or hard links) (e.g. `lhtml page.html` without `-o`). Includes are looked up in the input file's directory first, then in the current directory.
 
 ### Python API
 
@@ -222,7 +222,7 @@ Output:
 
 ### Links
 
-The URL of `link::`, `img::`, `video::` and `videoplay::` is never modified (`__`, `**`, `$` are kept). Parentheses that are part of the URL are kept (`Mercury_(planet)`, `fig(1).png`), while a group starting with `.` or `#` is a class/id group.
+The URL of `link::`, `img::`, `video::` and `videoplay::` is never modified (`__`, `**`, `$` are kept). Parentheses that are part of the URL are kept (`Mercury_(planet)`, `fig(1).png`), while a group starting with `.` or `#` is a class/id group. The URL may contain Jinja expressions (`img::{{ base }}/photo.jpg`, `link::{{ url_for('page') }}[Home]`), which are kept unchanged.
 
 ```
 link::https://example.com[Click here]
@@ -429,7 +429,9 @@ All keys for the `meta` dict passed to `lhtml.run()`:
 
 ## Design Principles
 
-- **HTML-first**: Raw HTML is never modified. Only LHTML syntax triggers conversions. In particular, the following are never transformed (not even by `include::` or `::#`): HTML tags and their attributes (URLs containing `__`, quoted values containing `>`, ...), `<script>` and `<style>` blocks (CSS `::before`, ...), HTML comments, and math (`$...$`, `$$...$$`, `\(...\)`, `\[...\]`, so that `$x**2$` reaches MathJax/KaTeX intact). Text between HTML tags is still processed.
+- **HTML-first**: Raw HTML is never modified. Only LHTML syntax triggers conversions. In particular, the following are never transformed (not even by `include::` or `::#`): HTML tags and their attributes, including tags spanning multiple lines (URLs containing `__`, quoted values containing `>`, ...), `<script>` and `<style>` blocks (CSS `::before`, ...), HTML comments, and math (`$...$`, `$$...$$`, `\(...\)`, `\[...\]`, so that `$x**2$` reaches MathJax/KaTeX intact). Text between HTML tags is still processed.
+- Styles, classes/IDs and HTML attributes in LHTML tag groups are preserved without inline formatting. Link labels still support formatting.
+- Jinja2 expressions (`{{ ... }}`), statements (`{% ... %}`) and comments (`{# ... #}`) pass through unchanged.
 - **Island grammar**: LHTML syntax "islands" float in a sea of opaque content (HTML, Jinja2 templates, LaTeX, etc.) that passes through untouched.
 - **Minimal**: A few symbols (`::`, `=`, `*`, `**`, `__`, `` ` ``) cover most needs. No complex configuration required.
 - **Composable**: LHTML works seamlessly with Jinja2 templates, making it suitable for static site generators.

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0}/src/lhtml_markup.egg-info/requires.txt

@@ -4,4 +4,3 @@ pyyaml>=6.0
 
 [dev]
 pytest>=7.0
-ansicolors

{lhtml_markup-2.3.0 → lhtml_markup-2.4.0}/test/test_lhtml.py

@@ -11,6 +11,7 @@ Tests cover:
 import os
 import re
 import glob
+import time
 import warnings
 
 import pytest
@@ -1118,3 +1119,214 @@ class TestLineBreaks:
         for flag in ('-b', '--line-breaks'):
             code, out, _ = TestCli._run_cli(self, monkeypatch, capsys, flag, str(f))
             assert code == 0 and out == 'a<br>\nb\n'
+
+
+class TestAuditRegressions:
+    @pytest.mark.parametrize('source', [
+        '<script>const s = "code::[]include::missing.txt code::[-]";</script>',
+        '<style>p::before {content:"verbatim::[]**x**verbatim::[-]"}</style>',
+        '<!-- code::[]include::missing.txt code::[-] -->',
+        '<a title="code::[]include::missing.txt code::[-]">**label**</a>',
+        '$code::[]include::missing.txt code::[-]$',
+    ])
+    def test_outer_zone_owns_block_directives(self, source):
+        assert lhtml.run(source) == source.replace('**label**', '<strong>label</strong>')
+
+    def test_inline_code_owns_block_directive_after_text(self):
+        source = '`example code::[]include::missing.txt code::[-]`'
+        assert lhtml.run(source) == '<code class="code-inline">' + source[1:-1] + '</code>'
+
+    @pytest.mark.parametrize('source', [
+        '<a\n href="page__draft__.html">**label**</a>',
+        '<a title="first\n**second**"\n href="x">**label**</a>',
+        '<input\n disabled\n data-path="__draft__">',
+    ])
+    def test_multiline_html_attributes(self, source):
+        assert lhtml.run(source) == source.replace('**label**', '<strong>label</strong>')
+
+    @pytest.mark.parametrize('source', [
+        '{{ "page__draft__.html" }}',
+        '{{ user.__class__.__name__ }}',
+        '{% set path = "page__draft__.html" %}',
+        '{# code::[]include::missing.txt code::[-] #}',
+        '{{ "}} **literal**" }}',
+        '{% set text = "%} __literal__" %}',
+        '{{\n "__literal__"\n }}',
+    ])
+    def test_jinja_preserved_with_surrounding_formatting(self, source):
+        assert lhtml.run('**before** ' + source + ' __after__') == (
+            '<strong>before</strong> ' + source + ' <em>after</em>')
+
+    @pytest.mark.parametrize('opening', ['{{', '{%'])
+    def test_unclosed_jinja_with_quotes_is_fast(self, opening):
+        source = opening + ' ' + 'il dit "oui" puis "non ' * 200 + '**b**'
+        start = time.perf_counter()
+        result = lhtml.run(source)
+        assert time.perf_counter() - start < 1
+        assert result.endswith('<strong>b</strong>')
+
+    def test_styles_classes_and_inline_attributes_are_not_formatted(self):
+        source = ('div::(.my__class__)[background:url(photo__small__x.png)]'
+                  '{data-path="a__b__"} **content** ::')
+        assert lhtml.run(source) == (
+            '<div class="my__class__" style="background:url(photo__small__x.png)"'
+            ' data-path="a__b__"> <strong>content</strong> </div>')
+
+    def test_link_label_keeps_formatting_and_inline_code(self):
+        assert lhtml.run('link::page__draft__.html(.my__class__)[**bold** `code`]') == (
+            '<a class="my__class__" href="page__draft__.html">'
+            '<strong>bold</strong> <code class="code-inline">code</code></a>')
+
+    def test_attribute_quotes_are_escaped_after_restoring(self):
+        assert lhtml.run('div::[font-family:"__font__"] x ::') == (
+            '<div style="font-family:&quot;__font__&quot;"> x </div>')
+
+    def test_protection_applies_in_includes(self, tmp_path):
+        source = '<a\n href="__draft__.html">{{ "__value__" }}</a>'
+        (tmp_path / 'part.html').write_text(source)
+        assert lhtml.run('include::part.html', {'directory_include': [tmp_path]}) == source
+
+
+class TestCliSourceProtection:
+    _run_cli = TestCli._run_cli
+
+    def test_batch_cannot_overwrite_another_input(self, tmp_path, monkeypatch, capsys):
+        first, second, third = [tmp_path / name for name in
+                                ('page.l.html', 'page.html', 'other.l.html')]
+        first.write_text('= First\n')
+        second.write_text('Original content\n')
+        third.write_text('= Other\n')
+        code, _, err = self._run_cli(monkeypatch, capsys, str(first), str(second), str(third))
+        assert code == 1 and 'overwrite' in err
+        assert first.read_text() == '= First\n'
+        assert second.read_text() == 'Original content\n'
+        assert '<h1>Other</h1>' in (tmp_path / 'other.html').read_text()
+
+    @pytest.mark.parametrize('alias_kind', ['symlink', 'hardlink'])
+    def test_output_alias_cannot_overwrite_source(self, tmp_path, monkeypatch, capsys, alias_kind):
+        source, destination = tmp_path / 'page.l.html', tmp_path / 'output.html'
+        source.write_text('= Original\n')
+        if alias_kind == 'symlink':
+            destination.symlink_to(source)
+        else:
+            os.link(source, destination)
+        code, _, err = self._run_cli(monkeypatch, capsys, str(source), '-o', str(destination))
+        assert code == 1 and 'overwrite' in err
+        assert source.read_text() == '= Original\n'
+
+
+@pytest.mark.parametrize('statement', ['{% if visible %}', '{# comment #}'])
+def test_jinja_statement_does_not_introduce_line_breaks(statement):
+    source = 'before\n' + statement + '\nafter'
+    assert lhtml.run(source, {'line-breaks': True}) == source
+
+
+class TestLineBreakRegressions:
+    ON = {'line-breaks': True}
+
+    @pytest.mark.parametrize('tag', ['pre', 'textarea'])
+    @pytest.mark.parametrize('wrapper', [
+        '<script>const s = "<{tag}>";</script>',
+        '<!-- <{tag}> -->',
+        '<span title="<{tag}>">label</span>',
+        '{{ "<{tag}>" }}',
+    ])
+    def test_fake_preformatted_tag_does_not_disable_following_breaks(self, tag, wrapper):
+        prefix = wrapper.replace('{tag}', tag)
+        result = lhtml.run(prefix + '\none\ntwo', self.ON)
+        assert result.startswith(prefix)
+        assert result.endswith('one<br>\ntwo')
+
+    @pytest.mark.parametrize('tag', ['pre-view', 'textarea-widget'])
+    def test_custom_element_is_not_preformatted(self, tag):
+        source = f'<{tag}>one\ntwo</{tag}>\nthree\nfour'
+        assert lhtml.run(source, self.ON) == (
+            f'<{tag}>one<br>\ntwo</{tag}><br>\nthree<br>\nfour')
+
+    @pytest.mark.parametrize('tag', ['pre', 'textarea', 'PRE', 'TEXTAREA'])
+    def test_real_preformatted_block_still_preserves_newlines(self, tag):
+        source = f'<{tag}\n class="sample">\none\ntwo\n</{tag}>\nthree\nfour'
+        assert lhtml.run(source, self.ON) == source.replace('three\nfour', 'three<br>\nfour')
+
+    def test_fake_close_in_comment_does_not_end_preformatted_block(self):
+        source = '<pre>\n<!-- </pre> -->\none\ntwo\n</pre>\nthree\nfour'
+        assert lhtml.run(source, self.ON) == source.replace('three\nfour', 'three<br>\nfour')
+
+    @pytest.mark.parametrize('source, expected', [
+        ('one ::# note\ntwo', 'one <br>\ntwo'),
+        ('one ::# note\n\ntwo', 'one <br>\n<br>\ntwo'),
+        ('one ::# note', 'one '),
+        ('one\n::# note\ntwo', 'one<br>\n<br>\ntwo'),
+    ])
+    def test_comment_removal_keeps_only_existing_newlines(self, source, expected):
+        assert lhtml.run(source, self.ON) == expected
+
+    @pytest.mark.parametrize('statement', [
+        '{%\n if visible\n%}',
+        '{#\n a comment\n#}',
+        '{%-\n set value = "text"\n-%}',
+    ])
+    def test_multiline_jinja_statement_is_structure(self, statement):
+        source = 'one\ntwo\n' + statement + '\nthree\nfour'
+        expected = 'one<br>\ntwo\n' + statement + '\nthree<br>\nfour'
+        assert lhtml.run(source, self.ON) == expected
+
+    def test_multiline_jinja_expression_remains_inline(self):
+        source = 'one {{\n value\n}}\ntwo'
+        assert lhtml.run(source, self.ON) == 'one {{\n value\n}}<br>\ntwo'
+
+
+class TestJinjaInUrlsAndHeadings:
+    @pytest.mark.parametrize('source, expected', [
+        ('img::{{ url }}', '<img src="{{ url }}" alt="{{ url }}">'),
+        ('img::{{ base }}/photo__small__.jpg[width:10px]',
+         '<img style="width:10px" src="{{ base }}/photo__small__.jpg"'
+         ' alt="{{ base }}/photo__small__.jpg">'),
+        ('link::{{ url }}[**go**]', '<a href="{{ url }}"><strong>go</strong></a>'),
+        ('link::/posts/{{ post.slug }}.html(.nav)[x]',
+         '<a class="nav" href="/posts/{{ post.slug }}.html">x</a>'),
+        ("link::{{ url_for('page', name='a b') }}[x]",
+         "<a href=\"{{ url_for('page', name='a b') }}\">x</a>"),
+    ])
+    def test_jinja_expression_in_url(self, source, expected):
+        assert lhtml.run(source) == expected
+
+    def test_jinja_double_quotes_are_not_escaped_in_attributes(self):
+        assert lhtml.run('link::{{ url_for("page") }}[x]') == (
+            '<a href="{{ url_for("page") }}">x</a>')
+        assert lhtml.run('div::[font-family:{{ font("a") }}; content:"x"] y ::') == (
+            '<div style="font-family:{{ font("a") }}; content:&quot;x&quot;"> y </div>')
+
+    def test_jinja_in_video_url(self):
+        assert '<source src="{{ base }}/clip.mp4"' in lhtml.run('video::{{ base }}/clip.mp4')
+
+    @pytest.mark.parametrize('source, expected', [
+        ('=(.t__x__) Title', '<h1 class="t__x__">Title</h1>\n'),
+        ('==(.a**b** #id__1__) **Bold** __it__',
+         '<h2 class="a**b**" id="id__1__"><strong>Bold</strong> <em>it</em></h2>\n'),
+        ('= Title (.not__class__)', '<h1>Title (.not<em>class</em>)</h1>\n'),
+    ])
+    def test_heading_classes_are_not_formatted(self, source, expected):
+        assert lhtml.run(source) == expected
+
+    @pytest.mark.parametrize('opening', ['{{', '{%'])
+    def test_many_unclosed_jinja_delimiters_are_fast(self, opening):
+        source = (opening + ' x ') * 20000 + '**b**'
+        start = time.perf_counter()
+        result = lhtml.run(source)
+        assert time.perf_counter() - start < 1
+        assert result.endswith('<strong>b</strong>')
+
+    def test_jinja_delimiter_in_string_still_matches(self):
+        assert lhtml.run('{{ "{{" }} __a__') == '{{ "{{" }} <em>a</em>'
+        assert lhtml.run('{% set x = "{%" %} __a__') == '{% set x = "{%" %} <em>a</em>'
+
+
+@pytest.mark.parametrize('source, expected', [
+    ('videoplay::v.mp4[width:400px;]', '<video autoplay loop muted style="width:400px;">'),
+    ('videoplay::v.mp4(.c)', '<video autoplay loop muted class="c">'),
+    ('videoplay::v.mp4', '<video autoplay loop muted>'),
+    ('video::v.mp4[width:400px;]', '<video style="width:400px;">'),
+])
+def test_video_opening_tag_spacing(source, expected):
+    assert lhtml.run(source).startswith(expected + '\n')