markdown-to-confluence 0.2.1__tar.gz → 0.2.2__tar.gz

{markdown_to_confluence-0.2.1 → markdown_to_confluence-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: markdown-to-confluence
-Version: 0.2.1
+Version: 0.2.2
 Summary: Publish Markdown files to Confluence wiki
 Home-page: https://github.com/hunyadi/md2conf
 Author: Levente Hunyadi
@@ -144,6 +144,12 @@ Provide generated-by prompt text in the Markdown file with a tag:
 
 Alternatively, use the `--generated-by GENERATED_BY` option. The tag takes precedence.
 
+### Ignoring files
+
+Skip files in a directory with rules defined in `.mdignore`. Each rule should occupy a single line. Rules follow the syntax of [fnmatch](https://docs.python.org/3/library/fnmatch.html#fnmatch.fnmatch). Specifically, `?` matches any single character, and `*` matches zero or more characters. For example, use `up-*.md` to exclude Markdown files that start with `up-`. Lines that start with `#` are treated as comments.
+
+Files that don't have the extension `*.md` are skipped automatically. Hidden directories (whose name starts with `.`) are not recursed into.
+
 ### Running the tool
 
 You execute the command-line tool `md2conf` to synchronize the Markdown file with Confluence:
@@ -195,18 +201,30 @@ options:
   --webui-links         Enable Confluence Web UI links.
 ```
 
-### Using the docker container
+### Using the Docker container
+
+You can run the Docker container via `docker run` or via `Dockerfile`. Either can accept the environment variables or arguments similar to the Python options. The final argument `./` corresponds to `mdpath` in the command-line utility.
+
+With `docker run`, you can pass Confluence domain, user, API and space key directly to `docker run`:
 
-You can run the docker container via `docker run` or via `Dockerfile`. Either can accept the environment variables or arguments similar to the Python options.  The final argument `./` corresponds to `mdpath` in the command-line utility.
+```sh
+docker run --rm --name md2conf -v $(pwd):/data leventehunyadi/md2conf -d instructure.atlassian.net -u levente.hunyadi@instructure.com -a 0123456789abcdef -s DAP ./
+```
+
+Alternatively, you can use a separate file `.env` to pass these parameters as environment variables:
 
 ```sh
-docker run --rm --name md2conf hunyadi/md2conf -d instructure.atlassian.net -u levente.hunyadi@instructure.com -a 0123456789abcdef -s DAP ./
+docker run --rm --env-file .env --name md2conf -v $(pwd):/data leventehunyadi/md2conf ./
 ```
 
-Note that the entry point for the docker container's base image is `ENTRYPOINT ["python3", "-m", "md2conf"]`.
+In each case, `-v $(pwd):/data` maps the current directory to Docker container's `WORKDIR` such *md2conf* can scan files and directories in the local file system.
+
+Note that the entry point for the Docker container's base image is `ENTRYPOINT ["python3", "-m", "md2conf"]`.
+
+With the `Dockerfile` approach, you can extend the base image:
 
 ```Dockerfile
-FROM hunyadi/md2conf:latest
+FROM leventehunyadi/md2conf:latest
 
 ENV CONFLUENCE_DOMAIN='instructure.atlassian.net'
 ENV CONFLUENCE_PATH='/wiki/'
@@ -220,7 +238,7 @@ CMD ["./"]
 Alternatively,
 
 ```Dockerfile
-FROM hunyadi/md2conf:latest
+FROM leventehunyadi/md2conf:latest
 
 CMD ["-d", "instructure.atlassian.net", "-u", "levente.hunyadi@instructure.com", "-a", "0123456789abcdef", "-s", "DAP", "./"]
 ```

{markdown_to_confluence-0.2.1 → markdown_to_confluence-0.2.2}/README.md

@@ -113,6 +113,12 @@ Provide generated-by prompt text in the Markdown file with a tag:
 
 Alternatively, use the `--generated-by GENERATED_BY` option. The tag takes precedence.
 
+### Ignoring files
+
+Skip files in a directory with rules defined in `.mdignore`. Each rule should occupy a single line. Rules follow the syntax of [fnmatch](https://docs.python.org/3/library/fnmatch.html#fnmatch.fnmatch). Specifically, `?` matches any single character, and `*` matches zero or more characters. For example, use `up-*.md` to exclude Markdown files that start with `up-`. Lines that start with `#` are treated as comments.
+
+Files that don't have the extension `*.md` are skipped automatically. Hidden directories (whose name starts with `.`) are not recursed into.
+
 ### Running the tool
 
 You execute the command-line tool `md2conf` to synchronize the Markdown file with Confluence:
@@ -164,18 +170,30 @@ options:
   --webui-links         Enable Confluence Web UI links.
 ```
 
-### Using the docker container
+### Using the Docker container
+
+You can run the Docker container via `docker run` or via `Dockerfile`. Either can accept the environment variables or arguments similar to the Python options. The final argument `./` corresponds to `mdpath` in the command-line utility.
+
+With `docker run`, you can pass Confluence domain, user, API and space key directly to `docker run`:
 
-You can run the docker container via `docker run` or via `Dockerfile`. Either can accept the environment variables or arguments similar to the Python options.  The final argument `./` corresponds to `mdpath` in the command-line utility.
+```sh
+docker run --rm --name md2conf -v $(pwd):/data leventehunyadi/md2conf -d instructure.atlassian.net -u levente.hunyadi@instructure.com -a 0123456789abcdef -s DAP ./
+```
+
+Alternatively, you can use a separate file `.env` to pass these parameters as environment variables:
 
 ```sh
-docker run --rm --name md2conf hunyadi/md2conf -d instructure.atlassian.net -u levente.hunyadi@instructure.com -a 0123456789abcdef -s DAP ./
+docker run --rm --env-file .env --name md2conf -v $(pwd):/data leventehunyadi/md2conf ./
 ```
 
-Note that the entry point for the docker container's base image is `ENTRYPOINT ["python3", "-m", "md2conf"]`.
+In each case, `-v $(pwd):/data` maps the current directory to Docker container's `WORKDIR` such *md2conf* can scan files and directories in the local file system.
+
+Note that the entry point for the Docker container's base image is `ENTRYPOINT ["python3", "-m", "md2conf"]`.
+
+With the `Dockerfile` approach, you can extend the base image:
 
 ```Dockerfile
-FROM hunyadi/md2conf:latest
+FROM leventehunyadi/md2conf:latest
 
 ENV CONFLUENCE_DOMAIN='instructure.atlassian.net'
 ENV CONFLUENCE_PATH='/wiki/'
@@ -189,7 +207,7 @@ CMD ["./"]
 Alternatively,
 
 ```Dockerfile
-FROM hunyadi/md2conf:latest
+FROM leventehunyadi/md2conf:latest
 
 CMD ["-d", "instructure.atlassian.net", "-u", "levente.hunyadi@instructure.com", "-a", "0123456789abcdef", "-s", "DAP", "./"]
 ```

{markdown_to_confluence-0.2.1 → markdown_to_confluence-0.2.2}/markdown_to_confluence.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: markdown-to-confluence
-Version: 0.2.1
+Version: 0.2.2
 Summary: Publish Markdown files to Confluence wiki
 Home-page: https://github.com/hunyadi/md2conf
 Author: Levente Hunyadi
@@ -144,6 +144,12 @@ Provide generated-by prompt text in the Markdown file with a tag:
 
 Alternatively, use the `--generated-by GENERATED_BY` option. The tag takes precedence.
 
+### Ignoring files
+
+Skip files in a directory with rules defined in `.mdignore`. Each rule should occupy a single line. Rules follow the syntax of [fnmatch](https://docs.python.org/3/library/fnmatch.html#fnmatch.fnmatch). Specifically, `?` matches any single character, and `*` matches zero or more characters. For example, use `up-*.md` to exclude Markdown files that start with `up-`. Lines that start with `#` are treated as comments.
+
+Files that don't have the extension `*.md` are skipped automatically. Hidden directories (whose name starts with `.`) are not recursed into.
+
 ### Running the tool
 
 You execute the command-line tool `md2conf` to synchronize the Markdown file with Confluence:
@@ -195,18 +201,30 @@ options:
   --webui-links         Enable Confluence Web UI links.
 ```
 
-### Using the docker container
+### Using the Docker container
+
+You can run the Docker container via `docker run` or via `Dockerfile`. Either can accept the environment variables or arguments similar to the Python options. The final argument `./` corresponds to `mdpath` in the command-line utility.
+
+With `docker run`, you can pass Confluence domain, user, API and space key directly to `docker run`:
 
-You can run the docker container via `docker run` or via `Dockerfile`. Either can accept the environment variables or arguments similar to the Python options.  The final argument `./` corresponds to `mdpath` in the command-line utility.
+```sh
+docker run --rm --name md2conf -v $(pwd):/data leventehunyadi/md2conf -d instructure.atlassian.net -u levente.hunyadi@instructure.com -a 0123456789abcdef -s DAP ./
+```
+
+Alternatively, you can use a separate file `.env` to pass these parameters as environment variables:
 
 ```sh
-docker run --rm --name md2conf hunyadi/md2conf -d instructure.atlassian.net -u levente.hunyadi@instructure.com -a 0123456789abcdef -s DAP ./
+docker run --rm --env-file .env --name md2conf -v $(pwd):/data leventehunyadi/md2conf ./
 ```
 
-Note that the entry point for the docker container's base image is `ENTRYPOINT ["python3", "-m", "md2conf"]`.
+In each case, `-v $(pwd):/data` maps the current directory to Docker container's `WORKDIR` such *md2conf* can scan files and directories in the local file system.
+
+Note that the entry point for the Docker container's base image is `ENTRYPOINT ["python3", "-m", "md2conf"]`.
+
+With the `Dockerfile` approach, you can extend the base image:
 
 ```Dockerfile
-FROM hunyadi/md2conf:latest
+FROM leventehunyadi/md2conf:latest
 
 ENV CONFLUENCE_DOMAIN='instructure.atlassian.net'
 ENV CONFLUENCE_PATH='/wiki/'
@@ -220,7 +238,7 @@ CMD ["./"]
 Alternatively,
 
 ```Dockerfile
-FROM hunyadi/md2conf:latest
+FROM leventehunyadi/md2conf:latest
 
 CMD ["-d", "instructure.atlassian.net", "-u", "levente.hunyadi@instructure.com", "-a", "0123456789abcdef", "-s", "DAP", "./"]
 ```

{markdown_to_confluence-0.2.1 → markdown_to_confluence-0.2.2}/markdown_to_confluence.egg-info/SOURCES.txt

@@ -16,10 +16,13 @@ md2conf/api.py
 md2conf/application.py
 md2conf/converter.py
 md2conf/entities.dtd
+md2conf/matcher.py
 md2conf/mermaid.py
 md2conf/processor.py
 md2conf/properties.py
+md2conf/puppeteer-config.json
 md2conf/py.typed
 tests/test_conversion.py
+tests/test_matcher.py
 tests/test_mermaid.py
 tests/test_processor.py

{markdown_to_confluence-0.2.1 → markdown_to_confluence-0.2.2}/md2conf/__init__.py

@@ -5,7 +5,7 @@ Parses Markdown files, converts Markdown content into the Confluence Storage For
 Confluence API endpoints to upload images and content.
 """
 
-__version__ = "0.2.1"
+__version__ = "0.2.2"
 __author__ = "Levente Hunyadi"
 __copyright__ = "Copyright 2022-2024, Levente Hunyadi"
 __license__ = "MIT"

{markdown_to_confluence-0.2.1 → markdown_to_confluence-0.2.2}/md2conf/application.py

@@ -13,6 +13,7 @@ from .converter import (
     extract_qualified_id,
     read_qualified_id,
 )
+from .matcher import Matcher, MatcherOptions
 
 LOGGER = logging.getLogger(__name__)
 
@@ -89,16 +90,18 @@ class Application:
 
         LOGGER.info(f"Indexing directory: {local_dir}")
 
+        matcher = Matcher(MatcherOptions(source=".mdignore", extension="md"), local_dir)
+
         files: List[Path] = []
         directories: List[Path] = []
         for entry in os.scandir(local_dir):
+            if matcher.is_excluded(entry.name):
+                continue
+
             if entry.is_file():
-                if entry.name.endswith(".md"):
-                    # skip non-markdown files
-                    files.append((Path(local_dir) / entry.name).absolute())
+                files.append((Path(local_dir) / entry.name).absolute())
             elif entry.is_dir():
-                if not entry.name.startswith("."):
-                    directories.append((Path(local_dir) / entry.name).absolute())
+                directories.append((Path(local_dir) / entry.name).absolute())
 
         # make page act as parent node in Confluence
         parent_id: Optional[ConfluenceQualifiedID] = None
@@ -141,7 +144,7 @@ class Application:
         else:
             if parent_id is None:
                 raise ValueError(
-                    "expected: Confluence page ID to act as parent for Markdown files with no linked Confluence page"
+                    f"expected: parent page ID for Markdown file with no linked Confluence page: {absolute_path}"
                 )
 
             confluence_page = self._create_page(

markdown_to_confluence-0.2.2/md2conf/matcher.py

@@ -0,0 +1,83 @@
+import os.path
+from dataclasses import dataclass
+from fnmatch import fnmatch
+from pathlib import Path
+from typing import Iterable, List, Optional
+
+
+@dataclass
+class MatcherOptions:
+    """
+    Options for checking against a list of exclude/include patterns.
+
+    :param source: File name to read exclusion rules from.
+    :param extension: Extension to narrow down search to.
+    """
+
+    source: str
+    extension: Optional[str] = None
+
+    def __post_init__(self) -> None:
+        if self.extension is not None and not self.extension.startswith("."):
+            self.extension = f".{self.extension}"
+
+
+class Matcher:
+    "Compares file and directory names against a list of exclude/include patterns."
+
+    options: MatcherOptions
+    rules: List[str]
+
+    def __init__(self, options: MatcherOptions, directory: Path) -> None:
+        self.options = options
+        if os.path.exists(directory / options.source):
+            with open(directory / options.source, "r") as f:
+                rules = f.read().splitlines()
+            self.rules = [rule for rule in rules if rule and not rule.startswith("#")]
+        else:
+            self.rules = []
+
+    def extension_matches(self, name: str) -> bool:
+        "True if the file name has the expected extension."
+
+        return self.options.extension is None or name.endswith(self.options.extension)
+
+    def is_excluded(self, name: str) -> bool:
+        "True if the file or directory name matches any of the exclusion patterns."
+
+        if name.startswith("."):
+            return True
+
+        if not self.extension_matches(name):
+            return True
+
+        for rule in self.rules:
+            if fnmatch(name, rule):
+                return True
+        else:
+            return False
+
+    def is_included(self, name: str) -> bool:
+        "True if the file or directory name matches none of the exclusion patterns."
+
+        return not self.is_excluded(name)
+
+    def filter(self, items: Iterable[str]) -> List[str]:
+        """
+        Returns only those elements from the input that don't match any of the exclusion rules.
+
+        :param items: A list of names to filter.
+        :returns: A filtered list of names that didn't match any of the exclusion rules.
+        """
+
+        return [item for item in items if self.is_included(item)]
+
+    def scandir(self, path: Path) -> List[str]:
+        """
+        Returns only those entries in a directory whose name doesn't match any of the exclusion rules.
+
+        :param path: Directory to scan.
+        :returns: A filtered list of entries whose name didn't match any of the exclusion rules.
+        """
+
+        return self.filter(entry.name for entry in os.scandir(path))

markdown_to_confluence-0.2.2/md2conf/mermaid.py

@@ -0,0 +1,76 @@
+import logging
+import os
+import os.path
+import shutil
+import subprocess
+from typing import Literal
+
+LOGGER = logging.getLogger(__name__)
+
+
+def is_docker() -> bool:
+    "True if the application is running in a Docker container."
+
+    return (
+        os.environ.get("CHROME_BIN") == "/usr/bin/chromium-browser"
+        and os.environ.get("PUPPETEER_SKIP_DOWNLOAD") == "true"
+    )
+
+
+def get_mmdc() -> str:
+    "Path to the Mermaid diagram converter."
+
+    if is_docker():
+        return "/home/md2conf/node_modules/.bin/mmdc"
+    elif os.name == "nt":
+        return "mmdc.cmd"
+    else:
+        return "mmdc"
+
+
+def has_mmdc() -> bool:
+    "True if Mermaid diagram converter is available on the OS."
+
+    executable = get_mmdc()
+    return shutil.which(executable) is not None
+
+
+def render(source: str, output_format: Literal["png", "svg"] = "png") -> bytes:
+    "Generates a PNG or SVG image from a Mermaid diagram source."
+
+    filename = f"tmp_mermaid.{output_format}"
+
+    cmd = [
+        get_mmdc(),
+        "--input",
+        "-",
+        "--output",
+        filename,
+        "--outputFormat",
+        output_format,
+    ]
+    if is_docker():
+        cmd.extend(
+            ["-p", os.path.join(os.path.dirname(__file__), "puppeteer-config.json")]
+        )
+    LOGGER.debug(f"Executing: {' '.join(cmd)}")
+    try:
+        proc = subprocess.Popen(
+            cmd,
+            stdout=subprocess.PIPE,
+            stdin=subprocess.PIPE,
+            stderr=subprocess.PIPE,
+            text=False,
+        )
+        stdout, stderr = proc.communicate(input=source.encode("utf-8"))
+        if proc.returncode:
+            raise RuntimeError(
+                f"failed to convert Mermaid diagram; exit code: {proc.returncode}, "
+                f"output:\n{stdout.decode('utf-8')}\n{stderr.decode('utf-8')}"
+            )
+        with open(filename, "rb") as image:
+            return image.read()
+
+    finally:
+        if os.path.exists(filename):
+            os.remove(filename)

{markdown_to_confluence-0.2.1 → markdown_to_confluence-0.2.2}/md2conf/processor.py

@@ -11,6 +11,7 @@ from .converter import (
     ConfluenceQualifiedID,
     extract_qualified_id,
 )
+from .matcher import Matcher, MatcherOptions
 from .properties import ConfluenceProperties
 
 LOGGER = logging.getLogger(__name__)
@@ -69,16 +70,18 @@ class Processor:
 
         LOGGER.info(f"Indexing directory: {local_dir}")
 
+        matcher = Matcher(MatcherOptions(source=".mdignore", extension="md"), local_dir)
+
         files: List[Path] = []
         directories: List[Path] = []
         for entry in os.scandir(local_dir):
+            if matcher.is_excluded(entry.name):
+                continue
+
             if entry.is_file():
-                if entry.name.endswith(".md"):
-                    # skip non-markdown files
-                    files.append((Path(local_dir) / entry.name).absolute())
+                files.append((Path(local_dir) / entry.name).absolute())
             elif entry.is_dir():
-                if not entry.name.startswith("."):
-                    directories.append((Path(local_dir) / entry.name).absolute())
+                directories.append((Path(local_dir) / entry.name).absolute())
 
         for doc in files:
             metadata = self._get_page(doc)

markdown_to_confluence-0.2.2/md2conf/puppeteer-config.json

@@ -0,0 +1,8 @@
+{
+    "executablePath": "/usr/bin/chromium-browser",
+    "args": [
+        "--no-sandbox",
+        "--disable-gpu",
+        "--disable-setuid-sandbox"
+    ]
+}

{markdown_to_confluence-0.2.1 → markdown_to_confluence-0.2.2}/setup.cfg

@@ -43,6 +43,7 @@ exclude =
 [options.package_data]
 md2conf = 
 	entities.dtd
+	puppeteer-config.json
 	py.typed
 
 [options.entry_points]

{markdown_to_confluence-0.2.1 → markdown_to_confluence-0.2.2}/tests/test_conversion.py

@@ -12,6 +12,7 @@ from md2conf.converter import (
     elements_from_string,
     elements_to_string,
 )
+from md2conf.matcher import Matcher, MatcherOptions
 from md2conf.mermaid import has_mmdc
 
 logging.basicConfig(
@@ -50,8 +51,12 @@ class TestConversion(unittest.TestCase):
         shutil.rmtree(self.out_dir)
 
     def test_markdown(self) -> None:
+        matcher = Matcher(
+            MatcherOptions(source=".mdignore", extension="md"), self.source_dir
+        )
+
         for entry in os.scandir(self.source_dir):
-            if not entry.name.endswith(".md"):
+            if matcher.is_excluded(entry.name):
                 continue
 
             name, _ = os.path.splitext(entry.name)

markdown_to_confluence-0.2.2/tests/test_matcher.py

@@ -0,0 +1,43 @@
+import logging
+import os
+import os.path
+import unittest
+from pathlib import Path
+
+from md2conf.matcher import Matcher, MatcherOptions
+
+logging.basicConfig(
+    level=logging.INFO,
+    format="%(asctime)s - %(levelname)s - %(funcName)s [%(lineno)d] - %(message)s",
+)
+
+
+class TestMatcher(unittest.TestCase):
+    def test_extension(self) -> None:
+        directory = Path(os.path.dirname(__file__))
+        expected = [
+            entry.name for entry in os.scandir(directory) if entry.name.endswith(".py")
+        ]
+
+        options = MatcherOptions(".mdignore", ".py")
+        matcher = Matcher(options, directory)
+        actual = matcher.scandir(directory)
+
+        self.assertCountEqual(expected, actual)
+
+    def test_rules(self) -> None:
+        directory = Path(os.path.dirname(__file__)) / "source"
+        expected = [
+            entry.name for entry in os.scandir(directory) if entry.name.endswith(".md")
+        ]
+        expected.remove("ignore.md")
+
+        options = MatcherOptions(".mdignore", ".md")
+        matcher = Matcher(options, directory)
+        actual = matcher.scandir(directory)
+
+        self.assertCountEqual(expected, actual)
+
+
+if __name__ == "__main__":
+    unittest.main()

markdown_to_confluence-0.2.1/md2conf/mermaid.py

@@ -1,54 +0,0 @@
-import os
-import os.path
-import shutil
-import subprocess
-from typing import Literal
-
-
-def has_mmdc() -> bool:
-    "True if Mermaid diagram converter is available on the OS."
-
-    if os.name == "nt":
-        executable = "mmdc.cmd"
-    else:
-        executable = "mmdc"
-    return shutil.which(executable) is not None
-
-
-def render(source: str, output_format: Literal["png", "svg"] = "png") -> bytes:
-    "Generates a PNG or SVG image from a Mermaid diagram source."
-
-    filename = f"tmp_mermaid.{output_format}"
-
-    if os.name == "nt":
-        executable = "mmdc.cmd"
-    else:
-        executable = "mmdc"
-    try:
-        cmd = [
-            executable,
-            "--input",
-            "-",
-            "--output",
-            filename,
-            "--outputFormat",
-            output_format,
-        ]
-        proc = subprocess.Popen(
-            cmd,
-            stdout=subprocess.PIPE,
-            stdin=subprocess.PIPE,
-            stderr=subprocess.PIPE,
-            text=False,
-        )
-        proc.communicate(input=source.encode("utf-8"))
-        if proc.returncode:
-            raise RuntimeError(
-                f"failed to convert Mermaid diagram; exit code: {proc.returncode}"
-            )
-        with open(filename, "rb") as image:
-            return image.read()
-
-    finally:
-        if os.path.exists(filename):
-            os.remove(filename)