Markdown-Checker 1.0.3__tar.gz → 1.2.0__tar.gz

markdown_checker-1.2.0/PKG-INFO

@@ -0,0 +1,118 @@
+Metadata-Version: 2.4
+Name: markdown-checker
+Version: 1.2.0
+Summary: A markdown link validation reporting tool.
+Author-email: John Aziz <johnaziz269@gmail.com>
+Maintainer-email: John Aziz <johnaziz269@gmail.com>
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-Expression: MIT
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Environment :: Console
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Typing :: Typed
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Software Development :: Quality Assurance
+License-File: LICENSE
+Requires-Dist: click>=8.3.1
+Requires-Dist: httpx2>=2.0.0
+Project-URL: Contributing, https://github.com/john0isaac/markdown-checker/pulls
+Project-URL: Documentation, https://markdown-checker.readthedocs.io/en/latest/
+Project-URL: Homepage, https://github.com/john0isaac/markdown-checker
+Project-URL: Issues, https://github.com/john0isaac/markdown-checker/issues
+Project-URL: Releases, https://github.com/john0isaac/markdown-checker/releases
+
+[![PyPi](https://img.shields.io/pypi/v/markdown-checker)](https://pypi.org/project/markdown-checker/)
+[![Documentation Status](https://readthedocs.org/projects/markdown-checker/badge/?version=latest)](https://markdown-checker.readthedocs.io/en/latest/?badge=latest)
+[![Downloads](https://img.shields.io/pypi/dm/markdown-checker)](https://pypi.org/project/markdown-checker/)
+
+[![GitHub issues](https://img.shields.io/badge/issue_tracking-github-blue.svg)](https://github.com/john0isaac/markdown-checker/issues)
+[![Contributing](https://img.shields.io/badge/PR-Welcome-%23FF8300.svg?)](https://github.com/john0isaac/markdown-checker/pulls)
+
+markdown-checker is a markdown link validation reporting tool for `.md` and
+`.ipynb` files. It flags broken relative paths, broken web URLs, missing
+locale segments, and missing tracking IDs, then writes the findings as a
+report your CI can post back to a pull request.
+
+## Features
+
+- Five built-in checks: broken relative paths, broken web URLs, locale
+  segments in URLs, and tracking IDs on URLs and paths.
+- Concurrent URL checking with cross-file deduplication, per-host rate
+  pacing, and `Retry-After`/429 handling, so large repositories don't
+  trip host rate limits.
+- Four report formats: `markdown`, `json`, `github-annotations`, and
+  `console`.
+- Error vs. warning severity - rate-limited or unverifiable links are
+  reported but never fail your CI.
+- Configuration via a `[tool.markdown-checker]` table in `pyproject.toml`,
+  so you don't have to repeat flags in every command or workflow file.
+
+## Installation
+
+```bash
+pip install markdown-checker
+```
+
+## Quickstart
+
+```bash
+markdown-checker . -f check_broken_paths
+```
+
+```text
+🔍 Checked 42 links in 10 files.
+All files are compliant with the guidelines. 🎉
+```
+
+When issues are found, a `comment.md` report is written and the command
+exits with a non-zero status:
+
+```text
+🔍 Checked 42 links in 10 files.
+😭 Found 1 issues in the following files:
+    File 'docs/index.md', line 5
+./missing.md is broken.
+```
+
+## Using `markdown-checker` in GitHub Actions
+
+Run this tool within a GitHub workflow using the
+[action-check-markdown](https://github.com/marketplace/actions/check-markdown)
+GitHub Action, which posts the report as a pull request comment:
+
+```yaml
+- uses: john0isaac/action-check-markdown@v1.1.0
+  with:
+    github-token: ${{ secrets.GITHUB_TOKEN }}
+    command: check_broken_paths
+    directory: ./
+    guide-url: "https://github.com/<owner>/<repo>/blob/main/CONTRIBUTING.md"
+```
+
+See the [GitHub Actions how-to](https://markdown-checker.readthedocs.io/en/latest/howto/github-actions/)
+for the full workflow, including running `check_broken_urls` (not yet
+supported by the wrapper action) directly as a step.
+
+## Documentation
+
+- [Tutorial](https://markdown-checker.readthedocs.io/en/latest/tutorial/) - install and run your first check.
+- [How-to guides](https://markdown-checker.readthedocs.io/en/latest/howto/) - configuration, rate limiting, report formats, CI.
+- [CLI reference](https://markdown-checker.readthedocs.io/en/latest/reference/cli/) - every command-line option.
+- [API reference](https://markdown-checker.readthedocs.io/en/latest/api/) - for programmatic use.
+- [Full documentation](https://markdown-checker.readthedocs.io/en/latest/).
+
+## Contributing
+
+Contributions are welcome - see the
+[Contributing guide](https://markdown-checker.readthedocs.io/en/latest/CONTRIBUTING/).
+
+## License
+
+[MIT](https://github.com/john0isaac/markdown-checker/blob/main/LICENSE)
+

markdown_checker-1.2.0/README.md

@@ -0,0 +1,88 @@
+[![PyPi](https://img.shields.io/pypi/v/markdown-checker)](https://pypi.org/project/markdown-checker/)
+[![Documentation Status](https://readthedocs.org/projects/markdown-checker/badge/?version=latest)](https://markdown-checker.readthedocs.io/en/latest/?badge=latest)
+[![Downloads](https://img.shields.io/pypi/dm/markdown-checker)](https://pypi.org/project/markdown-checker/)
+
+[![GitHub issues](https://img.shields.io/badge/issue_tracking-github-blue.svg)](https://github.com/john0isaac/markdown-checker/issues)
+[![Contributing](https://img.shields.io/badge/PR-Welcome-%23FF8300.svg?)](https://github.com/john0isaac/markdown-checker/pulls)
+
+markdown-checker is a markdown link validation reporting tool for `.md` and
+`.ipynb` files. It flags broken relative paths, broken web URLs, missing
+locale segments, and missing tracking IDs, then writes the findings as a
+report your CI can post back to a pull request.
+
+## Features
+
+- Five built-in checks: broken relative paths, broken web URLs, locale
+  segments in URLs, and tracking IDs on URLs and paths.
+- Concurrent URL checking with cross-file deduplication, per-host rate
+  pacing, and `Retry-After`/429 handling, so large repositories don't
+  trip host rate limits.
+- Four report formats: `markdown`, `json`, `github-annotations`, and
+  `console`.
+- Error vs. warning severity - rate-limited or unverifiable links are
+  reported but never fail your CI.
+- Configuration via a `[tool.markdown-checker]` table in `pyproject.toml`,
+  so you don't have to repeat flags in every command or workflow file.
+
+## Installation
+
+```bash
+pip install markdown-checker
+```
+
+## Quickstart
+
+```bash
+markdown-checker . -f check_broken_paths
+```
+
+```text
+🔍 Checked 42 links in 10 files.
+All files are compliant with the guidelines. 🎉
+```
+
+When issues are found, a `comment.md` report is written and the command
+exits with a non-zero status:
+
+```text
+🔍 Checked 42 links in 10 files.
+😭 Found 1 issues in the following files:
+    File 'docs/index.md', line 5
+./missing.md is broken.
+```
+
+## Using `markdown-checker` in GitHub Actions
+
+Run this tool within a GitHub workflow using the
+[action-check-markdown](https://github.com/marketplace/actions/check-markdown)
+GitHub Action, which posts the report as a pull request comment:
+
+```yaml
+- uses: john0isaac/action-check-markdown@v1.1.0
+  with:
+    github-token: ${{ secrets.GITHUB_TOKEN }}
+    command: check_broken_paths
+    directory: ./
+    guide-url: "https://github.com/<owner>/<repo>/blob/main/CONTRIBUTING.md"
+```
+
+See the [GitHub Actions how-to](https://markdown-checker.readthedocs.io/en/latest/howto/github-actions/)
+for the full workflow, including running `check_broken_urls` (not yet
+supported by the wrapper action) directly as a step.
+
+## Documentation
+
+- [Tutorial](https://markdown-checker.readthedocs.io/en/latest/tutorial/) - install and run your first check.
+- [How-to guides](https://markdown-checker.readthedocs.io/en/latest/howto/) - configuration, rate limiting, report formats, CI.
+- [CLI reference](https://markdown-checker.readthedocs.io/en/latest/reference/cli/) - every command-line option.
+- [API reference](https://markdown-checker.readthedocs.io/en/latest/api/) - for programmatic use.
+- [Full documentation](https://markdown-checker.readthedocs.io/en/latest/).
+
+## Contributing
+
+Contributions are welcome - see the
+[Contributing guide](https://markdown-checker.readthedocs.io/en/latest/CONTRIBUTING/).
+
+## License
+
+[MIT](https://github.com/john0isaac/markdown-checker/blob/main/LICENSE)

{markdown_checker-1.0.3 → markdown_checker-1.2.0}/docs/CONTRIBUTING.rst

@@ -16,13 +16,13 @@ Install all dependencies and the package in editable mode:
 
 .. code-block:: shell
 
-	 uv sync
+     uv sync
 
 Activate the virtual environment:
 
 .. code-block:: shell
 
-	 source .venv/bin/activate
+     source .venv/bin/activate
 
 Any changes you make to the source will take effect immediately.
 
@@ -39,13 +39,13 @@ These are the most common commands you will use:
 These are some more specific commands if you need them:
 
 - ``tox parallel``: Run all test environments that will be run in CI, in
-	parallel. Python versions that are not installed are skipped.
+    parallel. Python versions that are not installed are skipped.
 - ``pre-commit``: Run the linter and formatter tools. Only staged files are
-	checked, and this runs automatically before each commit.
+    checked, and this runs automatically before each commit.
 - ``pre-commit run --all-files``: Run the pre-commit hooks against all files,
-	including unchanged and unstaged files.
+    including unchanged and unstaged files.
 - ``tox run -e py3.11``: Run unit tests with a specific Python version. The
-	version must be installed.
+    version must be installed.
 - ``tox run -e style``: Run all pre-commit hooks on all files via tox.
 - ``tox run -e typing``: Run all typing checks.
 - ``tox run -e docs``: Build the documentation in strict mode.
@@ -56,5 +56,5 @@ These are some more specific commands if you need them:
 
 .. tip::
 
-	 ``tox run`` and ``tox parallel`` can be shortened to ``tox r`` and
-	 ``tox p``.
+     ``tox run`` and ``tox parallel`` can be shortened to ``tox r`` and
+     ``tox p``.

markdown_checker-1.2.0/docs/api/checker.rst

@@ -0,0 +1,10 @@
+Checker
+=======
+
+Runs a named check across a set of files and aggregates the results; used
+internally by the CLI (:doc:`cli`).
+
+.. automodule:: markdown_checker.checker
+    :members:
+    :undoc-members:
+    :show-inheritance:

markdown_checker-1.2.0/docs/api/checks.rst

@@ -0,0 +1,45 @@
+Checks
+======
+
+The five built-in checks and their shared base class. See
+:doc:`../reference/checks` for what each one flags.
+
+Base
+----
+
+.. automodule:: markdown_checker.checks.base
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Broken Paths
+------------
+
+.. automodule:: markdown_checker.checks.broken_paths
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Broken URLs
+-----------
+
+.. automodule:: markdown_checker.checks.broken_urls
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Locale
+------
+
+.. automodule:: markdown_checker.checks.locale
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Tracking
+--------
+
+.. automodule:: markdown_checker.checks.tracking
+    :members:
+    :undoc-members:
+    :show-inheritance:

markdown_checker-1.2.0/docs/api/cli.rst

@@ -0,0 +1,10 @@
+CLI
+===
+
+The ``markdown-checker`` command-line entry point. See
+:doc:`../reference/cli` for a user-facing option reference.
+
+.. automodule:: markdown_checker.cli
+    :members:
+    :undoc-members:
+    :show-inheritance:

markdown_checker-1.2.0/docs/api/main.rst

@@ -0,0 +1,14 @@
+Main
+====
+
+Top-level package exports: :class:`~markdown_checker.models.Config`,
+:class:`~markdown_checker.models.MarkdownPath`,
+:class:`~markdown_checker.models.MarkdownURL`,
+:class:`~markdown_checker.models.MarkdownLinkBase`, ``main``, and
+``__version__``. Documented canonically under :doc:`models`, :doc:`cli`,
+and :doc:`checker`; shown here without indexing to avoid duplicate targets.
+
+.. automodule:: markdown_checker
+    :members:
+    :undoc-members:
+    :noindex:

markdown_checker-1.2.0/docs/api/models.rst

@@ -0,0 +1,39 @@
+Models
+======
+
+Data models shared by the checks: extracted links (:class:`MarkdownPath
+<markdown_checker.models.path.MarkdownPath>`,
+:class:`MarkdownURL <markdown_checker.models.url.MarkdownURL>`) and runtime
+configuration (:class:`Config <markdown_checker.models.config.Config>`).
+
+Base
+----
+
+.. automodule:: markdown_checker.models.base
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Config
+------
+
+.. automodule:: markdown_checker.models.config
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Path
+----
+
+.. automodule:: markdown_checker.models.path
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+URL
+---
+
+.. automodule:: markdown_checker.models.url
+    :members:
+    :undoc-members:
+    :show-inheritance:

markdown_checker-1.2.0/docs/api/patterns.rst

@@ -0,0 +1,10 @@
+Patterns
+========
+
+Compiled regular expressions used across the package for link/scheme/fence/
+locale/tracking matching.
+
+.. automodule:: markdown_checker.patterns
+    :members:
+    :undoc-members:
+    :show-inheritance:

markdown_checker-1.2.0/docs/api/reports.rst

@@ -0,0 +1,71 @@
+Reports
+=======
+
+The format-agnostic report model, the renderer contract and
+implementations, and the writer that emits rendered reports. See
+:doc:`../reference/report-formats` for the user-facing shape of each format.
+
+Base
+----
+
+.. automodule:: markdown_checker.reports.base
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Model
+------
+
+.. automodule:: markdown_checker.reports.model
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Registry
+--------
+
+.. automodule:: markdown_checker.reports.registry
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+
+Writer
+------
+
+.. automodule:: markdown_checker.reports.writer
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Markdown Renderer
+------------------
+
+.. automodule:: markdown_checker.reports.renderers.markdown
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+JSON Renderer
+-------------
+
+.. automodule:: markdown_checker.reports.renderers.json
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+GitHub Annotations Renderer
+----------------------------
+
+.. automodule:: markdown_checker.reports.renderers.annotations
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Console Renderer
+-----------------
+
+.. automodule:: markdown_checker.reports.renderers.console
+    :members:
+    :undoc-members:
+    :show-inheritance:

markdown_checker-1.2.0/docs/api/utils.rst

@@ -0,0 +1,46 @@
+Utilities
+=========
+
+Standalone helpers: link extraction, file listing, the concurrent
+URL-checking pipeline, GitHub Actions environment detection, pyproject.toml
+config loading, and the CLI spinner.
+
+Extract Links
+-------------
+
+.. automodule:: markdown_checker.utils.extract_links
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+GitHub Environment
+-------------------
+
+.. automodule:: markdown_checker.utils.github_env
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+List Files
+----------
+
+.. automodule:: markdown_checker.utils.list_files
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Spinner
+-------
+
+.. automodule:: markdown_checker.utils.spinner
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+URL Pipeline
+------------
+
+.. automodule:: markdown_checker.utils.url_pipeline
+    :members:
+    :undoc-members:
+    :show-inheritance:

{markdown_checker-1.0.3 → markdown_checker-1.2.0}/docs/conf.py

@@ -15,6 +15,7 @@ extensions = [
     "sphinx.ext.autodoc",
     "sphinx.ext.extlinks",
     "sphinx.ext.intersphinx",
+    "sphinx.ext.napoleon",
     "sphinxcontrib.log_cabinet",
     "sphinx_tabs.tabs",
     "sphinx_copybutton",
@@ -23,6 +24,12 @@ extensions = [
 autodoc_member_order = "bysource"
 autodoc_typehints = "description"
 autodoc_preserve_defaults = True
+autodoc_default_options = {
+    "no-value": True,
+}
+napoleon_google_docstring = True
+napoleon_numpy_docstring = False
+napoleon_use_rtype = False
 extlinks = {
     "issue": ("https://github.com/john0isaac/markdown-checker/issues/%s", "#%s"),
     "pr": ("https://github.com/john0isaac/markdown-checker/pull/%s", "#%s"),

markdown_checker-1.2.0/docs/explanation/index.rst

@@ -0,0 +1,12 @@
+Explanation
+============
+
+Background and design discussion for understanding *why* markdown-checker
+behaves the way it does, rather than step-by-step instructions. For tasks,
+see :doc:`../howto/index`; for exact facts, see :doc:`../reference/index`.
+
+.. toctree::
+   :maxdepth: 1
+
+   url-checking
+   link-detection

markdown_checker-1.2.0/docs/explanation/link-detection.rst

@@ -0,0 +1,53 @@
+How Link Detection Works
+===========================
+
+Every check starts from the same extraction step: reading a markdown file
+line by line and pulling out link destinations. This page explains what
+counts as a checkable link and what's deliberately excluded.
+
+What's matched
+-----------------
+
+Only the destination of an inline Markdown link is considered - the part
+inside parentheses in ``[text](destination)``, or inside angle brackets in
+``[text](<destination>)``. Reference-style links (``[text][ref]`` with a
+separate ``[ref]: destination`` definition) are not currently matched.
+
+Code fences are skipped entirely: lines between an opening and matching
+closing fence (three or more backtick or tilde characters) are ignored, so
+example links inside a code sample are never checked.
+
+How a destination is classified
+------------------------------------
+
+Each matched destination falls into one of three buckets:
+
+- **URL**: starts with ``http://`` or ``https://`` (protocol-relative
+  ``//host/...`` links are treated as ``https://``). Handled by
+  ``check_broken_urls``, ``check_urls_locale``, and ``check_urls_tracking``.
+- **Ignored**: starts with another URI scheme (e.g. ``mailto:``, ``tel:``),
+  or is a bare same-page anchor (``#section``). These can't be meaningfully
+  checked as either a URL or a local file, so no check ever sees them.
+- **Path**: anything else - this includes root-relative paths
+  (``/docs/usage.md``), explicitly relative paths (``./img.png``,
+  ``../docs/usage.md``), and bare relative paths with no prefix
+  (``docs/usage.md``). All of these are treated identically and handled by
+  ``check_broken_paths`` and ``check_paths_tracking``.
+
+Fragments on paths
+----------------------
+
+A path may carry a trailing ``?query`` and/or ``#anchor`` - for example
+``docs/usage.md#configuration``. Both checks that operate on paths strip
+this fragment before resolving the file, so only the ``docs/usage.md``
+part needs to exist on disk; the anchor itself is never validated against
+the target file's headings.
+
+Resolving a path to a file
+--------------------------------
+
+``check_broken_paths`` tries to resolve a path relative to the markdown
+file it was found in first (e.g. ``../img.png`` in ``docs/usage.md``
+resolves against ``docs/``), then falls back to resolving it as an
+absolute or current-working-directory-relative path. A path is reported as
+broken only if neither resolution finds an existing file.

markdown_checker-1.2.0/docs/explanation/url-checking.rst

@@ -0,0 +1,78 @@
+How URL Checking Works
+========================
+
+``check_broken_urls`` (and the other URL-based checks) can end up sending
+thousands of requests across a large repository. This page explains the
+pipeline that makes that practical without hammering any single host, and
+why some failures are reported as warnings rather than errors.
+
+The life of a URL
+--------------------
+
+1. **Extraction.** Every markdown link destination is extracted per file
+   (see :doc:`link-detection`) and classified as a URL if it starts with
+   ``http://``/``https://`` (protocol-relative ``//host/...`` links are
+   normalized to ``https://``).
+
+2. **Normalization and dedupe.** Before checking, each URL is normalized
+   (scheme and host lowercased, trailing slash and fragment dropped; the
+   query string is kept). If the same normalized URL appears in fifty
+   files, it is only ever requested once - every other occurrence shares
+   the same in-flight result.
+
+3. **Per-host queueing and pacing.** Requests are grouped by host and paced
+   with a minimum delay between two requests to the same host
+   (``--per-host-delay``), so a link-heavy file doesn't burst a host with
+   many simultaneous requests just because it happens to link to it
+   repeatedly.
+
+4. **The request itself.** Each URL is checked with a ``HEAD`` request
+   first; if that isn't a success, a ``GET`` is attempted as a fallback,
+   since some servers reject ``HEAD`` but allow ``GET``. A successful
+   (``2xx``) response on either is treated as ``alive``.
+
+5. **Hard-failure retries.** If neither request succeeds and there's no
+   rate-limit or auth signal, the attempt is retried with exponential
+   backoff up to ``--retries`` times before being reported as ``broken``.
+
+6. **Rate limiting and auth errors return immediately.** A ``429`` (or any
+   non-success response carrying ``Retry-After``) is reported as
+   ``rate_limited`` right away - it is *not* retried in place. Likewise, a
+   ``401``/``403`` response is reported as ``unverifiable`` immediately:
+   the server is reachable but blocking automated access, and retrying
+   won't change that. Retrying either case would only generate more
+   traffic against a host that's already unhappy with us.
+
+7. **Host circuit breaker.** When a host produces two rate-limited
+   responses in a row, it's treated as persistently throttled: every other
+   URL already queued for that host is resolved as ``rate_limited``
+   without spending a network request on each, and the host is blocked
+   from new requests until its ``Retry-After`` window elapses.
+
+8. **Backpressure.** The number of URL checks in flight at once is bounded
+   (``max_pending`` in the underlying configuration), so checking a huge
+   repository doesn't grow memory unboundedly - submitting more work simply
+   waits for a slot to free up.
+
+Why warnings don't fail the build
+-------------------------------------
+
+``rate_limited``, ``unverifiable``, and ``transient_error`` all mean the
+checker *couldn't determine* whether the link is actually broken - the host
+is either throttling, blocking bots, or momentarily unreachable. Treating
+these the same as a confirmed ``404`` would make CI fail on
+infrastructure noise rather than real broken links, so they're reported as
+warnings instead: visible, but never blocking. Only ``broken`` (a
+confirmed non-2xx response with no rate-limit/auth signal) is an
+error. See :doc:`../reference/exit-codes` for the full status table.
+
+Overlapping file processing
+-------------------------------
+
+Separately from URL-level concurrency, files themselves are processed
+through an adaptive window: a file's links are submitted for checking as
+soon as they're extracted, without waiting for earlier files' checks to
+finish first. This overlaps link extraction and I/O wait time across
+files, while still bounding how many files can be "in flight"
+simultaneously so memory use stays predictable regardless of how many
+links a single file happens to contain.