docs-site 0.1.1__tar.gz

docs_site-0.1.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Lucas Rollin Ferreira
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

docs_site-0.1.1/PKG-INFO

@@ -0,0 +1,95 @@
+Metadata-Version: 2.4
+Name: docs-site
+Version: 0.1.1
+Summary: Turn a folder of dumped HTML + PDF files into a browsable, searchable local documentation site.
+Author-email: Lucas Rollin Ferreira <lucasrollinferreira@gmail.com>
+License: MIT
+Project-URL: Homepage, https://github.com/lucas-rollin/docs-site
+Project-URL: Repository, https://github.com/lucas-rollin/docs-site
+Project-URL: Issues, https://github.com/lucas-rollin/docs-site/issues
+Keywords: documentation,pdf,html,search,static-site,reader
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Education
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Documentation
+Classifier: Topic :: Text Processing :: Markup :: HTML
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: beautifulsoup4>=4.12
+Requires-Dist: jinja2>=3.1
+Requires-Dist: pypdf>=4.0
+Dynamic: license-file
+
+# docs-site
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Python: 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
+
+Turn a folder of dumped HTML and PDF files into a clean, searchable, dark-mode-ready static documentation portal.
+
+## Features
+
+- **Static & Offline**: Generates self-contained static HTML/CSS/JS with zero runtime server required.
+- **Fast Client-side Search**: Full-text search across all HTML and PDF content, with highlight previews and keyboard navigation (`↑`/`↓`/`Enter`/`Esc`).
+- **Automatic Table of Contents**: Extracts heading outlines from HTML tags (`<h1>`–`<h6>`) and bookmark outlines from PDF files into a side navigation panel.
+- **Dark Mode**: Built-in dark/light theme toggle, including synchronized dark theme styling for framed HTML documents.
+- **Zero Configuration**: Point it at any nested folder and it recursively scans and indexes supported files.
+
+## Installation
+
+Install globally as a CLI tool using [uv](https://docs.astral.sh/uv/):
+
+```bash
+uv tool install docs-site
+```
+
+Alternatively, install with `pipx`:
+
+```bash
+pipx install docs-site
+```
+
+## Usage
+
+```bash
+# Build and open the site in your default browser
+docs-site ~/notes/linux-learning
+
+# Specify a custom output directory (default is <folder>/_site)
+docs-site ~/notes/linux-learning -o ~/public_html/docs
+
+# Build without automatically opening the browser
+docs-site ~/notes/linux-learning --no-open
+
+# View all options
+docs-site --help
+```
+
+Re-running the command rebuilds the site from scratch.
+
+## Project Scope & Constraints
+
+- **Supported Formats**: `.html`, `.htm`, and `.pdf` files.
+- **HTML Assets**: HTML files are wrapped and rendered inside an iframe. Relative links to external assets (e.g. images or local stylesheets) should be self-contained or absolute.
+- **PDF Viewing**: PDFs are displayed directly using your browser's native PDF reader.
+- **Indexing Cap**: Indexed full text is capped at 400,000 characters per file to keep the client-side search index responsive and lightweight.
+
+## Contributing
+
+Looking to work on `docs-site` itself? See [CONTRIBUTING.md](CONTRIBUTING.md) for the development setup, testing, and architecture overview.
+
+## Project Status
+
+This tool was initially built to solve personal study and document organization workflows. Feedback, bug reports, and contributions are welcome.
+
+## License
+
+This project is licensed under the [MIT License](LICENSE).

docs_site-0.1.1/README.md

@@ -0,0 +1,65 @@
+# docs-site
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
+[![Python: 3.11+](https://img.shields.io/badge/python-3.11+-blue.svg)](https://www.python.org/downloads/)
+
+Turn a folder of dumped HTML and PDF files into a clean, searchable, dark-mode-ready static documentation portal.
+
+## Features
+
+- **Static & Offline**: Generates self-contained static HTML/CSS/JS with zero runtime server required.
+- **Fast Client-side Search**: Full-text search across all HTML and PDF content, with highlight previews and keyboard navigation (`↑`/`↓`/`Enter`/`Esc`).
+- **Automatic Table of Contents**: Extracts heading outlines from HTML tags (`<h1>`–`<h6>`) and bookmark outlines from PDF files into a side navigation panel.
+- **Dark Mode**: Built-in dark/light theme toggle, including synchronized dark theme styling for framed HTML documents.
+- **Zero Configuration**: Point it at any nested folder and it recursively scans and indexes supported files.
+
+## Installation
+
+Install globally as a CLI tool using [uv](https://docs.astral.sh/uv/):
+
+```bash
+uv tool install docs-site
+```
+
+Alternatively, install with `pipx`:
+
+```bash
+pipx install docs-site
+```
+
+## Usage
+
+```bash
+# Build and open the site in your default browser
+docs-site ~/notes/linux-learning
+
+# Specify a custom output directory (default is <folder>/_site)
+docs-site ~/notes/linux-learning -o ~/public_html/docs
+
+# Build without automatically opening the browser
+docs-site ~/notes/linux-learning --no-open
+
+# View all options
+docs-site --help
+```
+
+Re-running the command rebuilds the site from scratch.
+
+## Project Scope & Constraints
+
+- **Supported Formats**: `.html`, `.htm`, and `.pdf` files.
+- **HTML Assets**: HTML files are wrapped and rendered inside an iframe. Relative links to external assets (e.g. images or local stylesheets) should be self-contained or absolute.
+- **PDF Viewing**: PDFs are displayed directly using your browser's native PDF reader.
+- **Indexing Cap**: Indexed full text is capped at 400,000 characters per file to keep the client-side search index responsive and lightweight.
+
+## Contributing
+
+Looking to work on `docs-site` itself? See [CONTRIBUTING.md](CONTRIBUTING.md) for the development setup, testing, and architecture overview.
+
+## Project Status
+
+This tool was initially built to solve personal study and document organization workflows. Feedback, bug reports, and contributions are welcome.
+
+## License
+
+This project is licensed under the [MIT License](LICENSE).

docs_site-0.1.1/pyproject.toml

@@ -0,0 +1,73 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "docs-site"
+version = "0.1.1"
+description = "Turn a folder of dumped HTML + PDF files into a browsable, searchable local documentation site."
+readme = "README.md"
+requires-python = ">=3.11"
+license = { text = "MIT" }
+authors = [
+    { name = "Lucas Rollin Ferreira", email = "lucasrollinferreira@gmail.com" }
+]
+keywords = ["documentation", "pdf", "html", "search", "static-site", "reader"]
+classifiers = [
+    "Development Status :: 4 - Beta",
+    "Environment :: Console",
+    "Intended Audience :: Developers",
+    "Intended Audience :: Education",
+    "License :: OSI Approved :: MIT License",
+    "Programming Language :: Python :: 3",
+    "Programming Language :: Python :: 3.11",
+    "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
+    "Topic :: Documentation",
+    "Topic :: Text Processing :: Markup :: HTML",
+]
+dependencies = [
+    "beautifulsoup4>=4.12",
+    "jinja2>=3.1",
+    "pypdf>=4.0",
+]
+
+[project.urls]
+Homepage = "https://github.com/lucas-rollin/docs-site"
+Repository = "https://github.com/lucas-rollin/docs-site"
+Issues = "https://github.com/lucas-rollin/docs-site/issues"
+
+[dependency-groups]
+dev = [
+    "djlint>=1.46.2",
+    "mypy>=1.10",
+    "pytest>=8.0",
+    "ruff>=0.16.9",
+]
+
+[project.scripts]
+docs-site = "docs_site.cli:main"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+docs_site = ["py.typed", "templates/*", "static/css/*", "static/js/*"]
+
+[tool.mypy]
+mypy_path = "src"
+warn_return_any = true
+warn_unused_configs = true
+warn_redundant_casts = true
+
+[[tool.mypy.overrides]]
+module = ["bs4.*"]
+ignore_missing_imports = true
+
+[tool.djlint]
+profile = "jinja"
+extension = "html.jinja"
+indent = 2
+exclude = ".venv"
+ignore = "H030,J004"

docs_site-0.1.1/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

docs_site-0.1.1/src/docs_site/__init__.py

@@ -0,0 +1,5 @@
+"""docs_site, turn a folder of HTML + PDF files into a searchable local doc site."""
+
+from .site_builder import build_site
+
+__all__ = ["build_site"]

docs_site-0.1.1/src/docs_site/__main__.py

@@ -0,0 +1,4 @@
+from .cli import main
+
+if __name__ == "__main__":
+    raise SystemExit(main())

docs_site-0.1.1/src/docs_site/cli.py

@@ -0,0 +1,59 @@
+"""Command-line entry point: `docs-site /path/to/folder`."""
+
+import argparse
+import sys
+import webbrowser
+from pathlib import Path
+
+from .site_builder import build_site
+
+
+def main(argv=None) -> int:
+    parser = argparse.ArgumentParser(
+        prog="docs-site",
+        description="Build a searchable local doc site from a folder of HTML + PDF files.",
+    )
+    parser.add_argument(
+        "folder",
+        type=Path,
+        help="Folder to scan (recursively) for .html/.htm/.pdf files",
+    )
+    parser.add_argument(
+        "-o",
+        "--output",
+        type=Path,
+        default=None,
+        help="Output directory (default: <folder>/_site)",
+    )
+    parser.add_argument(
+        "--no-open",
+        action="store_true",
+        help="Don't open the site in a browser after building",
+    )
+    args = parser.parse_args(argv)
+
+    root = args.folder.resolve()
+    if not root.is_dir():
+        print(f"error: not a folder: {root}", file=sys.stderr)
+        return 1
+
+    output_dir = (args.output or root / "_site").resolve()
+
+    manifest = build_site(root, output_dir, site_title=root.name)
+
+    n_html = sum(1 for f in manifest["files"] if f["type"] == "html")
+    n_pdf = sum(1 for f in manifest["files"] if f["type"] == "pdf")
+    print(f"Indexed {n_html} HTML file(s) and {n_pdf} PDF file(s).")
+    print(f"Site written to: {output_dir}")
+
+    if not manifest["files"]:
+        print(f"(No .html/.htm/.pdf files found under {root})")
+
+    if not args.no_open:
+        webbrowser.open((output_dir / "index.html").as_uri())
+
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

docs_site-0.1.1/src/docs_site/html_processing.py

@@ -0,0 +1,123 @@
+"""Everything needed to turn one arbitrary HTML file into a processed content page."""
+
+import re
+from pathlib import Path
+from typing import TypedDict
+
+from bs4 import BeautifulSoup
+
+from .types import Heading
+from .utils import slugify
+
+HTML_TAG_RE = re.compile(r"<html[\s>]", re.IGNORECASE)
+HEAD_TAG_RE = re.compile(r"<head[\s>]", re.IGNORECASE)
+BODY_TAG_RE = re.compile(r"<body[\s>]", re.IGNORECASE)
+HEAD_CLOSE_RE = re.compile(r"</head>", re.IGNORECASE)
+HTML_OPEN_RE = re.compile(r"(<html[^>]*>)", re.IGNORECASE)
+HTML_CLOSE_RE = re.compile(r"(</html>)", re.IGNORECASE)
+HEADING_TAG_RE = re.compile(r"^h[1-6]$")
+
+# Relative path (from output/pages/*.html) to the shared frontend static assets.
+CONTENT_CSS_HREF = "../static/css/content.css"
+CONTENT_DARK_LISTENER_SRC = "../static/js/content-dark-listener.js"
+
+
+class HtmlPageResult(TypedDict):
+    html: str
+    title: str
+    headings: list[Heading]
+    text: str
+
+
+def ensure_full_document(document: str, title: str) -> str:
+    """Wrap a bare HTML fragment into a full <html><head><body> document.
+
+    Leaves already-complete documents untouched; fills in only whichever of
+    <html>/<head>/<body> is missing.
+    """
+    if not HTML_TAG_RE.search(document):
+        return (
+            "<!DOCTYPE html>\n<html>\n<head>\n"
+            '<meta charset="utf-8">\n'
+            f"<title>{title}</title>\n"
+            "</head>\n<body>\n"
+            f"{document}\n"
+            "</body>\n</html>"
+        )
+
+    if not HEAD_TAG_RE.search(document):
+        document = HTML_OPEN_RE.sub(
+            r"\1\n<head>\n"
+            '<meta charset="utf-8">\n'
+            f"<title>{title}</title>\n"
+            "</head>",
+            document,
+            count=1,
+        )
+
+    if not BODY_TAG_RE.search(document):
+        if HEAD_CLOSE_RE.search(document):
+            document = HEAD_CLOSE_RE.sub(r"\g<0>\n<body>", document, count=1)
+        else:
+            document = HTML_OPEN_RE.sub(r"\1\n<body>", document, count=1)
+        document = HTML_CLOSE_RE.sub("</body>\n\\1", document, count=1)
+
+    return document
+
+
+def process_html_file(path: Path) -> HtmlPageResult:
+    """Parse, wrap-if-needed, id-tag headings, and link in the reader stylesheet/script."""
+    raw = path.read_text(encoding="utf-8", errors="replace")
+    raw = ensure_full_document(raw, title=path.stem)
+
+    soup = BeautifulSoup(raw, "html.parser")
+    html_tag = soup.html
+    if html_tag is None:
+        # Extremely malformed input; fall back to a minimal shell around it.
+        soup = BeautifulSoup(ensure_full_document(str(soup), path.stem), "html.parser")
+        html_tag = soup.html
+    if html_tag is None:
+        # ensure_full_document() always produces a top-level <html>
+        raise ValueError(f"could not construct a valid <html> root for {path}")
+
+    head_tag = soup.head
+    if head_tag is None:
+        head_tag = soup.new_tag("head")
+        html_tag.insert(0, head_tag)
+
+    body_tag = soup.body
+    if body_tag is None:
+        body_tag = soup.new_tag("body")
+        html_tag.append(body_tag)
+
+    title = path.stem
+    if soup.title and soup.title.string and soup.title.string.strip():
+        title = soup.title.string.strip()
+
+    used_ids = {tag.get("id") for tag in soup.find_all(id=True) if tag.get("id")}
+    headings: list[Heading] = []
+    for tag in soup.find_all(HEADING_TAG_RE):
+        text = tag.get_text(strip=True)
+        if not text:
+            continue
+        hid = tag.get("id")
+        if not hid:
+            hid = slugify(text, used_ids)
+            tag["id"] = hid
+        else:
+            used_ids.add(hid)
+        headings.append({"level": int(tag.name[1]), "id": str(hid), "text": text})
+
+    plain_text = soup.get_text(separator=" ", strip=True)
+
+    link_tag = soup.new_tag("link", rel="stylesheet", href=CONTENT_CSS_HREF)
+    head_tag.append(link_tag)
+    script_tag = soup.new_tag("script", src=CONTENT_DARK_LISTENER_SRC)
+    body_tag.append(script_tag)
+
+    return {
+        "html": str(soup),
+        "title": title,
+        "headings": headings,
+        "text": plain_text,
+    }

docs_site-0.1.1/src/docs_site/manifest.py

@@ -0,0 +1,111 @@
+"""Scan a folder for HTML/PDF files and build the JSON manifest the frontend reads."""
+
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+from typing import Any, Literal, TypedDict
+
+from .html_processing import process_html_file
+from .pdf_processing import process_pdf_file
+from .types import Heading
+from .utils import safe_id
+
+INDEXABLE_SUFFIXES = (".html", ".htm", ".pdf")
+
+# Per-file cap on indexed text so the manifest can't explode on huge PDFs.
+MAX_INDEXED_CHARS = 400_000
+
+
+@dataclass
+class ManifestEntry:
+    """One indexed file, in the shape the frontend's JS expects (see
+    static/js/shell.js). `page_count` is only meaningful for PDFs but is
+    always present so the JSON shape is uniform across entries.
+    """
+
+    id: str
+    type: Literal["html", "pdf"]
+    title: str
+    relpath: str
+    src: str
+    headings: list[Heading] = field(default_factory=list)
+    text: str = ""
+    page_count: int | None = None
+
+    def to_dict(self) -> dict[str, Any]:
+        return asdict(self)
+
+
+class Manifest(TypedDict):
+    files: list[dict[str, Any]]
+
+
+def scan_folder(root: Path, output_dir: Path) -> list[Path]:
+    """Recursively find all indexable files under root, skipping the output dir itself."""
+    files = []
+    for p in sorted(root.rglob("*")):
+        if not p.is_file():
+            continue
+        try:
+            p.relative_to(output_dir)
+            continue  # already inside the generated site — don't re-index it
+        except ValueError:
+            pass
+        if p.suffix.lower() in INDEXABLE_SUFFIXES:
+            files.append(p)
+    return files
+
+
+def build_manifest(root: Path, files: list[Path], pages_dir: Path) -> Manifest:
+    """Process every file, write out HTML content pages, and return the site manifest."""
+    manifest_entries: list[ManifestEntry] = []
+    used_ids: set[str] = set()
+
+    for path in files:
+        relpath = path.relative_to(root).as_posix()
+        fid = safe_id(relpath, used_ids)
+        suffix = path.suffix.lower()
+        entry: ManifestEntry
+
+        if suffix in (".html", ".htm"):
+            try:
+                html_result = process_html_file(path)
+            except Exception as exc:  # noqa: BLE001 - keep building the rest of the site
+                print(f"warning: skipping {relpath} ({exc})")
+                continue
+            out_path = pages_dir / f"{fid}.html"
+            out_path.write_text(html_result["html"], encoding="utf-8")
+            entry = ManifestEntry(
+                id=fid,
+                type="html",
+                title=html_result["title"],
+                relpath=relpath,
+                src=f"pages/{out_path.name}",
+                headings=html_result["headings"],
+                text=html_result["text"][:MAX_INDEXED_CHARS],
+            )
+
+        elif suffix == ".pdf":
+            try:
+                pdf_result = process_pdf_file(path)
+            except Exception as exc:  # noqa: BLE001
+                print(f"warning: skipping {relpath} ({exc})")
+                continue
+            # Reference the original PDF in place rather than duplicating it.
+            src = (Path("..") / path.relative_to(root)).as_posix()
+            entry = ManifestEntry(
+                id=fid,
+                type="pdf",
+                title=pdf_result["title"],
+                relpath=relpath,
+                src=src,
+                headings=pdf_result["headings"],
+                text=pdf_result["text"][:MAX_INDEXED_CHARS],
+                page_count=pdf_result["page_count"],
+            )
+        else:
+            continue
+
+        manifest_entries.append(entry)
+
+    manifest_entries.sort(key=lambda e: e.relpath.lower())
+    return {"files": [e.to_dict() for e in manifest_entries]}

docs_site-0.1.1/src/docs_site/pdf_processing.py

@@ -0,0 +1,69 @@
+"""Everything needed to index one PDF file (no rendering, the browser's
+native PDF viewer handles that inside the iframe)."""
+
+from pathlib import Path
+from typing import Any, TypedDict
+
+from pypdf import PdfReader
+from pypdf.errors import PyPdfError
+
+from .types import Heading
+
+
+class PdfResult(TypedDict):
+    title: str
+    headings: list[Heading]
+    text: str
+    page_count: int
+
+
+def _extract_headings(
+    reader: PdfReader, outline: list[Any], level: int = 1
+) -> list[Heading]:
+    headings: list[Heading] = []
+    for item in outline:
+        if isinstance(item, list):
+            headings.extend(_extract_headings(reader, item, level + 1))
+        else:
+            try:
+                page_num = reader.get_destination_page_number(item)
+                title = getattr(item, "title", "") or ""
+                if page_num is not None and title.strip():
+                    headings.append(
+                        {"level": level, "page": page_num + 1, "text": title.strip()}
+                    )
+            except (PyPdfError, AttributeError, TypeError, ValueError):
+                continue
+    return headings
+
+
+def process_pdf_file(path: Path) -> PdfResult:
+    reader = PdfReader(str(path))
+    metadata = reader.metadata
+    raw_title = metadata.title if metadata else ""
+    title = raw_title.strip() if raw_title else path.stem
+
+    try:
+        outline = reader.outline or []
+        headings = _extract_headings(reader, outline)
+    except (PyPdfError, AttributeError, TypeError, ValueError):
+        headings = []
+
+    page_count = len(reader.pages)
+    text_chunks: list[str] = []
+    for page in reader.pages:
+        try:
+            page_text = page.extract_text()
+            if page_text:
+                text_chunks.append(page_text)
+        except (PyPdfError, AttributeError, TypeError, ValueError):
+            continue
+
+    plain_text = " ".join(text_chunks)
+
+    return {
+        "title": title,
+        "headings": headings,
+        "text": plain_text,
+        "page_count": page_count,
+    }