loremfile 0.1.0__tar.gz

loremfile-0.1.0/PKG-INFO

@@ -0,0 +1,95 @@
+Metadata-Version: 2.4
+Name: loremfile
+Version: 0.1.0
+Summary: Download CC0 sample files from loremfile.dev and verify every byte
+Author: the loremfile.dev contributors
+License-Expression: MIT
+Project-URL: Homepage, https://loremfile.dev
+Project-URL: Source, https://github.com/kumarprabhashanand/loremfile
+Project-URL: Issues, https://github.com/kumarprabhashanand/loremfile/issues
+Keywords: fixtures,test-data,sample-files,cc0,download
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development :: Testing
+Classifier: Environment :: Console
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+
+# loremfile
+
+Download [loremfile.dev](https://loremfile.dev) sample files and check every byte against
+the published manifest.
+
+```console
+$ pip install loremfile
+$ loremfile get pdf/minimal.pdf mp4/720p-5s.mp4
+  written  mp4/720p-5s.mp4  1,974,574 bytes
+  written  pdf/minimal.pdf  317 bytes
+files=2, dest=loremfile-fixtures
+```
+
+Everything it does can be done with `curl` and `sha256sum -c`. What it adds is doing it
+correctly by default: every file is verified, and a file that fails is not written.
+
+## Commands
+
+| Command | |
+|---|---|
+| `loremfile get PATH…` | Download those fixtures into `--dest` (default `loremfile-fixtures`), verifying each. |
+| `loremfile get --format svg --format pdf` | The same for every published file of a format. |
+| `loremfile list [--format F] [--tag T] [--max-bytes N]` | What is published, without downloading. |
+| `loremfile verify [PATH…]` | Check files already on disk against the manifest. |
+
+Every command takes `--json` for scripts, `--quiet`, and `--catalog-version X.Y.Z` to
+refuse to run if loremfile.dev has moved on. `get` also takes `--force` (overwrite) and
+`--dry-run` (what it would fetch, and how many bytes).
+
+Exit codes: `0` fine, `1` a file failed verification or is missing, `2` the request was
+wrong, `3` loremfile.dev could not be read.
+
+## What it will not do
+
+- **Write a file it could not verify.** There is no `--no-verify`.
+- **Talk to any host but loremfile.dev.** No mirror option, and redirects off the host are
+  not followed.
+- **Take credentials.** Nothing it does needs authentication, so it accepts none.
+- **Exceed the published rate limit**: one file at a time, a pause between them, and a
+  429 is waited out rather than retried tightly.
+- **Generate fixtures.** Those are built in a pinned toolchain image; a client that made
+  its own would produce different bytes and undermine the only promise the project makes.
+
+## Python versions
+
+3.11, 3.12, 3.13 and 3.14, tested in CI on Linux, macOS and Windows. The floor is the
+oldest CPython still receiving security fixes when a release is cut; a new one is
+supported within six weeks of its release, and an end-of-life one is dropped in the next
+minor release.
+
+Zero dependencies — standard library only.
+
+## In GitHub Actions, prefer the action
+
+```yaml
+- uses: kumarprabhashanand/loremfile/action@action-v1
+  with:
+    paths: pdf/minimal.pdf mp4/720p-5s.mp4
+```
+
+It is shell only and needs no install step. This client is for everywhere else: a laptop,
+a Makefile, a CI system that is not Actions, or Windows — which the action does not
+support because it needs `sha256sum`.
+
+## One name, two tools
+
+The console script installed here is `loremfile`, which is also the name of the
+repository's own development CLI (the one that generates and publishes the fixtures).
+They are never installed together — that tool lives only inside the project's toolchain
+image — but a contributor who installs both into one environment gets whichever was
+installed last. The published package contains the client only: no generators, no
+validators, no uploader, no infrastructure code.
+
+## Licence
+
+MIT for the client; the files it downloads are CC0.

loremfile-0.1.0/README.md

@@ -0,0 +1,76 @@
+# loremfile
+
+Download [loremfile.dev](https://loremfile.dev) sample files and check every byte against
+the published manifest.
+
+```console
+$ pip install loremfile
+$ loremfile get pdf/minimal.pdf mp4/720p-5s.mp4
+  written  mp4/720p-5s.mp4  1,974,574 bytes
+  written  pdf/minimal.pdf  317 bytes
+files=2, dest=loremfile-fixtures
+```
+
+Everything it does can be done with `curl` and `sha256sum -c`. What it adds is doing it
+correctly by default: every file is verified, and a file that fails is not written.
+
+## Commands
+
+| Command | |
+|---|---|
+| `loremfile get PATH…` | Download those fixtures into `--dest` (default `loremfile-fixtures`), verifying each. |
+| `loremfile get --format svg --format pdf` | The same for every published file of a format. |
+| `loremfile list [--format F] [--tag T] [--max-bytes N]` | What is published, without downloading. |
+| `loremfile verify [PATH…]` | Check files already on disk against the manifest. |
+
+Every command takes `--json` for scripts, `--quiet`, and `--catalog-version X.Y.Z` to
+refuse to run if loremfile.dev has moved on. `get` also takes `--force` (overwrite) and
+`--dry-run` (what it would fetch, and how many bytes).
+
+Exit codes: `0` fine, `1` a file failed verification or is missing, `2` the request was
+wrong, `3` loremfile.dev could not be read.
+
+## What it will not do
+
+- **Write a file it could not verify.** There is no `--no-verify`.
+- **Talk to any host but loremfile.dev.** No mirror option, and redirects off the host are
+  not followed.
+- **Take credentials.** Nothing it does needs authentication, so it accepts none.
+- **Exceed the published rate limit**: one file at a time, a pause between them, and a
+  429 is waited out rather than retried tightly.
+- **Generate fixtures.** Those are built in a pinned toolchain image; a client that made
+  its own would produce different bytes and undermine the only promise the project makes.
+
+## Python versions
+
+3.11, 3.12, 3.13 and 3.14, tested in CI on Linux, macOS and Windows. The floor is the
+oldest CPython still receiving security fixes when a release is cut; a new one is
+supported within six weeks of its release, and an end-of-life one is dropped in the next
+minor release.
+
+Zero dependencies — standard library only.
+
+## In GitHub Actions, prefer the action
+
+```yaml
+- uses: kumarprabhashanand/loremfile/action@action-v1
+  with:
+    paths: pdf/minimal.pdf mp4/720p-5s.mp4
+```
+
+It is shell only and needs no install step. This client is for everywhere else: a laptop,
+a Makefile, a CI system that is not Actions, or Windows — which the action does not
+support because it needs `sha256sum`.
+
+## One name, two tools
+
+The console script installed here is `loremfile`, which is also the name of the
+repository's own development CLI (the one that generates and publishes the fixtures).
+They are never installed together — that tool lives only inside the project's toolchain
+image — but a contributor who installs both into one environment gets whichever was
+installed last. The published package contains the client only: no generators, no
+validators, no uploader, no infrastructure code.
+
+## Licence
+
+MIT for the client; the files it downloads are CC0.

loremfile-0.1.0/pyproject.toml

@@ -0,0 +1,51 @@
+# The published client, built from this directory alone.
+#
+# The split is the point: the repository's own tooling — generators, validators, the
+# uploader, the infrastructure client — lives in `src/loremfile/` at the repository root
+# and is never published. This tree holds the client and nothing else, so the wheel
+# cannot carry the deploy tooling by accident; `tests/test_packaging.py` reads the built
+# wheel and says so.
+[build-system]
+requires = ["setuptools>=77"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "loremfile"
+version = "0.1.0"
+description = "Download CC0 sample files from loremfile.dev and verify every byte"
+readme = "README.md"
+# A range, and a policy: the floor is the oldest CPython still receiving security fixes
+# when a release is cut. 3.10 reaches end of life on 2026-10-31 and starts out rather
+# than shipping and immediately going; every version below is tested in CI, on Linux,
+# macOS and Windows. A version nobody tests is not supported.
+requires-python = ">=3.11"
+license = "MIT"
+authors = [{ name = "the loremfile.dev contributors" }]
+keywords = ["fixtures", "test-data", "sample-files", "cc0", "download"]
+classifiers = [
+  "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
+  "Programming Language :: Python :: 3.13",
+  "Programming Language :: Python :: 3.14",
+  "Topic :: Software Development :: Testing",
+  "Environment :: Console",
+]
+# Deliberately empty, and a decision rather than an omission: the client is standard
+# library only. A request that needs a dependency is a conversation, not a patch.
+dependencies = []
+
+[project.scripts]
+loremfile = "loremfile_client.cli:main"
+
+[project.urls]
+Homepage = "https://loremfile.dev"
+Source = "https://github.com/kumarprabhashanand/loremfile"
+Issues = "https://github.com/kumarprabhashanand/loremfile/issues"
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+# So that `pytest` inside this directory is this package's suite, not the repository's.
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-ra"

loremfile-0.1.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

loremfile-0.1.0/src/loremfile.egg-info/PKG-INFO

@@ -0,0 +1,95 @@
+Metadata-Version: 2.4
+Name: loremfile
+Version: 0.1.0
+Summary: Download CC0 sample files from loremfile.dev and verify every byte
+Author: the loremfile.dev contributors
+License-Expression: MIT
+Project-URL: Homepage, https://loremfile.dev
+Project-URL: Source, https://github.com/kumarprabhashanand/loremfile
+Project-URL: Issues, https://github.com/kumarprabhashanand/loremfile/issues
+Keywords: fixtures,test-data,sample-files,cc0,download
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Software Development :: Testing
+Classifier: Environment :: Console
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+
+# loremfile
+
+Download [loremfile.dev](https://loremfile.dev) sample files and check every byte against
+the published manifest.
+
+```console
+$ pip install loremfile
+$ loremfile get pdf/minimal.pdf mp4/720p-5s.mp4
+  written  mp4/720p-5s.mp4  1,974,574 bytes
+  written  pdf/minimal.pdf  317 bytes
+files=2, dest=loremfile-fixtures
+```
+
+Everything it does can be done with `curl` and `sha256sum -c`. What it adds is doing it
+correctly by default: every file is verified, and a file that fails is not written.
+
+## Commands
+
+| Command | |
+|---|---|
+| `loremfile get PATH…` | Download those fixtures into `--dest` (default `loremfile-fixtures`), verifying each. |
+| `loremfile get --format svg --format pdf` | The same for every published file of a format. |
+| `loremfile list [--format F] [--tag T] [--max-bytes N]` | What is published, without downloading. |
+| `loremfile verify [PATH…]` | Check files already on disk against the manifest. |
+
+Every command takes `--json` for scripts, `--quiet`, and `--catalog-version X.Y.Z` to
+refuse to run if loremfile.dev has moved on. `get` also takes `--force` (overwrite) and
+`--dry-run` (what it would fetch, and how many bytes).
+
+Exit codes: `0` fine, `1` a file failed verification or is missing, `2` the request was
+wrong, `3` loremfile.dev could not be read.
+
+## What it will not do
+
+- **Write a file it could not verify.** There is no `--no-verify`.
+- **Talk to any host but loremfile.dev.** No mirror option, and redirects off the host are
+  not followed.
+- **Take credentials.** Nothing it does needs authentication, so it accepts none.
+- **Exceed the published rate limit**: one file at a time, a pause between them, and a
+  429 is waited out rather than retried tightly.
+- **Generate fixtures.** Those are built in a pinned toolchain image; a client that made
+  its own would produce different bytes and undermine the only promise the project makes.
+
+## Python versions
+
+3.11, 3.12, 3.13 and 3.14, tested in CI on Linux, macOS and Windows. The floor is the
+oldest CPython still receiving security fixes when a release is cut; a new one is
+supported within six weeks of its release, and an end-of-life one is dropped in the next
+minor release.
+
+Zero dependencies — standard library only.
+
+## In GitHub Actions, prefer the action
+
+```yaml
+- uses: kumarprabhashanand/loremfile/action@action-v1
+  with:
+    paths: pdf/minimal.pdf mp4/720p-5s.mp4
+```
+
+It is shell only and needs no install step. This client is for everywhere else: a laptop,
+a Makefile, a CI system that is not Actions, or Windows — which the action does not
+support because it needs `sha256sum`.
+
+## One name, two tools
+
+The console script installed here is `loremfile`, which is also the name of the
+repository's own development CLI (the one that generates and publishes the fixtures).
+They are never installed together — that tool lives only inside the project's toolchain
+image — but a contributor who installs both into one environment gets whichever was
+installed last. The published package contains the client only: no generators, no
+validators, no uploader, no infrastructure code.
+
+## Licence
+
+MIT for the client; the files it downloads are CC0.

loremfile-0.1.0/src/loremfile.egg-info/SOURCES.txt

@@ -0,0 +1,14 @@
+README.md
+pyproject.toml
+src/loremfile.egg-info/PKG-INFO
+src/loremfile.egg-info/SOURCES.txt
+src/loremfile.egg-info/dependency_links.txt
+src/loremfile.egg-info/entry_points.txt
+src/loremfile.egg-info/top_level.txt
+src/loremfile_client/__init__.py
+src/loremfile_client/__main__.py
+src/loremfile_client/api.py
+src/loremfile_client/cli.py
+tests/test_commands.py
+tests/test_only_loremfile_dev.py
+tests/test_packaging.py

loremfile-0.1.0/src/loremfile.egg-info/dependency_links.txt

@@ -0,0 +1 @@
+

loremfile-0.1.0/src/loremfile.egg-info/entry_points.txt

@@ -0,0 +1,2 @@
+[console_scripts]
+loremfile = loremfile_client.cli:main

loremfile-0.1.0/src/loremfile.egg-info/top_level.txt

@@ -0,0 +1 @@
+loremfile_client

loremfile-0.1.0/src/loremfile_client/__init__.py

@@ -0,0 +1,3 @@
+"""A small client for loremfile.dev: download CC0 sample files and verify every byte."""
+
+__version__ = "0.1.0"

loremfile-0.1.0/src/loremfile_client/__main__.py

@@ -0,0 +1,3 @@
+from loremfile_client.cli import main
+
+raise SystemExit(main())

loremfile-0.1.0/src/loremfile_client/api.py

@@ -0,0 +1,246 @@
+"""Fetching fixtures from loremfile.dev, and checking every byte that arrives.
+
+Standard library only, on purpose (the CLI spec): a client whose job is to download a
+file and hash it should not ask anyone to resolve a dependency tree, and the smaller the
+supply chain the less there is to audit.
+
+The one thing to know about this module: `BASE_URL` is a constant. There is no flag, no
+configuration file and no environment variable that points the client at another host —
+only a loopback override this project's own tests inject, which is rejected for anything
+that is not 127.0.0.1, ::1 or localhost. A mirror flag would contradict the README's
+"do not rely on any other mirror", and an override that accepted a public host would be a
+way to make `loremfile get` fetch from somewhere else entirely.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import http.client
+import json
+import os
+import tempfile
+import time
+import urllib.error
+import urllib.request
+from collections.abc import Iterator
+from dataclasses import dataclass
+from pathlib import Path, PurePosixPath
+from typing import Any, cast
+
+#: The only host this client talks to.
+BASE_URL = "https://loremfile.dev/"
+
+#: Injected by this project's tests so they never touch production. Loopback only.
+OVERRIDE = "LOREMFILE_BASE_URL"
+LOOPBACK = ("http://127.0.0.1:", "http://localhost:", "http://[::1]:")
+
+#: The published rate limit is 30 requests a second per client; one file at a time with a
+#: pause between them is well under it, and a 429 is answered with patience.
+PAUSE_SECONDS = 0.5
+BACKOFF_SECONDS = (2, 4, 8)
+TIMEOUT_SECONDS = 300
+USER_AGENT = "loremfile-client"
+
+#: Read and hashed a megabyte at a time, so a 100 MB fixture costs a megabyte of memory.
+CHUNK_BYTES = 1024 * 1024
+
+
+class Refused(Exception):
+    """The request was wrong: an unknown path, a bad override, a version that has moved."""
+
+
+class Unreachable(Exception):
+    """loremfile.dev could not be read."""
+
+
+class Mismatch(Exception):
+    """Bytes arrived that are not the bytes the manifest describes."""
+
+
+def base_url() -> str:
+    override = os.environ.get(OVERRIDE, "").strip()
+    if not override:
+        return BASE_URL
+    if not override.startswith(LOOPBACK):
+        raise Refused(
+            f"{OVERRIDE} accepts a loopback address only, so that this project's tests can "
+            f"run without touching production; {override!r} is not one. There is no way to "
+            "point this client at another host, by design."
+        )
+    return override if override.endswith("/") else override + "/"
+
+
+def _open(path: str) -> http.client.HTTPResponse:
+    """An open response, retrying only a 429 — our own rate limit asking for patience.
+
+    The retry is here, around opening, because that is where a 429 arrives. A stream
+    that fails halfway is not retried: the caller would have to decide what to do with
+    the bytes it already has, and this client's answer to a half-file is to delete it.
+    """
+    url = base_url() + path.lstrip("/")
+    request = urllib.request.Request(url, headers={"User-Agent": USER_AGENT})  # noqa: S310
+    for pause in (*BACKOFF_SECONDS, None):
+        try:
+            response = urllib.request.urlopen(request, timeout=TIMEOUT_SECONDS)  # noqa: S310
+        except urllib.error.HTTPError as exc:
+            if exc.code == 429 and pause is not None:  # noqa: PLR2004 - the HTTP status
+                time.sleep(pause)
+                continue
+            raise Unreachable(f"{url} answered {exc.code}") from exc
+        except urllib.error.URLError as exc:
+            raise Unreachable(f"{url} could not be read: {exc.reason}") from exc
+        if response.geturl().split("/")[2] != url.split("/")[2]:
+            response.close()
+            raise Refused(f"{url} redirected off loremfile.dev, which is never followed")
+        # urlopen is typed as returning Any for non-HTTP schemes; this client only ever
+        # opens http(s), which `base_url` is the single gate for. A cast rather than an
+        # assert: `python -O` strips asserts, and shipped code should not depend on a
+        # statement that may not be there.
+        return cast("http.client.HTTPResponse", response)
+    raise Unreachable(f"{url} is still answering 429")
+
+
+def fetch(path: str) -> bytes:
+    """The whole body at once. For the manifest, which is JSON and has to be parsed."""
+    with _open(path) as response:
+        return bytes(response.read())
+
+
+def stream(path: str) -> Iterator[bytes]:
+    """The body in chunks, for fixtures — which run to 100 MB.
+
+    Reading one of those into memory to hash it would be 100 MB of a CI container's
+    often 512 MB, for no benefit: sha256 is happy to be fed a chunk at a time.
+    """
+    with _open(path) as response:
+        while chunk := response.read(CHUNK_BYTES):
+            yield bytes(chunk)
+
+
+def manifest(*, catalog_version: str | None = None) -> dict[str, Any]:
+    try:
+        document = json.loads(fetch("manifest.json"))
+    except json.JSONDecodeError as exc:
+        raise Unreachable(f"the manifest is not JSON: {exc}") from exc
+    if not isinstance(document, dict):
+        raise Unreachable("the manifest is not a JSON object")
+    published = str(document.get("catalog_version", ""))
+    if catalog_version and published != catalog_version:
+        raise Refused(f"loremfile.dev publishes catalog {published}, not {catalog_version}")
+    return dict(document)
+
+
+def active(document: dict[str, Any]) -> list[dict[str, Any]]:
+    return [e for e in document.get("fixtures", []) if e.get("status", "active") == "active"]
+
+
+def select(
+    entries: list[dict[str, Any]],
+    *,
+    paths: list[str] | None = None,
+    formats: list[str] | None = None,
+) -> list[dict[str, Any]]:
+    """The entries a request names, or `Refused` naming what does not exist."""
+    wanted_paths, wanted_formats = set(paths or []), set(formats or [])
+    if not wanted_paths and not wanted_formats:
+        raise Refused("name at least one path or format")
+    by_path = {e["path"]: e for e in entries}
+    unknown = sorted(wanted_paths - set(by_path))
+    if unknown:
+        raise Refused(f"not published: {', '.join(unknown)}")
+    published_formats = {e["format"] for e in entries}
+    unknown_formats = sorted(wanted_formats - published_formats)
+    if unknown_formats:
+        raise Refused(f"no such format: {', '.join(unknown_formats)}")
+    chosen = {
+        e["path"]: e for e in entries if e["path"] in wanted_paths or e["format"] in wanted_formats
+    }
+    return [chosen[path] for path in sorted(chosen)]
+
+
+def target(dest: Path, path: str) -> Path:
+    """Where a fixture lands, refusing anything that would escape `dest`.
+
+    The paths come from our own manifest, so this is not the day's most likely failure —
+    but a client that writes where a downloaded document tells it to is the shape of
+    problem this project publishes a fixture about (`edge/zip-directory-traversal-name.zip`).
+    """
+    relative = PurePosixPath(path)
+    if relative.is_absolute() or ".." in relative.parts:
+        raise Refused(f"refusing {path!r}: not the shape of a fixture path")
+    resolved = (dest / Path(*relative.parts)).resolve()
+    if not str(resolved).startswith(str(dest.resolve())):
+        raise Refused(f"refusing {path!r}: it would be written outside {dest}")
+    return resolved
+
+
+def pace() -> None:
+    """The wait between two downloads. One place, so a test can make it instant and the
+    rate manners stay a property of this module rather than of whoever loops."""
+    time.sleep(PAUSE_SECONDS)
+
+
+@dataclass(frozen=True)
+class Result:
+    """What happened to one fixture."""
+
+    path: str
+    status: str  # written | skipped | ok | missing | changed
+    detail: str = ""
+
+    @property
+    def ok(self) -> bool:
+        return self.status in {"written", "skipped", "ok"}
+
+
+def download(entry: dict[str, Any], dest: Path, *, force: bool = False) -> Result:
+    """Fetch one fixture, hashing as it arrives, and put it at its path only if it matches.
+
+    The bytes go to a temporary file **in the destination directory**, not to memory and
+    not to the system temporary directory: the first would cost 100 MB of RAM for the
+    largest fixtures, and the second could be on another filesystem, where the final move
+    would be a copy rather than a rename. `Path.replace` within one directory is atomic,
+    so a reader of that directory sees either no file or the whole verified one.
+
+    The guarantee is unchanged: nothing unverified is ever at the published path. What is
+    new is that nothing unverified survives anywhere — the temporary file is removed on a
+    mismatch, on an interrupted download, and on any other failure.
+    """
+    destination = target(dest, entry["path"])
+    if destination.exists() and not force:
+        return Result(entry["path"], "skipped", "already here; --force overwrites")
+    destination.parent.mkdir(parents=True, exist_ok=True)
+
+    digest = hashlib.sha256()
+    written = 0
+    handle = tempfile.NamedTemporaryFile(  # noqa: SIM115 - closed by the `with` below
+        dir=destination.parent, prefix=f".{destination.name}.", suffix=".part", delete=False
+    )
+    partial = Path(handle.name)
+    try:
+        with handle:
+            for chunk in stream(entry["path"]):
+                digest.update(chunk)
+                written += handle.write(chunk)
+        if digest.hexdigest() != entry["sha256"]:
+            raise Mismatch(
+                f"{entry['path']}: downloaded sha256 {digest.hexdigest()}, "
+                f"the manifest says {entry['sha256']}"
+            )
+        partial.replace(destination)
+    finally:
+        # A no-op once the rename has happened, and the whole point otherwise.
+        partial.unlink(missing_ok=True)
+    return Result(entry["path"], "written", f"{written:,} bytes")
+
+
+def verify(entry: dict[str, Any], dest: Path) -> Result:
+    """Check a fixture already on disk against the published manifest."""
+    destination = target(dest, entry["path"])
+    if not destination.is_file():
+        return Result(entry["path"], "missing", f"not in {dest}")
+    digest = hashlib.sha256(destination.read_bytes()).hexdigest()
+    if digest != entry["sha256"]:
+        expected = str(entry["sha256"])[:12]
+        return Result(entry["path"], "changed", f"sha256 {digest[:12]}, expected {expected}")
+    return Result(entry["path"], "ok", f"{destination.stat().st_size:,} bytes")

loremfile-0.1.0/src/loremfile_client/cli.py

@@ -0,0 +1,150 @@
+"""`loremfile get | list | verify` (the CLI spec).
+
+argparse rather than a framework, because the package has no dependencies and this is
+three subcommands. Exit codes are the contract: 0 fine, 1 a file failed verification or
+is missing, 2 the request was wrong, 3 loremfile.dev could not be read.
+"""
+
+from __future__ import annotations
+
+import argparse
+import json
+import sys
+from collections.abc import Sequence
+from pathlib import Path
+from typing import Any
+
+from loremfile_client import __version__, api
+
+OK, FAILED, REFUSED, UNREACHABLE = 0, 1, 2, 3
+DEFAULT_DEST = "loremfile-fixtures"
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="loremfile",
+        description="Download CC0 sample files from loremfile.dev and check every byte.",
+        epilog="Files come from loremfile.dev and nowhere else; there is no mirror option.",
+    )
+    parser.add_argument("--version", action="version", version=f"loremfile {__version__}")
+    subcommands = parser.add_subparsers(dest="command", required=True)
+
+    def shared(sub: argparse.ArgumentParser) -> None:
+        sub.add_argument("--json", action="store_true", help="Print one JSON object.")
+        sub.add_argument("--quiet", action="store_true", help="Print nothing but failures.")
+        sub.add_argument(
+            "--catalog-version",
+            metavar="X.Y.Z",
+            help="Refuse to run unless loremfile.dev publishes this catalog version.",
+        )
+
+    get = subcommands.add_parser("get", help="Download fixtures and verify them.")
+    get.add_argument("paths", nargs="*", metavar="PATH", help="e.g. pdf/minimal.pdf")
+    get.add_argument(
+        "--format",
+        action="append",
+        default=[],
+        metavar="FMT",
+        help="Every published file of this format. Repeatable.",
+    )
+    get.add_argument("--dest", default=DEFAULT_DEST, type=Path, help=f"Default: {DEFAULT_DEST}")
+    get.add_argument("--force", action="store_true", help="Overwrite files already there.")
+    get.add_argument("--dry-run", action="store_true", help="List what would be fetched.")
+    shared(get)
+
+    listing = subcommands.add_parser("list", help="What is published, without downloading.")
+    listing.add_argument("--format", action="append", default=[], metavar="FMT")
+    listing.add_argument("--tag", action="append", default=[], metavar="TAG")
+    listing.add_argument("--max-bytes", type=int, metavar="N")
+    shared(listing)
+
+    check = subcommands.add_parser("verify", help="Check files on disk against the manifest.")
+    check.add_argument("paths", nargs="*", metavar="PATH")
+    check.add_argument("--format", action="append", default=[], metavar="FMT")
+    check.add_argument("--dest", default=DEFAULT_DEST, type=Path, help=f"Default: {DEFAULT_DEST}")
+    shared(check)
+    return parser
+
+
+def emit(args: argparse.Namespace, summary: dict[str, Any], results: list[api.Result]) -> None:
+    if args.json:
+        print(json.dumps({"summary": summary, "items": [vars(r) for r in results]}, indent=2))
+        return
+    for result in results:
+        if not result.ok or not args.quiet:
+            print(
+                f"  {result.status:8} {result.path}{'  ' + result.detail if result.detail else ''}"
+            )
+    if not args.quiet:
+        print(", ".join(f"{key}={value}" for key, value in summary.items()))
+
+
+def run_get(args: argparse.Namespace) -> int:
+    entries = api.select(
+        api.active(api.manifest(catalog_version=args.catalog_version)),
+        paths=args.paths,
+        formats=args.format,
+    )
+    if args.dry_run:
+        total = sum(e["bytes"] for e in entries)
+        emit(
+            args,
+            {"files": len(entries), "bytes": total, "dest": str(args.dest), "dry-run": True},
+            [api.Result(e["path"], "would fetch", f"{e['bytes']:,} bytes") for e in entries],
+        )
+        return OK
+    results = []
+    for index, entry in enumerate(entries):
+        results.append(api.download(entry, args.dest, force=args.force))
+        if index + 1 < len(entries):
+            api.pace()
+    emit(args, {"files": len(results), "dest": str(args.dest)}, results)
+    return OK
+
+
+def run_list(args: argparse.Namespace) -> int:
+    entries = api.active(api.manifest(catalog_version=args.catalog_version))
+    if args.format:
+        entries = [e for e in entries if e["format"] in set(args.format)]
+    if args.tag:
+        entries = [e for e in entries if set(args.tag) & set(e.get("tags", []))]
+    if args.max_bytes is not None:
+        entries = [e for e in entries if e["bytes"] <= args.max_bytes]
+    results = [
+        api.Result(e["path"], "listed", f"{e['bytes']:,} bytes  {e['mime']}") for e in entries
+    ]
+    emit(args, {"files": len(results)}, results)
+    return OK
+
+
+def run_verify(args: argparse.Namespace) -> int:
+    entries = api.active(api.manifest(catalog_version=args.catalog_version))
+    if args.paths or args.format:
+        entries = api.select(entries, paths=args.paths, formats=args.format)
+    else:
+        # Everything the destination actually holds, so `verify` alone checks a directory.
+        entries = [e for e in entries if (args.dest / Path(e["path"])).is_file()]
+    results = [api.verify(entry, args.dest) for entry in entries]
+    failing = [r for r in results if not r.ok]
+    emit(args, {"checked": len(results), "failing": len(failing)}, results)
+    return FAILED if failing else OK
+
+
+def main(argv: Sequence[str] | None = None) -> int:
+    args = build_parser().parse_args(argv)
+    runner = {"get": run_get, "list": run_list, "verify": run_verify}[args.command]
+    try:
+        return runner(args)
+    except api.Refused as exc:
+        print(f"loremfile: {exc}", file=sys.stderr)
+        return REFUSED
+    except api.Mismatch as exc:
+        print(f"loremfile: {exc}", file=sys.stderr)
+        return FAILED
+    except api.Unreachable as exc:
+        print(f"loremfile: {exc}", file=sys.stderr)
+        return UNREACHABLE
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

loremfile-0.1.0/tests/test_commands.py

@@ -0,0 +1,184 @@
+"""`get`, `list` and `verify` against the loopback server (conftest.py)."""
+
+from __future__ import annotations
+
+import json
+from pathlib import Path
+
+import pytest
+from loremfile_client import api, cli
+
+from conftest import BODIES, Handler
+
+
+def run(*argv: str) -> int:
+    return cli.main(list(argv))
+
+
+def test_get_writes_the_files_it_verified(server: str, dest: Path) -> None:
+    assert run("get", "pdf/small.pdf", "txt/notes.txt", "--dest", str(dest)) == cli.OK
+    assert (dest / "pdf" / "small.pdf").read_bytes() == BODIES["pdf/small.pdf"]
+    assert (dest / "txt" / "notes.txt").read_bytes() == BODIES["txt/notes.txt"]
+
+
+def test_get_by_format_takes_every_file_of_it(server: str, dest: Path) -> None:
+    assert run("get", "--format", "pdf", "--dest", str(dest)) == cli.OK
+    assert sorted(p.name for p in (dest / "pdf").iterdir()) == ["other.pdf", "small.pdf"]
+
+
+def test_a_withdrawn_entry_is_not_offered(server: str, dest: Path) -> None:
+    assert run("get", "pdf/withdrawn.pdf", "--dest", str(dest)) == cli.REFUSED
+
+
+def test_bytes_that_do_not_match_are_not_written(server: str, dest: Path) -> None:
+    """The reason the package exists: a file that fails its hash must not reach the disk,
+    and the command must not report success."""
+    Handler.corrupt = {"pdf/small.pdf"}
+    assert run("get", "pdf/small.pdf", "--dest", str(dest)) == cli.FAILED
+    assert not (dest / "pdf" / "small.pdf").exists()
+
+
+def test_an_unknown_path_is_refused_before_anything_is_fetched(server: str, dest: Path) -> None:
+    assert run("get", "pdf/not-real.pdf", "--dest", str(dest)) == cli.REFUSED
+    assert not dest.exists()
+
+
+def test_naming_nothing_is_refused(server: str, dest: Path) -> None:
+    assert run("get", "--dest", str(dest)) == cli.REFUSED
+
+
+def test_a_catalog_version_that_has_moved_on_is_refused(server: str, dest: Path) -> None:
+    assert (
+        run("get", "pdf/small.pdf", "--dest", str(dest), "--catalog-version", "1.0.0")
+        == cli.REFUSED
+    )
+    assert run("get", "pdf/small.pdf", "--dest", str(dest), "--catalog-version", "9.9.9") == cli.OK
+
+
+def test_an_existing_file_is_kept_unless_force(server: str, dest: Path) -> None:
+    assert run("get", "pdf/small.pdf", "--dest", str(dest)) == cli.OK
+    (dest / "pdf" / "small.pdf").write_bytes(b"mine")
+    assert run("get", "pdf/small.pdf", "--dest", str(dest)) == cli.OK
+    assert (dest / "pdf" / "small.pdf").read_bytes() == b"mine", "it overwrote without --force"
+    assert run("get", "pdf/small.pdf", "--dest", str(dest), "--force") == cli.OK
+    assert (dest / "pdf" / "small.pdf").read_bytes() == BODIES["pdf/small.pdf"]
+
+
+def test_a_dry_run_fetches_nothing(server: str, dest: Path, capsys: pytest.CaptureFixture) -> None:
+    assert run("get", "--format", "pdf", "--dest", str(dest), "--dry-run", "--json") == cli.OK
+    report = json.loads(capsys.readouterr().out)
+    assert report["summary"]["files"] == 2
+    assert report["summary"]["bytes"] == sum(
+        len(BODIES[p]) for p in ("pdf/small.pdf", "pdf/other.pdf")
+    )
+    assert not dest.exists()
+
+
+def test_list_filters_without_downloading(server: str, capsys: pytest.CaptureFixture) -> None:
+    assert run("list", "--format", "pdf", "--json") == cli.OK
+    assert {item["path"] for item in json.loads(capsys.readouterr().out)["items"]} == {
+        "pdf/small.pdf",
+        "pdf/other.pdf",
+    }
+    assert run("list", "--max-bytes", "13", "--json") == cli.OK
+    listed = {item["path"] for item in json.loads(capsys.readouterr().out)["items"]}
+    assert listed == {"txt/notes.txt"}, listed
+
+
+def test_verify_reports_missing_and_changed_files(server: str, dest: Path) -> None:
+    assert run("get", "--format", "pdf", "--dest", str(dest)) == cli.OK
+    assert run("verify", "--dest", str(dest)) == cli.OK
+
+    (dest / "pdf" / "small.pdf").write_bytes(b"not what was published")
+    assert run("verify", "--dest", str(dest)) == cli.FAILED
+
+    (dest / "pdf" / "small.pdf").unlink()
+    assert run("verify", "pdf/small.pdf", "--dest", str(dest)) == cli.FAILED
+
+
+def test_verify_on_an_empty_directory_checks_nothing_and_says_so(
+    server: str, dest: Path, capsys: pytest.CaptureFixture
+) -> None:
+    """A verify that finds no files must not report success over an empty set silently."""
+    dest.mkdir(parents=True)
+    assert run("verify", "--dest", str(dest), "--json") == cli.OK
+    assert json.loads(capsys.readouterr().out)["summary"] == {"checked": 0, "failing": 0}
+
+
+def test_a_host_that_is_not_answering_is_a_separate_exit_code(
+    monkeypatch: pytest.MonkeyPatch, dest: Path
+) -> None:
+    """Told apart from a verification failure on purpose: one is a broken network, the
+    other is a broken file, and a script should be able to retry only the first."""
+    monkeypatch.setenv(api.OVERRIDE, "http://127.0.0.1:1/")
+    assert run("get", "pdf/small.pdf", "--dest", str(dest)) == cli.UNREACHABLE
+
+
+def test_a_fixture_path_that_would_escape_the_destination_is_refused(dest: Path) -> None:
+    with pytest.raises(api.Refused, match="not the shape of a fixture path"):
+        api.target(dest, "../elsewhere.txt")
+    with pytest.raises(api.Refused, match="not the shape of a fixture path"):
+        api.target(dest, "/etc/passwd")
+
+
+# --- how the bytes arrive (a 100 MB fixture must not be a 100 MB process) ----------------
+
+
+def test_the_body_is_streamed_rather_than_held_in_memory(
+    server: str, dest: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """`fetch` reads a whole body at once and is for the manifest only. If a fixture ever
+    goes through it again, the largest ones are 100 MB of a 512 MB container."""
+    whole = api.fetch
+
+    def only_the_manifest(path: str) -> bytes:
+        if path != "manifest.json":
+            raise AssertionError(f"{path} was read into memory instead of streamed")
+        return whole(path)
+
+    monkeypatch.setattr(api, "fetch", only_the_manifest)
+    assert run("get", "pdf/small.pdf", "--dest", str(dest)) == cli.OK
+    assert (dest / "pdf" / "small.pdf").read_bytes() == BODIES["pdf/small.pdf"]
+
+
+def test_a_mismatch_leaves_nothing_behind(server: str, dest: Path) -> None:
+    """Not only nothing at the published path: nothing at all. A half-verified file under
+    another name is still a file somebody's glob will pick up."""
+    Handler.corrupt = {"pdf/small.pdf"}
+    assert run("get", "pdf/small.pdf", "--dest", str(dest)) == cli.FAILED
+    assert sorted((dest / "pdf").iterdir()) == [], "a partial file survived the mismatch"
+
+
+def test_an_interrupted_download_leaves_nothing_behind(
+    server: str, dest: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """The same promise when the network drops rather than when the hash differs."""
+
+    def cut_off(_path: str) -> object:
+        yield b"%PDF-1.4 par"
+        raise api.Unreachable("the connection went away")
+
+    monkeypatch.setattr(api, "stream", cut_off)
+    assert run("get", "pdf/small.pdf", "--dest", str(dest)) == cli.UNREACHABLE
+    assert sorted((dest / "pdf").iterdir()) == []
+
+
+def test_the_partial_file_is_written_beside_its_destination(
+    server: str, dest: Path, monkeypatch: pytest.MonkeyPatch
+) -> None:
+    """In the destination directory, so the final move is a rename on one filesystem and
+    not a copy from /tmp — and so a crash cannot leave a stray file somewhere else."""
+    seen: list[list[str]] = []
+    streaming = api.stream
+
+    def watching(path: str):
+        for chunk in streaming(path):
+            seen.append(sorted(p.name for p in (dest / "pdf").iterdir()))
+            yield chunk
+
+    monkeypatch.setattr(api, "stream", watching)
+    assert run("get", "pdf/small.pdf", "--dest", str(dest)) == cli.OK
+    assert any(
+        names and names[0].startswith(".small.pdf") and names[0].endswith(".part") for names in seen
+    ), seen
+    assert sorted(p.name for p in (dest / "pdf").iterdir()) == ["small.pdf"]

loremfile-0.1.0/tests/test_only_loremfile_dev.py

@@ -0,0 +1,115 @@
+"""The client talks to loremfile.dev and to nothing else.
+
+Four ways of asking, because one of them alone would be easy to satisfy and still be
+wrong: the constant is the production host; the override refuses anything that is not
+loopback; no flag or environment variable reaches the base URL; and no other host appears
+in the source at all. The override exists only so the rest of this suite can run without
+touching production — the same seam, and the same loopback-only rule, as the repository's
+GitHub Action.
+"""
+
+from __future__ import annotations
+
+import ast
+import re
+from pathlib import Path
+
+import pytest
+from loremfile_client import api, cli
+
+PACKAGE = Path(api.__file__).parent
+
+
+def test_the_constant_is_production() -> None:
+    assert api.BASE_URL == "https://loremfile.dev/"
+
+
+def test_with_no_override_it_is_production(monkeypatch: pytest.MonkeyPatch) -> None:
+    monkeypatch.delenv(api.OVERRIDE, raising=False)
+    assert api.base_url() == "https://loremfile.dev/"
+
+
+@pytest.mark.parametrize(
+    "value",
+    [
+        "https://loremfile.dev.evil.test/",  # the host name as a prefix
+        "https://evil.test/loremfile.dev/",  # and as a path
+        "http://127.0.0.1.evil.test:8080/",  # and the loopback address as a prefix
+        "https://example.com/",
+        "file:///etc/passwd",
+        "http://169.254.169.254/",  # the cloud metadata address
+        " https://evil.test/",  # leading space
+    ],
+)
+def test_the_override_refuses_anything_but_loopback(
+    monkeypatch: pytest.MonkeyPatch, value: str
+) -> None:
+    monkeypatch.setenv(api.OVERRIDE, value)
+    with pytest.raises(api.Refused, match="loopback address only"):
+        api.base_url()
+
+
+@pytest.mark.parametrize(
+    "value", ["http://127.0.0.1:8080", "http://localhost:9/", "http://[::1]:1234"]
+)
+def test_the_override_accepts_loopback(monkeypatch: pytest.MonkeyPatch, value: str) -> None:
+    """The control: the refusals above are refusals, not a function that rejects everything."""
+    monkeypatch.setenv(api.OVERRIDE, value)
+    assert api.base_url().startswith(value.rstrip("/"))
+
+
+def test_no_command_line_flag_reaches_the_host() -> None:
+    """A mirror flag would contradict the README, so there must not be one — checked on
+    the parser rather than by reading the source, so a flag added later is caught."""
+    parser = cli.build_parser()
+    options = {
+        option
+        for action in parser._subparsers._group_actions[0].choices.values()
+        for action_option in action._actions
+        for option in action_option.option_strings
+    } | {option for action in parser._actions for option in action.option_strings}
+    assert not {o for o in options if re.search(r"url|host|base|mirror|origin|server", o)}, options
+
+
+def test_no_other_host_appears_in_the_source() -> None:
+    """The last way: whatever the code does with them, these are the only hosts in it."""
+    urls: set[str] = set()
+    for source in PACKAGE.glob("*.py"):
+        urls |= set(re.findall(r"https?://[A-Za-z0-9.\[\]:-]+", source.read_text(encoding="utf-8")))
+    assert urls == {
+        "https://loremfile.dev",
+        "http://127.0.0.1:",
+        "http://localhost:",
+        "http://[::1]:",
+    }, urls
+
+
+def test_the_client_imports_nothing_but_the_standard_library() -> None:
+    """Zero dependencies is a promise in the README and in pyproject; this is what keeps
+    it true. Named modules, not a count: a new import is a decision."""
+    allowed = {
+        "argparse",
+        "collections",
+        "dataclasses",
+        "hashlib",
+        "http",
+        "json",
+        "os",
+        "pathlib",
+        "sys",
+        "tempfile",
+        "time",
+        "typing",
+        "urllib",
+        "loremfile_client",
+        "__future__",
+    }
+    imported: set[str] = set()
+    for source in PACKAGE.glob("*.py"):
+        for node in ast.walk(ast.parse(source.read_text(encoding="utf-8"))):
+            if isinstance(node, ast.Import):
+                imported |= {alias.name.split(".")[0] for alias in node.names}
+            elif isinstance(node, ast.ImportFrom) and node.module and node.level == 0:
+                imported.add(node.module.split(".")[0])
+    assert imported, "empty-set control: the scan found the imports"
+    assert imported <= allowed, imported - allowed

loremfile-0.1.0/tests/test_packaging.py

@@ -0,0 +1,54 @@
+"""The wheel carries the client and nothing else (the CLI spec's packaging split).
+
+Read off the built artifact, not the source tree: the point is what a user installs. CI
+builds the wheel into `dist/` before running these, and the absence of one fails rather
+than skips — a packaging guard that quietly does not run is the packaging guard that lets
+the deploy tooling out.
+"""
+
+from __future__ import annotations
+
+import zipfile
+from pathlib import Path
+
+CLIENT = Path(__file__).resolve().parents[1]
+#: Nothing whose path contains these may ever be in the wheel. Named, not counted.
+NEVER = ("infra", "generators", "validators", "site", "upload", "release", "catalog", "manifest")
+
+
+def wheel() -> Path:
+    built = sorted((CLIENT / "dist").glob("*.whl"))
+    assert built, f"no wheel in {CLIENT / 'dist'}; build it first: python -m build --wheel"
+    return built[-1]
+
+
+def members() -> list[str]:
+    with zipfile.ZipFile(wheel()) as archive:
+        return archive.namelist()
+
+
+def test_every_module_in_the_wheel_is_the_client() -> None:
+    modules = [name for name in members() if name.endswith(".py")]
+    assert modules, "empty-set control: the wheel was read and holds modules"
+    assert all(name.startswith("loremfile_client/") for name in modules), modules
+
+
+def test_the_wheel_carries_no_repository_tooling() -> None:
+    for name in members():
+        lowered = name.lower()
+        assert not any(f"/{part}" in lowered or lowered.startswith(part) for part in NEVER), name
+
+
+def test_the_console_script_is_the_client_entry_point() -> None:
+    with zipfile.ZipFile(wheel()) as archive:
+        entry_points = next(n for n in archive.namelist() if n.endswith("entry_points.txt"))
+        text = archive.read(entry_points).decode()
+    assert "loremfile = loremfile_client.cli:main" in text
+
+
+def test_the_wheel_declares_no_dependencies() -> None:
+    with zipfile.ZipFile(wheel()) as archive:
+        metadata = next(n for n in archive.namelist() if n.endswith("METADATA"))
+        text = archive.read(metadata).decode()
+    assert "Requires-Dist:" not in text, "the client must install nothing else"
+    assert "Requires-Python: >=3.11" in text