notanda 1.3.0__tar.gz

notanda-1.3.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Rudolf Kiechle
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

notanda-1.3.0/PKG-INFO

@@ -0,0 +1,16 @@
+Metadata-Version: 2.4
+Name: notanda
+Version: 1.3.0
+Summary: Open-Access API Harvester: resumable, idempotent, headless CLI for harvesting Open-Access scholarly documents.
+License: MIT
+Requires-Python: >=3.10
+License-File: LICENSE
+License-File: THIRD_PARTY_NOTICES.md
+License-File: TRADEMARKS.md
+Requires-Dist: httpx>=0.27
+Requires-Dist: pypdf>=4.0
+Requires-Dist: defusedxml>=0.7
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == "dev"
+Requires-Dist: pytest-cov>=5.0; extra == "dev"
+Dynamic: license-file

notanda-1.3.0/README.md

@@ -0,0 +1,194 @@
+# Notanda — Open-Access API Harvester
+
+**Notanda** is the public product name and PyPI distribution name. The repository, Python import
+package and CLI keep their established technical names
+`Open-Access-API-Harvester`, `harvester` and `harvester`.
+
+This repository publishes the reviewed 1.3.0 product-core source. It is not a hosted
+service and the recommended accompanied beta path remains `beta@notanda.io`.
+
+A public **Installer Preview** is available under GitHub Releases solely as a
+technical pre-release:
+https://github.com/RudiRegenwurm/Open-Access-API-Harvester/releases/tag/v1.3.0-installer-preview.1 Those installers are unsigned on Windows/Linux and ad-hoc signed
+on macOS, may trigger SmartScreen/Gatekeeper warnings, and are **not the recommended
+installation path for non-technical users**.
+
+A headless, resumable, idempotent CLI pipeline that discovers Open-Access scholarly
+works through **OpenAlex Topics**, cross-checks them against **Europe PMC**, falls back
+to **Unpaywall** for OA locations, validates every downloaded artifact, and stores the
+result as a deterministic flat corpus ready for a downstream pre-ingestion pipeline.
+Schema 4 additionally retains provider observations, merge decisions and acquisitions
+in an append-only Evidence Ledger while the established read model remains compatible.
+
+```text
+DISCOVERY → NORMALIZATION → DEDUPLICATION → ACQUISITION → VALIDATION → INGESTION → PROVENANCE/REPORTING
+```
+
+It is not a web application, document-management system, search engine, OCR system or
+LLM application. It is a document harvesting pipeline with persistent state.
+
+---
+
+## Quick start
+
+```bash
+pipx install notanda
+```
+
+```bash
+python3 -m venv .venv && . .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+### With the local control center (no command line needed after this)
+
+```bash
+harvester serve
+```
+
+Opens a local web UI at <http://localhost:8765/> for harvesting, browsing the corpus,
+inspecting runs and provenance, verifying integrity, retrying failures and editing
+settings. Credentials can be entered in its Settings page on first run.
+
+**New Harvest** offers two search modes. *Conventional Search* uses the query you type,
+exactly as typed. *Assisted Search* turns a plain-language research question into one
+compact retrieval query you can edit, previews up to ten real discovery results without
+downloading anything, and only then hands the approved query to the same harvest
+pipeline. Assisted Search needs a query-advisor credential; Conventional Search does not
+and is unaffected when the advisor is missing or unavailable.
+
+### From the command line
+
+```bash
+export HARVESTER_OPENALEX_API_KEY='…'          # free: openalex.org/settings/api
+export HARVESTER_CONTACT_EMAIL='ops@example.org'  # required by Unpaywall
+export HARVESTER_ADVISOR_API_KEY='…'           # optional: Assisted Search in the UI only
+
+harvester discover --topic-id T10159 --limit 25    # dry run, downloads nothing
+harvester harvest  --topic-id T10159 --limit 25    # real harvest
+harvester verify --deep                            # check the corpus against state
+```
+
+Both drive the same engine and the same state database: a run started in the browser
+can be resumed from the terminal.
+
+## Output
+
+The downstream contract is a **flat** directory of deterministic sibling files:
+
+```text
+<storage_root>/
+├── doi_10_1371_journal_pone_0123456_5d41402abc4b.pdf     primary full text
+├── doi_10_1371_journal_pone_0123456_5d41402abc4b.xml     when legitimately available
+└── doi_10_1371_journal_pone_0123456_5d41402abc4b.json    mandatory sidecar
+```
+
+The sidecar keeps bibliographic metadata, artifact metadata, provenance and harvest
+state in separate blocks. Missing information is `null`; nothing is ever invented.
+
+```json
+{
+  "document_id": "doi_10_1371_journal_pone_0123456_5d41402abc4b",
+  "doi": "10.1371/journal.pone.0123456",
+  "title": "…",
+  "abstract": "…",
+  "oa_status": "gold",
+  "oa_status_source": "openalex",
+  "domain_tags": ["Social Sciences", "Psychology", "…"],
+  "artifacts": { "pdf": { "sha256": "…", "size_bytes": 123456, "…": "…" } },
+  "provenance": {
+    "discovered_via": ["openalex"],
+    "cross_checked_via": ["europe_pmc"],
+    "acquired_via": "unpaywall",
+    "resolved_url": "…",
+    "http_status": 200
+  },
+  "harvest": { "run_id": "…", "status": "COMPLETED", "attempts": 1 }
+}
+```
+
+## What it guarantees
+
+| Guarantee | How |
+| --- | --- |
+| **Nothing false succeeds** | HTTP 200 is never enough: magic bytes, trailer, parser openability, size bounds and SHA-256 must all pass before an artifact exists under its final name. |
+| **Atomic artifacts** | Every download goes to a unique `.part` file and is published with a single atomic rename after validation. |
+| **Resumable** | Discovery cursors and document state are checkpointed continuously; `harvester resume` continues without repeating completed work. |
+| **Idempotent** | Re-running the same harvest downloads nothing and produces a byte-identical corpus. |
+| **Deduplicated** | One logical document per canonical DOI, however many providers report it. |
+| **No silent loss** | Every failure is a structured row in the state database and appears in the run report. |
+| **Historical evidence** | Re-observations, field-level merge decisions and acquisitions append immutable rows; current `documents`, `source_records` and `artifacts` remain compatible projections. |
+| **Budget-safe** | Provider daily-budget exhaustion checkpoints and suspends cleanly (exit 4) instead of hammering the API. |
+| **Secret-safe** | API keys and contact addresses are redacted from logs, provenance, reports and state. |
+| **Lawful** | Only OA locations that a provider explicitly flags as open are fetched. Access controls are recorded, never circumvented. |
+
+## Commands
+
+| Command | Purpose |
+| --- | --- |
+| `serve` | open the local web control center |
+| `harvest` | discover + acquire |
+| `discover` | discovery and normalization only (dry run) |
+| `resume` | continue an interrupted or suspended run |
+| `status` | run and corpus status (`--json`) |
+| `inspect` | one document's canonical record (`--json`) |
+| `verify` | check the corpus against state (`--deep`, `--json`) |
+| `retry-failed` | re-queue failed documents |
+| `evidence-export` | write a deterministic, independently readable ledger JSON bundle (`--output`) |
+| `evidence-restore` | validate and restore a bundle into an empty ledger (`--input`) |
+
+Exit codes: `0` success · `1` failure · `2` configuration error · `3` completed with
+document failures · `4` suspended (provider budget) · `130` interrupted.
+
+## Tests
+
+```bash
+pytest            # offline suite — no network, no credentials required
+pytest -m live    # controlled real-provider smoke tests
+```
+
+The offline suite runs against deterministic in-process mock providers covering normal
+responses, pagination, throttling, budget exhaustion, timeouts, malformed payloads,
+missing full text, alternative locations and duplicates.
+
+## Documentation
+
+| File | Contents |
+| --- | --- |
+| [`LICENSE`](LICENSE) | MIT license for the product code and documentation, subject to the stated exceptions |
+| [`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) | bundled font licenses and separately licensed dependencies |
+| [`TRADEMARKS.md`](TRADEMARKS.md) | treatment of the Notanda name and visual identity assets |
+| [`CODE_SIGNING_POLICY.md`](CODE_SIGNING_POLICY.md) | Windows code signing policy for the SignPath Foundation application |
+| [`INSTALL-MACOS.md`](INSTALL-MACOS.md) | macOS Installer Preview and Gatekeeper exception procedure |
+
+## Code signing policy
+
+**Free code signing provided by SignPath.io, certificate by SignPath Foundation.**
+
+Windows release signing is intended to use SignPath Foundation's free Open Source path.
+Every production signing request requires explicit manual approval by the maintainer.
+The complete policy is documented in [`CODE_SIGNING_POLICY.md`](CODE_SIGNING_POLICY.md).
+No current unsigned Installer Preview should be interpreted as already signed or approved
+by SignPath Foundation.
+
+## Dependencies
+
+`httpx` (streaming HTTP with a pluggable transport), `pypdf` (PDF structural
+validation), `defusedxml` (safe XML parsing). Everything else is the standard library:
+configuration, SQLite persistence, CLI, logging, hashing, concurrency, retry — and the
+web UI, which uses `http.server` plus a front end with no build step, so `harvester
+serve` is the only command needed to run it.
+
+## License
+
+Unless a file or notice says otherwise, the product source code and documentation
+in this repository are licensed under the [MIT License](LICENSE), copyright 2026
+Rudolf Kiechle.
+
+The bundled IBM Plex and Source Serif 4 font files are not MIT-licensed; they remain
+under the SIL Open Font License 1.1 included beside the files. External Python
+dependencies are not vendored and retain their own licenses. The Notanda name,
+emblem, wordmark, lockup, favicons and avatar are excluded from the MIT grant.
+Details and exact file scopes are in
+[`THIRD_PARTY_NOTICES.md`](THIRD_PARTY_NOTICES.md) and
+[`TRADEMARKS.md`](TRADEMARKS.md).

notanda-1.3.0/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,54 @@
+# Third-party notices
+
+The root MIT license applies to the Notanda / Open-Access API Harvester product
+code and documentation except where this file or another file states a different
+license.
+
+## Bundled fonts
+
+The following font software is distributed with the local web UI and is not
+covered by the root MIT license.
+
+### IBM Plex
+
+Copyright © 2017 IBM Corp. with Reserved Font Name "Plex".
+
+License: SIL Open Font License 1.1. The complete license text is in
+`src/harvester/webui/static/LICENSE-IBMPlex-OFL.txt`.
+
+Files:
+
+- `src/harvester/webui/static/ibm-plex-mono-latin-400-normal.woff2`
+- `src/harvester/webui/static/ibm-plex-sans-latin-400-italic.woff2`
+- `src/harvester/webui/static/ibm-plex-sans-latin-400-normal.woff2`
+- `src/harvester/webui/static/ibm-plex-sans-latin-500-normal.woff2`
+
+### Source Serif 4
+
+Copyright 2014–2023 Adobe, with Reserved Font Name "Source".
+
+License: SIL Open Font License 1.1. The complete license text is in
+`src/harvester/webui/static/LICENSE-SourceSerif4-OFL.txt`.
+
+Files:
+
+- `src/harvester/webui/static/source-serif-4-latin-wght-italic.woff2`
+- `src/harvester/webui/static/source-serif-4-latin-wght-normal.woff2`
+
+## External Python dependencies
+
+The following packages are declared dependencies but are not copied into this
+repository. They are installed separately and remain governed by their own
+licenses. This list records the declared dependency boundary; the corresponding
+project metadata is authoritative for any version actually installed.
+
+| Dependency | Role | Declared license |
+| --- | --- | --- |
+| `httpx` | runtime HTTP client | BSD 3-Clause |
+| `pypdf` | runtime PDF validation | BSD 3-Clause |
+| `defusedxml` | runtime hardened XML parsing | Python Software Foundation License |
+| `pytest` | optional development/test dependency | MIT |
+| `pytest-cov` | optional development/test dependency | MIT |
+
+The Notanda visual identity files have a separate scope described in
+[`TRADEMARKS.md`](TRADEMARKS.md).

notanda-1.3.0/TRADEMARKS.md

@@ -0,0 +1,23 @@
+# Notanda name and visual identity
+
+The MIT License in this repository does not grant rights to use the Notanda name,
+emblem, wordmark, logo, lockup or other source-identifying marks to suggest that a
+modified distribution is official, approved or endorsed by the Notanda project.
+
+The following bundled visual identity files are excluded from the root MIT
+license and remain copyright © 2026 Rudolf Kiechle:
+
+- `src/harvester/webui/static/favicon.ico`
+- `src/harvester/webui/static/favicon.svg`
+- `src/harvester/webui/static/notanda_avatar_180.png`
+- `src/harvester/webui/static/notanda_emblem_farbig_dunkel.svg`
+- `src/harvester/webui/static/notanda_lockup_dunkel_pfade.svg`
+
+They may be redistributed without modification as part of an unmodified copy of
+this software. Modified distributions and forks must remove or replace these
+files and must not present themselves as an official Notanda release unless they
+have separate written permission from the rights holder.
+
+Accurate, descriptive references to the Notanda project remain permitted. No
+statement in this notice restricts rights that applicable trademark law already
+permits.

notanda-1.3.0/pyproject.toml

@@ -0,0 +1,43 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "notanda"
+version = "1.3.0"
+description = "Open-Access API Harvester: resumable, idempotent, headless CLI for harvesting Open-Access scholarly documents."
+requires-python = ">=3.10"
+license = { text = "MIT" }
+dependencies = [
+    # HTTP client with streaming, timeouts and redirect control. Recommended by the specification.
+    "httpx>=0.27",
+    # PDF parser/openability check required by MASTER_SPEC section 15 (structural validation).
+    "pypdf>=4.0",
+    # Hardened XML parsing (XXE / entity-expansion protection) required by MASTER_SPEC section 37/50 L4.
+    "defusedxml>=0.7",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=8.0", "pytest-cov>=5.0"]
+
+[project.scripts]
+harvester = "harvester.cli:main"
+
+[tool.setuptools]
+# Keep every licensing boundary with any wheel built from this source snapshot.
+license-files = ["LICENSE", "THIRD_PARTY_NOTICES.md", "TRADEMARKS.md"]
+
+[tool.setuptools.packages.find]
+where = ["src"]
+
+[tool.setuptools.package-data]
+# The web UI ships as plain assets: no build step, no toolchain to install.
+"harvester.webui" = ["static/*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+markers = [
+    "live: tests that require real network access and credentials (deselected by default)",
+]
+addopts = "-m 'not live'"
+filterwarnings = ["error::DeprecationWarning:harvester.*"]

notanda-1.3.0/setup.cfg

@@ -0,0 +1,4 @@
+[egg_info]
+tag_build = 
+tag_date = 0
+

notanda-1.3.0/src/harvester/__init__.py

@@ -0,0 +1,9 @@
+"""Open-Access API Harvester.
+
+A resumable, idempotent, headless CLI pipeline that discovers Open-Access scholarly
+works via OpenAlex Topics, cross-checks them against Europe PMC, falls back to
+Unpaywall for OA locations, and stores validated PDF/XML artifacts plus mandatory
+JSON sidecars in a flat downstream corpus.
+"""
+
+__version__ = "1.3.0"

notanda-1.3.0/src/harvester/acquisition.py

@@ -0,0 +1,245 @@
+"""Artifact acquisition: bounded streaming download, validation, atomic publication.
+
+MASTER_SPEC sections 14, 17, 37, 38, 39.
+
+The invariant enforced here: **a final artifact filename never exists unless the bytes
+behind it were fully downloaded, validated and hashed.** Everything is written to a
+``.part`` file first and published with a single atomic rename.
+"""
+
+from __future__ import annotations
+
+import logging
+import os
+import tempfile
+import time
+from collections.abc import Callable
+from dataclasses import dataclass
+from pathlib import Path
+
+from .config import Config
+from .errors import (
+    BudgetExhaustedError,
+    DownloadError,
+    HarvesterError,
+    SizeLimitExceededError,
+    StorageError,
+)
+from .http import ProviderClient, redact_url
+from .identity import artifact_path, validate_remote_url
+from .models import ArtifactKind, ArtifactRecord, FulltextCandidate
+from .util import coerce_int, utc_now_iso
+from .validation import ValidationResult, content_type_is_plausible, validate_pdf, validate_xml
+
+LOGGER = logging.getLogger("harvester.acquisition")
+
+PART_SUFFIX = ".part"
+_CHUNK_SIZE = 64 * 1024
+
+
+@dataclass(slots=True)
+class AcquisitionOutcome:
+    artifact: ArtifactRecord
+    validation: ValidationResult
+    #: How many HTTP attempts this artifact needed (1 = no retry).
+    attempts: int = 1
+
+
+class Acquirer:
+    """Downloads and publishes one artifact at a time."""
+
+    def __init__(self, config: Config, client: ProviderClient) -> None:
+        self._config = config
+        self._client = client
+        self._storage_root = Path(config.storage_root)
+
+    def acquire(
+        self,
+        *,
+        document_id: str,
+        candidate: FulltextCandidate,
+        on_attempt: Callable[[int, HarvesterError | None, int | None], None] | None = None,
+    ) -> AcquisitionOutcome:
+        """Download, validate and atomically publish one candidate.
+
+        Retryable transport conditions (timeout, reset, 429, 5xx) are retried with the
+        configured backoff; a rejected *artifact* is not retried, because the same URL
+        will keep returning the same bytes. Raises a :class:`HarvesterError` subclass
+        on final failure. The temporary file is always removed, so an interrupted or
+        rejected download leaves nothing behind.
+        """
+        url = validate_remote_url(
+            candidate.url, allow_private_hosts=self._config.downloads.allow_private_hosts
+        )
+        final_path = artifact_path(self._storage_root, document_id, candidate.kind.value)
+        final_path.parent.mkdir(parents=True, exist_ok=True)
+        max_attempts = max(1, self._config.retry.max_attempts)
+
+        for attempt in range(1, max_attempts + 1):
+            part_path = _new_part_path(final_path)
+            try:
+                resolved_url, status, content_type, _size = self._download(url, part_path)
+                validation = self._validate(candidate.kind, part_path)
+                if not content_type_is_plausible(candidate.kind.value, content_type):
+                    LOGGER.info(
+                        "artifact %s: content-type %r disagrees with validated %s content",
+                        document_id,
+                        content_type,
+                        candidate.kind.value,
+                    )
+                _publish(part_path, final_path)
+            except HarvesterError as exc:
+                _discard(part_path)
+                if on_attempt is not None:
+                    on_attempt(attempt, exc, exc.http_status)
+                if (
+                    isinstance(exc, BudgetExhaustedError)
+                    or not exc.retryable
+                    or attempt >= max_attempts
+                ):
+                    raise
+                self._client.sleep_before_retry(attempt, exc)
+                continue
+            except BaseException:
+                _discard(part_path)
+                raise
+
+            if on_attempt is not None:
+                on_attempt(attempt, None, status)
+            return AcquisitionOutcome(
+                artifact=ArtifactRecord(
+                    kind=candidate.kind,
+                    filename=final_path.name,
+                    sha256=validation.sha256,
+                    size_bytes=validation.size_bytes,
+                    retrieved_at=utc_now_iso(),
+                    source=candidate.source,
+                    original_url=redact_url(candidate.url),
+                    resolved_url=redact_url(resolved_url),
+                    http_status=status,
+                    content_type=content_type,
+                ),
+                validation=validation,
+                attempts=attempt,
+            )
+        raise AssertionError("unreachable: the retry loop always returns or raises")
+
+    # ------------------------------------------------------------------ internals
+
+    def _download(self, url: str, part_path: Path) -> tuple[str, int, str | None, int]:
+        limit = self._config.downloads.max_download_size_bytes
+        with self._client.stream("GET", url) as response:
+            status = response.status_code
+            content_type = response.headers.get("content-type")
+            resolved_url = str(response.url)
+
+            # Reject before transferring anything when the server announces a size we
+            # will not accept (MASTER_SPEC section 39).
+            declared = coerce_int(response.headers.get("content-length"))
+            if declared is not None and declared > limit:
+                raise SizeLimitExceededError(
+                    f"server declared {declared} bytes, above the configured limit of {limit}",
+                    http_status=status,
+                    url=redact_url(url),
+                )
+
+            written = 0
+            try:
+                with open(part_path, "wb") as handle:
+                    for chunk in response.iter_bytes(_CHUNK_SIZE):
+                        if not chunk:
+                            continue
+                        written += len(chunk)
+                        # The limit is enforced during transfer too: Content-Length may
+                        # be absent, wrong, or the response may be chunked.
+                        if written > limit:
+                            raise SizeLimitExceededError(
+                                f"download exceeded the configured limit of {limit} bytes",
+                                http_status=status,
+                                url=redact_url(url),
+                            )
+                        handle.write(chunk)
+                    handle.flush()
+                    os.fsync(handle.fileno())
+            except SizeLimitExceededError:
+                raise
+            except HarvesterError:
+                raise
+            except OSError as exc:
+                raise StorageError(
+                    f"could not write {part_path.name}: {exc}", url=redact_url(url)
+                ) from exc
+            except Exception as exc:  # transport failure part-way through the stream
+                raise DownloadError(
+                    f"download interrupted after {written} bytes: {exc}",
+                    http_status=status,
+                    url=redact_url(url),
+                ) from exc
+
+        if written == 0:
+            raise DownloadError(
+                "server returned an empty body", http_status=status, url=redact_url(url)
+            )
+        return resolved_url, status, content_type, written
+
+    def _validate(self, kind: ArtifactKind, path: Path) -> ValidationResult:
+        if kind is ArtifactKind.PDF:
+            return validate_pdf(path, min_size_bytes=self._config.downloads.min_pdf_size_bytes)
+        return validate_xml(path, min_size_bytes=self._config.downloads.min_xml_size_bytes)
+
+
+def _new_part_path(final_path: Path) -> Path:
+    """A unique temporary path beside the final artifact.
+
+    Uniqueness matters: two workers must never share a ``.part`` file, and a stale
+    ``.part`` left by a killed process must never be appended to or mistaken for
+    live work.
+    """
+    handle, name = tempfile.mkstemp(
+        prefix=f"{final_path.name}.", suffix=PART_SUFFIX, dir=str(final_path.parent)
+    )
+    os.close(handle)
+    return Path(name)
+
+
+def _publish(part_path: Path, final_path: Path) -> None:
+    """Atomically move the validated temporary file into place."""
+    try:
+        os.replace(part_path, final_path)
+    except OSError as exc:
+        raise StorageError(f"could not publish {final_path.name}: {exc}") from exc
+
+
+def _discard(part_path: Path) -> None:
+    try:
+        if part_path.exists():
+            part_path.unlink()
+    except OSError:  # pragma: no cover - best effort cleanup
+        LOGGER.warning("could not remove temporary file %s", part_path)
+
+
+def sweep_stale_parts(storage_root: Path, *, min_age_seconds: float = 300.0) -> list[str]:
+    """Remove ``.part`` files left behind by a terminated process.
+
+    MASTER_SPEC section 17: stale temporary files must be safely detectable and
+    recoverable. They carry no validated content, so removing them costs nothing —
+    the affected documents are re-acquired from their recorded state.
+
+    Only files untouched for *min_age_seconds* are removed. A second harvester
+    process working against the same corpus may have a download in flight, and its
+    temporary file must not be deleted underneath it.
+    """
+    root = Path(storage_root)
+    if not root.exists():
+        return []
+    cutoff = time.time() - max(0.0, min_age_seconds)
+    removed: list[str] = []
+    for path in sorted(root.glob(f"*{PART_SUFFIX}")):
+        try:
+            if path.stat().st_mtime > cutoff:
+                continue
+            path.unlink()
+            removed.append(path.name)
+        except OSError:  # pragma: no cover - concurrent sweep
+            LOGGER.warning("could not remove stale temporary file %s", path)
+    return removed