pathapi 0.1.0__py3-none-any.whl

pathapi/README.md

@@ -0,0 +1,205 @@
+# pathapi
+
+[![Version](https://img.shields.io/pypi/v/pathapi.svg)](https://pypi.org/project/pathapi/)
+[![Python versions](https://img.shields.io/pypi/pyversions/pathapi.svg)](https://pypi.org/project/pathapi/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/jose-pr/pathapi/blob/main/LICENSE)
+[![Docs](https://img.shields.io/badge/docs-latest-blue.svg)](https://jose-pr.github.io/pathapi/)
+[![CI](https://img.shields.io/github/actions/workflow/status/jose-pr/pathapi/test.yml)](https://github.com/jose-pr/pathapi/actions/workflows/test.yml)
+
+A **robust, extensible pathlib-like base** for any resource addressable as a
+path. Same method names, signatures, semantics and exception types as
+`pathlib.Path` wherever a `pathlib.Path` equivalent exists: write code once
+against `Path` and it runs against your local disk, an in-memory tree, an FTP
+server or the members of a zip or tar file, and against any path class built on
+it. It has no dependencies. Deliberate differences from `pathlib` are listed in
+[Divergences from pathlib](https://jose-pr.github.io/pathapi/divergences/).
+
+`pathapi` was published as `pathlib-next` up to 0.10.0; `pip install pathlib-next`
+now installs a shim that depends on this package. See the changelog for the
+migration from `pathlib_next`.
+
+## Features
+
+| Class | Read | Write | List | Stat | mkdir | Delete | rename |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| local path (`LocalPath`) | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
+| in-memory (`MemPath`) | Yes | Yes | Yes | Yes | Yes | Yes | No (`move()` copies) |
+| FTP and FTPS (`FTPPath`) | Yes | Yes | Yes (MLSD, NLST fallback) | Yes | Yes | Yes | Yes |
+| zip members (`ZipPath`) | Yes | Local archive only | Yes | Yes | Local archive only | Local archive only | Local archive only |
+| tar members (`TarPath`, incl. gz/bz2/xz) | Yes | No | Yes | Yes | No | No | No |
+| `ArchivePath` | As `ZipPath`/`TarPath` for the format it detects | | | | | | |
+
+Every class gets `glob()`, `walk()`, `copy()`/`move()`, `rm()` and `PathSyncer`
+from the base classes; `LocalPath` and the other classes replace some of them
+where the backend has a faster way.
+
+- **One path interface** over local files, in-memory trees, FTP servers and
+  archive members, and over any class built on `Path`.
+- **`MemPath`**: a virtual filesystem for tests, mocks and transient data.
+- **`FTPPath`, `ZipPath`, `TarPath`, `ArchivePath`**: an FTP server and the
+  members of a zip or tar file as paths, on the standard library alone.
+- **`pathapi.http`**: `HTTPClient` and a transport contract for HTTP
+  requests on the standard library (redirects that never carry
+  `Authorization` to another origin, TLS verified by default, a `proxies=`
+  provider); the `requests` library plugs in as a transport through the
+  `pathapi-requests` distribution.
+- **`PathSyncer`**: one-way, content-compared tree sync between any two
+  implementations, with dry runs and event hooks.
+- **Extensible**: subclass `Path` for any resource, and verify it with the
+  bundled pytest contracts.
+
+Paths for other places are separate distributions built on this one:
+
+| Distribution | Provides |
+| --- | --- |
+| [`uripath`](https://github.com/jose-pr/uripath) | `URIPath`: one class for any URI, with the schemes `file:`, `data:`, `ftp:`, `zip:`, `tar:`, `http:` and `dav:`, and the `uripath` command |
+| [`sftppath`](https://github.com/jose-pr/sftppath) | `SFTPPath` (paramiko or asyncssh) |
+| [`objpath`](https://github.com/jose-pr/objpath) | `S3Path`, `GSPath` and `AZPath` (Amazon S3, Google Cloud Storage, Azure Blob Storage) |
+| [`repopath`](https://github.com/jose-pr/repopath) | `RepoPath`, `GitHubPath` and `GitLabPath` (the REST APIs of the hosting services) |
+| [`pathapi-requests`](https://github.com/jose-pr/pathapi-requests) | `RequestsTransport`, the HTTP transport that sends through a `requests.Session` |
+
+## Installation
+
+```bash
+pip install pathapi
+```
+
+There are no extras and no dependencies: everything above runs on the standard
+library. `pytest` is needed only to import `pathapi.testing` and
+`pathapi.http.testing`.
+
+## Quick start
+
+**Local filesystem**, as `pathlib.Path`:
+
+```python
+from pathapi import Path
+
+p = Path("data") / "report.txt"
+p.parent.mkdir(parents=True, exist_ok=True)
+p.write_text("hello")
+print(p.read_text())
+```
+
+**In memory**, no disk I/O:
+
+```python
+from pathapi.mempath import MemPath
+
+p = MemPath("/config/settings.json")
+p.parent.mkdir(parents=True, exist_ok=True)
+p.write_text('{"debug": true}')
+print([child.name for child in p.parent.iterdir()])
+```
+
+**Archive members**: the segments are the member path and the archive is any
+path:
+
+```python
+import zipfile
+from pathapi import ZipPath
+
+with zipfile.ZipFile("backup.zip", "w") as archive:
+    archive.writestr("etc/config.ini", "[main]\n")
+
+member = ZipPath("etc/config.ini", archive="backup.zip")
+print(member.read_text())
+print([child.name for child in member.parent.iterdir()])
+```
+
+**FTP**: nothing connects until the first operation:
+
+<!-- example: skip: reads from a remote host -->
+```python
+from pathapi import FTPPath
+
+for child in FTPPath("/pub", host="ftp.example.com", user="anonymous").iterdir():
+    print(child.name, child.stat().st_size)
+```
+
+**Copy and sync across classes**:
+
+```python
+from pathapi import Path
+from pathapi.mempath import MemPath
+from pathapi.sync import PathSyncer
+
+source = MemPath("/site")
+(source / "css").mkdir(parents=True)
+(source / "index.html").write_text("<h1>hi</h1>")
+
+PathSyncer(remove_missing=True).sync(source, Path("site-copy"))
+print(sorted(p.name for p in Path("site-copy").iterdir()))
+```
+
+## API overview
+
+| Module | Purpose |
+| --- | --- |
+| `pathapi.path` | `Pathname`/`Path` base classes and the derived operations |
+| `pathapi.fspath` | `LocalPath`, `PosixPathname`, `WindowsPathname` |
+| `pathapi.mempath` | `MemPath` in-memory filesystem |
+| `pathapi.ftp` | `FTPPath`, `FTPBackend` (FTP and FTPS on `ftplib`) |
+| `pathapi.archive` | `ZipPath`, `TarPath`, `ArchivePath` (members of a zip or tar file), `make_archive`, `unpack_archive` |
+| `pathapi.http` | `HTTPClient`, `HTTPTransport`, `HTTPResponse` (HTTP requests on `http.client`) |
+| `pathapi.protocols` | `Stat`, `Chmod`, `BinaryOpen`, `NativeChecksum` |
+| `pathapi.sync` | `PathSyncer`, `SyncEvent`, `PathAndStat` |
+| `pathapi.glob` | the glob engine behind `Path.glob()` |
+| `pathapi.checksum` | `md5`, `sha256`, `stream`, `native` |
+| `pathapi.providers` | the helpers a class built on `Path` calls |
+| `pathapi.exceptions` | `PathError`, `NonRelativePatternError` |
+| `pathapi.testing` | pytest contracts for custom implementations |
+| `pathapi.http.testing` | testing support (not stable API): the pytest suite that holds an HTTP transport to the promises of the built-in one |
+
+The full reference is on the [documentation site](https://jose-pr.github.io/pathapi/);
+an agent-oriented API summary ships inside the package as
+`pathapi/AGENTS.md`.
+
+## Extending
+
+Subclass `Path` for a resource of your own (`MemPath` is the reference), as
+worked through in
+[Extending](https://jose-pr.github.io/pathapi/guides/extending/).
+`pathapi.testing` provides the pytest contracts (`PurePathContract`,
+`ReadPathContract`, `PathContract`) and `populate_fixture_tree()` that every
+class of the package and of the distributions above is verified with.
+
+## Supported Python versions
+
+Python 3.9 through 3.14. CI runs the full suite on Ubuntu for every version
+from 3.9 to 3.14, on Windows and macOS for 3.9 and 3.14, and with nothing but
+`pytest` and `pyftpdlib` installed on 3.9 and 3.14.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+python -m pytest -q -rs
+```
+
+Tests that need an optional package are skipped when it is missing, so check
+the skip list (`-rs`). Contributor notes (virtual environments, formatting, CI,
+releases) are in the repository's
+[`AGENTS.md`](https://github.com/jose-pr/pathapi/blob/main/AGENTS.md);
+benchmarks are described in
+[Benchmarks](https://jose-pr.github.io/pathapi/benchmarks/). To preview
+the documentation: `pip install -e ".[docs]"` then `mkdocs serve`.
+
+Changes are recorded in the
+[changelog](https://github.com/jose-pr/pathapi/blob/main/CHANGELOG.md).
+
+### Releasing
+
+1. Move the `[Unreleased]` entries of `CHANGELOG.md` under a new
+   `## [x.y.z] - <date>` heading and add its link definition at the bottom.
+2. Bump `version` in `pyproject.toml` in the same commit.
+3. Before 1.0 the minor version changes only when the documented API breaks;
+   new methods, new optional arguments and fixes are patch releases.
+4. Run the full suite on the oldest and the newest supported Python,
+   `mkdocs build --strict` and `python -m build`.
+5. Push `main`, then the `v*` tag: the release workflow tests, builds and
+   publishes to PyPI and creates the GitHub release.
+
+## License
+
+MIT; see [LICENSE](https://github.com/jose-pr/pathapi/blob/main/LICENSE).

pathapi/__init__.py

@@ -0,0 +1,62 @@
+from __future__ import annotations
+
+import importlib as _importlib
+from importlib import metadata as _metadata
+
+__all__ = [
+    "ArchivePath",
+    "BinaryOpen",
+    "Chmod",
+    "FTPPath",
+    "FileStat",
+    "FsPathLike",
+    "LocalPath",
+    "MemPath",
+    "NonRelativePatternError",
+    "Path",
+    "PathError",
+    "PathLike",
+    "Pathname",
+    "PosixPathname",
+    "PurePathLike",
+    "Stat",
+    "TarPath",
+    "WindowsPathname",
+    "ZipPath",
+    "__version__",
+    "glob",
+    "sync",
+]
+
+try:
+    __version__ = _metadata.version("pathapi")
+except _metadata.PackageNotFoundError:  # a source tree that is not installed
+    __version__ = "0+unknown"
+
+from .fspath import LocalPath, PosixPathname, WindowsPathname
+from .mempath import MemPath
+from .path import FsPathLike, Path, PathLike, Pathname, PurePathLike
+from .protocols import BinaryOpen, Chmod, Stat
+from . import glob as glob
+from . import sync as sync
+from ._filestat import FileStat
+from .exceptions import NonRelativePatternError, PathError
+
+#: Path classes of the standard-library providers, imported on first use so
+#: that `import pathapi` loads neither `ftplib` nor `ssl`, and no archive
+#: code.
+_LAZY = {
+    "ArchivePath": ".archive",
+    "FTPPath": ".ftp",
+    "TarPath": ".archive",
+    "ZipPath": ".archive",
+}
+
+
+def __getattr__(name: str):
+    module = _LAZY.get(name)
+    if module is not None:
+        value = getattr(_importlib.import_module(module, __name__), name)
+        globals()[name] = value
+        return value
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

pathapi/_commit.py

@@ -0,0 +1,116 @@
+from __future__ import annotations
+
+import io as _io
+import typing as _ty
+import warnings as _warnings
+
+__all__ = ["CommitOnClose", "DiscardOnError"]
+
+
+class DiscardOnError:
+    """Mixin for a write stream that commits when it is closed. Leaving a
+    `with` block through an exception (any `BaseException`) calls `discard()`
+    and then closes; leaving it normally just closes. A stream that is
+    garbage collected unclosed while `_holds_uncommitted()` is true is
+    discarded too, with a `ResourceWarning`: what was never finished is not
+    committed. Put it first in the bases, so that its `__exit__` and
+    `__del__` win over `io.IOBase`'s."""
+
+    __slots__ = ()
+
+    def discard(self) -> None:
+        """Drop what was written so that `close()` commits nothing. It does
+        nothing more when called twice or after `close()`."""
+        raise NotImplementedError
+
+    def _holds_uncommitted(self) -> bool:
+        """Whether closing now would commit something. A subclass returns
+        false for a stream that has nothing to commit (never modified, or
+        already discarded) so that collecting it is silent."""
+        return True
+
+    def __del__(self):
+        try:
+            if not self.closed and self._holds_uncommitted():
+                self.discard()
+                _warnings.warn(
+                    f"unclosed {type(self).__name__} discarded without "
+                    "committing; close the file",
+                    ResourceWarning,
+                    source=self,
+                )
+        finally:
+            super().__del__()
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        try:
+            if exc_type is not None:
+                self.discard()
+        except BaseException:
+            super().__exit__(exc_type, exc_value, traceback)
+            raise
+        return super().__exit__(exc_type, exc_value, traceback)
+
+
+class CommitOnClose(DiscardOnError, _io.BytesIO):
+    """A write buffer held in memory whose content is handed to `commit` when
+    it is closed, once, if it was written to. For a backend with no way to
+    write a file piecewise (an FTP `STOR`, a rewrite of an archive): the cost
+    is the whole file in memory for the duration of the write.
+
+    `initial` (an `open("r+")`) is the file's content, with the position at 0,
+    and `commit` runs only if the buffer was modified; without it, `commit`
+    runs at the first close even if nothing was written, which creates an
+    empty file. `commit(buffer)` runs before the buffer is closed, so it can
+    read from it.
+
+    `discard()` drops the buffer so that a later `close()` commits nothing;
+    leaving a `with` block through an exception calls it before closing, and
+    so does collecting the stream unclosed, which also warns."""
+
+    _discarded = False
+    _dirty = False
+
+    def __init__(
+        self,
+        commit: "_ty.Callable[[_io.BytesIO], object]",
+        initial: "bytes | None" = None,
+    ):
+        super().__init__(b"" if initial is None else initial)
+        self._commit = commit
+        self._dirty = initial is None
+
+    def write(self, data):
+        self._dirty = True
+        return super().write(data)
+
+    def writelines(self, lines):
+        self._dirty = True
+        return super().writelines(lines)
+
+    def truncate(self, size=None):
+        self._dirty = True
+        return super().truncate(size)
+
+    def _holds_uncommitted(self):
+        return self._dirty and not self._discarded
+
+    def discard(self):
+        """Drop what was written: `close()` commits nothing, whatever is
+        written after this. Calling it twice, or after `close()`, does
+        nothing more."""
+        self._discarded = True
+        if not self.closed:
+            super().truncate(0)
+            super().seek(0)
+
+    def close(self):
+        if self.closed:
+            return
+        try:
+            if self._dirty and not self._discarded:
+                self._commit(self)
+        finally:
+            # Closed even when the commit fails, so `IOBase.__del__` does not
+            # retry it (over newer content) at garbage collection.
+            super().close()

pathapi/_discard.py

@@ -0,0 +1,92 @@
+"""What `BinaryOpen.open()` does in text mode when the `with` block of a write
+stream raised.
+
+A stream that commits on close (an upload, an archive rewrite) has a
+`discard()` method. When the block ended in an exception, `discard()` is called
+before the stream is closed, so the close commits nothing; when it ended
+normally, or the file is closed with `close()`, nothing changes; a wrapper
+collected unclosed discards as well, with a `ResourceWarning`. In binary mode
+the stream's own `__exit__` does this (`DiscardOnError`).
+"""
+
+from __future__ import annotations
+
+import io as _io
+import typing as _ty
+import warnings as _warnings
+
+
+class DiscardingTextIO(_io.TextIOWrapper):
+    """A `TextIOWrapper` over a stream with `discard()`. Leaving a `with` block
+    through an exception drops the text the wrapper still holds, calls
+    `discard`, then closes: a stock wrapper would write that text to the stream
+    and let the close commit it."""
+
+    _dropping = False
+    _discard: "_ty.Callable[[], object] | None" = None
+
+    def __init__(self, stream, discard, *args, **kwargs):
+        super().__init__(stream, *args, **kwargs)
+        self._discard = discard
+
+    def flush(self):
+        if not self._dropping:
+            super().flush()
+
+    def close(self):
+        if self.closed:
+            return
+        try:
+            self.flush()
+        except BaseException:
+            # The stream has not been given all the text: it must not commit
+            # what it has.
+            self._dropping = True
+            try:
+                self._discard()
+            finally:
+                super().close()
+            raise
+        super().close()
+
+    def _unclosed(self):
+        try:
+            return not self.closed
+        except ValueError:  # the constructor raised before initialising it
+            return False
+
+    def __del__(self):
+        # Collected unclosed: the text still held and the stream's content were
+        # never finished, so neither is committed.
+        try:
+            if self._unclosed() and self._discard is not None:
+                self._dropping = True
+                self._discard()
+                _warnings.warn(
+                    f"unclosed {type(self).__name__} discarded without "
+                    "committing; close the file",
+                    ResourceWarning,
+                    source=self,
+                )
+        finally:
+            super().__del__()
+
+    def __exit__(self, exc_type, exc_value, traceback):
+        if exc_type is None:
+            return super().__exit__(exc_type, exc_value, traceback)
+        self._dropping = True
+        try:
+            self._discard()
+        finally:
+            super().__exit__(exc_type, exc_value, traceback)
+
+
+def abandon(stream: _ty.Any) -> None:
+    """Close a stream whose `open()` failed after the backend gave it out,
+    having it discard what it holds first so the failed call commits nothing."""
+    try:
+        discard = getattr(stream, "discard", None)
+        if callable(discard):
+            discard()
+    finally:
+        stream.close()

pathapi/_errorpolicy.py

@@ -0,0 +1,106 @@
+"""The error and follow policies an operation takes as arguments."""
+
+from __future__ import annotations
+
+import typing as _ty
+
+__all__ = [
+    "OfferedOnce",
+    "as_error_handler",
+    "as_error_predicate",
+    "check_follow",
+    "copy_error_predicate",
+]
+
+
+def as_error_handler(
+    ignore_error: _ty.Union[bool, _ty.Callable[..., bool], None],
+    *,
+    default: bool = False,
+) -> _ty.Callable[..., bool]:
+    """Normalize an `ignore_error` argument into a *callable* error policy.
+
+    Every `ignore_error` (and `copy()`'s `on_error`) parameter in this library
+    accepts either a bool or a callable, but the callables have **different
+    arities** per call site (`Path.rm()` and `Path.copy(on_error=)` ->
+    `(error, path)`, `Path.copy(ignore_error=)` -> `(error)`,
+    `PathSyncer.sync()` -> `(error, source, target, event)`). Changing the
+    arity of an existing keyword would break its callers, so this helper only
+    normalizes the *bool* case and passes a supplied callable through
+    untouched -- it is invoked with whatever arguments its own call site
+    already uses.
+
+    `None` means "no policy supplied": it resolves to `default` (False for
+    every current caller, i.e. raise on the first error), which preserves
+    `Path.copy(ignore_error=None)`'s documented meaning.
+
+    Centralizing this keeps a fourth call site from drifting back into
+    calling a bool (see `PathSyncer.sync()`'s symlink branch, which did
+    exactly that and raised `TypeError: 'bool' object is not callable`).
+    """
+    if callable(ignore_error):
+        return ignore_error
+    if ignore_error is None:
+        ignore_error = default
+    result = bool(ignore_error)
+    return lambda *args, **kwargs: result
+
+
+class OfferedOnce:
+    """An `(error, path) -> bool` error predicate that sees each error once.
+
+    `True` swallows the error. An error it answered `False` for, and an
+    exception it raised itself, are not offered again when they travel up
+    through the enclosing directories of a recursive operation: they are
+    raised as they are. Build one with `as_error_predicate()`.
+    """
+
+    __slots__ = ("_predicate", "_seen")
+
+    def __init__(self, predicate: _ty.Callable[[Exception, _ty.Any], bool]) -> None:
+        self._predicate = predicate
+        self._seen: "list[BaseException]" = []
+
+    def __call__(self, error: Exception, path: _ty.Any) -> bool:
+        if any(error is seen for seen in self._seen):
+            return False
+        try:
+            swallow = self._predicate(error, path)
+        except BaseException as raised:
+            self._seen.append(raised)
+            raise
+        if swallow:
+            return True
+        self._seen.append(error)
+        return False
+
+
+def as_error_predicate(
+    on_error: _ty.Union[bool, _ty.Callable[[Exception, _ty.Any], bool], None],
+) -> OfferedOnce:
+    """`on_error` as an `OfferedOnce` predicate: a bool becomes a constant
+    answer (`None` is `False`), a callable is called as `(error, path)`, and
+    an `OfferedOnce` is returned as it is, so a recursive call that is handed
+    the predicate of its caller shares what that predicate has seen."""
+    if isinstance(on_error, OfferedOnce):
+        return on_error
+    return OfferedOnce(as_error_handler(on_error))
+
+
+def check_follow(name: str, policy) -> None:
+    """Raise `ValueError` naming `name` unless `policy` is `True`, `False`,
+    `None` or a callable: what `rm()` accepts for `follow_symlinks=` and
+    `follow_binds=`. An `rm()` override calls it before removing anything."""
+    if not (policy is None or callable(policy) or isinstance(policy, bool)):
+        raise ValueError(f"{name} must be True, False, None or a callable")
+
+
+def copy_error_predicate(ignore_error, on_error) -> "OfferedOnce | None":
+    """The `(error, path) -> bool` predicate for `copy()`'s `on_error=`, or
+    None when it was not given. Raises `TypeError` when `ignore_error=` was
+    given too: a `copy()` override calls this before it copies anything."""
+    if on_error is None:
+        return None
+    if ignore_error is not None:
+        raise TypeError("copy() takes ignore_error or on_error, not both")
+    return as_error_predicate(on_error)