pathapi-uri 0.1.0__py3-none-any.whl

uripath/README.md

@@ -0,0 +1,213 @@
+# uripath
+
+[![Version](https://img.shields.io/pypi/v/pathapi-uri.svg)](https://pypi.org/project/pathapi-uri/)
+[![Python versions](https://img.shields.io/pypi/pyversions/pathapi-uri.svg)](https://pypi.org/project/pathapi-uri/)
+[![License: MIT](https://img.shields.io/badge/license-MIT-blue.svg)](https://github.com/jose-pr/uripath/blob/main/LICENSE)
+[![Docs](https://img.shields.io/badge/docs-latest-blue.svg)](https://jose-pr.github.io/uripath/)
+[![CI](https://img.shields.io/github/actions/workflow/status/jose-pr/uripath/test.yml)](https://github.com/jose-pr/uripath/actions/workflows/test.yml)
+
+Installed as `pathapi-uri` from PyPI; imported, and run as a command, as `uripath`.
+
+**One `pathlib`-style API for URIs.** `URIPath("https://...")`, `URIPath("zip:...")`
+and `URIPath("ftp://...")` return the class registered for the scheme, and each
+has the method names, signatures and exception types of `pathlib.Path`: write
+code once and run it against a local file, an archive member, an FTP or WebDAV
+server, a web directory or, through the `repopath` distribution, a GitHub or GitLab
+repository. It builds on
+[`pathapi`](https://github.com/jose-pr/pathapi) and is documented at
+<https://jose-pr.github.io/uripath/>.
+
+## Features
+
+| Scheme (class) | Needs | Read | Write | List | Stat | mkdir | Delete | rename |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| `file:` (`FileURIPath`) | nothing | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
+| `data:` (`DataURIPath`, RFC 2397) | nothing | Yes | No | No | Yes | No | No | No |
+| `zip:` (`ZipURIPath`) | nothing | Yes | Local `file:` archive only | Yes | Yes | Local archive only | Local archive only | Local archive only |
+| `tar:` (`TarURIPath`, incl. gz/bz2/xz) | nothing | Yes | No | Yes | Yes | No | No | No |
+| `archive:`, `archive+zip:`, `archive+tar:` | nothing | As `zip:`/`tar:` for the detected or pinned format | | | | | | |
+| `ftp:` / `ftps:` (`FTPURIPath`) | nothing | Yes | Yes | Yes (MLSD, NLST fallback) | Yes | Yes | Yes | Yes |
+| `http:` / `https:` (`HTTPURIPath`) | nothing | Yes | Yes (PUT, configurable) | Yes (HTML index) | Yes | No | Yes (DELETE) | No |
+| `dav:` / `davs:` (`DAVURIPath`, WebDAV) | nothing | Yes | Yes (no append) | Yes (PROPFIND) | Yes | Yes | Yes | Yes |
+| `sftp:` | `sftppath[paramiko]` or `[asyncssh]` | Yes | Yes | Yes | Yes | Yes | Yes | Yes |
+| `s3:` | `objpath[s3]` | Yes | Yes (no append) | Yes (prefixes) | Yes | Yes (marker object) | Yes | Yes (same bucket) |
+| `gs:` | `objpath[gs]` | Yes | Yes (no append) | Yes (prefixes) | Yes | Yes (marker object) | Yes | Yes (same bucket) |
+| `az:` | `objpath[az]` | Yes | Yes (no append) | Yes (prefixes) | Yes | Yes (marker blob) | Yes | Yes (same container) |
+| `repo:` | `repopath` | Yes | No | Yes | Yes | No | No | No |
+| `github:` | `repopath` | Yes | No | Yes | Yes | No | No | No |
+| `gitlab:` | `repopath` | Yes | No | Yes | Yes | No | No | No |
+
+The last seven rows are other distributions: a path of one of those schemes
+builds, joins and prints without it, and its first use of the server raises an
+`ImportError` that names what to install. The `requests` HTTP transport
+(`with_session()`, `~/.netrc`, SOCKS proxies) is the distribution
+`pathapi-requests`: once it is installed, `with_session()` and
+`PATHAPI_HTTP_TRANSPORT=requests` use it, with nothing to configure.
+
+Every scheme gets `glob()`, `walk()`, `copy()`/`move()`, `rm()` and `PathSyncer`
+from the `pathapi` base classes. The per-scheme notes (timeouts,
+certificate verification, authentication, limits) are in
+[Schemes](https://jose-pr.github.io/uripath/guides/schemes/).
+
+- **Pure URI algebra**: `URI` parses, joins and compares URIs with no I/O and
+  no scheme class needed; `Source` and `Query` are its authority and query.
+- **HTTP on the standard library**: `http:` and `dav:` send through a
+  replaceable transport built on `http.client`.
+- **Extensible**: subclass `URIPath` and declare `__SCHEMES` for a new scheme,
+  or register it through a `uripath.schemes` entry point.
+- **`uripath` command**: `read`, `write`, `cp`, `rm` and `sync` across local
+  paths and URIs, with `-` for stdin/stdout (`pip install "pathapi-uri[cli]"`).
+
+## Installation
+
+```bash
+pip install pathapi-uri
+```
+
+Requires `pathapi` and `netimps`, which pip installs with it.
+
+| Install | Adds |
+| --- | --- |
+| `pip install pathapi-uri` | every scheme whose Needs is nothing in the table above |
+| `pip install "pathapi-uri[cli]"` | the `uripath` command (its only extra) |
+| `pip install "sftppath[paramiko]"` or `[asyncssh]` | `sftp:` |
+| `pip install "objpath[s3]"`, `[gs]` or `[az]` | `s3:`, `gs:`, `az:` |
+| `pip install pathapi-requests` | the `requests` HTTP transport |
+
+## Quick start
+
+**Any URI**: `URIPath` returns the class registered for the scheme:
+
+```python
+from uripath import URIPath
+
+report = URIPath("file:data/report.txt")                    # FileURIPath
+report.parent.mkdir(parents=True, exist_ok=True)
+report.write_text("hello")
+print(report.read_text())
+print(URIPath("data:text/plain;base64,aGVsbG8=").read_text())  # DataURIPath
+```
+
+**Archive members**: `<scheme>:<archive-uri>!/<member>`, where the archive
+itself is any URI (`file:`, `http:`, `ftp:`, ...):
+
+```python
+import zipfile
+from uripath import URIPath
+
+with zipfile.ZipFile("backup.zip", "w") as archive:
+    archive.writestr("etc/config.ini", "[main]\n")
+
+member = URIPath("zip:file:backup.zip!/etc/config.ini")
+print(member.read_text())
+print([child.name for child in member.parent.iterdir()])
+```
+
+**Remote paths** use the same API:
+
+<!-- example: skip: reads from remote hosts -->
+```python
+from uripath import URIPath
+
+for child in URIPath("https://example.com/data/").iterdir():
+    print(child.name, child.stat().st_size)
+
+print(URIPath("ftp://user@host/pub/README").read_text())
+```
+
+**Copy and sync across schemes**, with the `pathapi` syncer:
+
+```python
+from pathapi.sync import PathSyncer
+from uripath import URIPath
+
+source = URIPath("file:site")
+(source / "css").mkdir(parents=True)
+(source / "index.html").write_text("<h1>hi</h1>")
+
+PathSyncer(remove_missing=True).sync(source, URIPath("file:site-copy"))
+print(sorted(p.name for p in URIPath("file:site-copy").iterdir()))
+```
+
+## Command line
+
+`uripath` reads, writes, copies, removes and syncs local paths and URIs, with
+`-` for stdin or stdout. It is built on `duho` and needs the `cli` extra,
+`pip install "pathapi-uri[cli]"`; without it the command says so and exits with
+status 1. These commands run as written in an empty directory:
+
+```bash
+mkdir site
+uripath write site/index.html "<h1>hi</h1>"
+uripath read site/index.html
+uripath cp site/index.html site/copy.html
+uripath sync --dry-run --remove-missing site site-copy
+uripath sync site site-copy
+uripath rm --recursive site-copy
+```
+
+Either side can be a URI instead:
+
+<!-- example: skip: needs an FTP host and a web server -->
+```bash
+uripath cp report.txt ftp://host/tmp/report.txt
+uripath sync --dry-run --remove-missing site/ ftp://host/site/
+```
+
+The exit status is 0 on success, 1 when the operation failed, 2 for a wrong
+invocation, 130 on Ctrl-C and 141 when the reader of stdout went away. Every
+flag is described in the
+[CLI guide](https://jose-pr.github.io/uripath/guides/cli/), and `uripath` can
+be served to an agent as a set of tools.
+
+## API overview
+
+| Module | Purpose |
+| --- | --- |
+| `uripath` | `URI`, `URIPath`, `URILike`, `Source`, `Query`, `QueryLike`, the functions `as_path` and `copy_stream`, the errors `URIPathError`, `URISyntaxError` and `UnsupportedSchemeError`, and `__version__` |
+| `uripath.source` | `Source` (scheme, userinfo, host, port), `same_authority` and the helpers that read and compose a URI's text |
+| `uripath.query` | `Query`, an immutable query value, and `QueryLike` |
+| `uripath.exceptions` | The errors above |
+| `uripath.schemes` | The contract and registry of schemes, `DerivedBackend`, `DelegatingURIPath`, and every built-in scheme's classes: `FileURIPath`, `DataURIPath`, `ZipURIPath`, `TarURIPath`, `ArchiveURIPath`, `FTPURIPath`, `HTTPURIPath`, `DAVURIPath` and their backends |
+
+The full reference is on the [documentation site](https://jose-pr.github.io/uripath/);
+an agent-oriented API summary ships inside the package as `uripath/AGENTS.md`.
+
+## Extending
+
+Subclass `URIPath` and declare `__SCHEMES` for a new URI scheme; it is
+dispatched as soon as its module is imported, or through a `uripath.schemes`
+entry point. A scheme name belongs to one class: a second claim raises unless
+the class statement says `replace=True`, and `URIPath.schemes()` /
+`URIPath.scheme_class(name)` read the registry. Worked examples:
+[Extending](https://jose-pr.github.io/uripath/guides/extending/). The pytest
+contracts that verify a scheme are in `pathapi.testing`.
+
+## Development
+
+```bash
+pip install -e ".[dev]"
+python -m pytest -q -rs
+```
+
+Tests for a server that is not installed are skipped, so check the skip list
+(`-rs`). Contributor notes (virtual environments, formatting, CI, releases) are
+in the repository's
+[`AGENTS.md`](https://github.com/jose-pr/uripath/blob/main/AGENTS.md). To
+preview the documentation: `pip install -e ".[docs]"` then `mkdocs serve`.
+
+Changes are recorded in the
+[changelog](https://github.com/jose-pr/uripath/blob/main/CHANGELOG.md).
+
+### Releasing
+
+This project follows [Semantic Versioning](https://semver.org/) and keeps a
+[`CHANGELOG.md`](https://github.com/jose-pr/uripath/blob/main/CHANGELOG.md). Pushing a tag matching `v*` triggers the release
+workflow: test gate → build (checking the tag names the version built) → a strict
+docs build as a gate → GitHub release → publish. The release workflow never deploys
+the docs site itself: for a final release its last job dispatches the docs workflow
+at the tag, which owns every Pages deploy.
+
+## License
+
+MIT — see [LICENSE](https://github.com/jose-pr/uripath/blob/main/LICENSE).

uripath/__init__.py

@@ -0,0 +1,27 @@
+"""pathlib-style paths for URIs."""
+
+from __future__ import annotations
+
+from ._pathtext import as_path
+from ._transfer import copy_stream
+from ._uri import URI, URILike
+from ._version import __version__
+from .exceptions import URIPathError, URISyntaxError, UnsupportedSchemeError
+from .query import Query, QueryLike
+from .schemes import URIPath
+from .source import Source
+
+__all__ = [
+    "__version__",
+    "as_path",
+    "copy_stream",
+    "Query",
+    "QueryLike",
+    "Source",
+    "URI",
+    "URILike",
+    "URIPath",
+    "URIPathError",
+    "URISyntaxError",
+    "UnsupportedSchemeError",
+]

uripath/__main__.py

@@ -0,0 +1,7 @@
+"""`python -m uripath`: the same program as the `uripath` console script."""
+
+from __future__ import annotations
+
+from .cli import main
+
+raise SystemExit(main())

uripath/_pathtext.py

@@ -0,0 +1,32 @@
+"""`as_path`: text that is a local path or a URI, read as the path it names."""
+
+from __future__ import annotations
+
+import typing as _ty
+
+from pathapi import LocalPath
+
+from .schemes._path import URIPath, _looks_like_uri
+
+__all__ = ["as_path"]
+
+
+def as_path(text: str) -> "_ty.Union[LocalPath, URIPath]":
+    """The path `text` names: a `URIPath` when it is a URI, a `LocalPath` when
+    it is a local path.
+
+    `text` is a URI when it starts with an RFC 3986 scheme and either holds
+    `://` or the scheme is one a class registers (`data:`, `zip:`, `file:`,
+    `sftp:`, ...). Anything else is local: `C:/data` (a one-letter scheme is a
+    drive), `12:30.txt`, `notes:draft` and `./a:b`. A scheme another
+    distribution provides gives a `URIPath` whose first I/O raises the
+    `ImportError` that names the distribution to install.
+
+    Raises `TypeError` for what is not a `str`. Nothing is read or connected
+    to.
+    """
+    if not isinstance(text, str):
+        raise TypeError(f"path text must be a str, not {type(text).__name__!r}")
+    if _looks_like_uri(text):
+        return URIPath(text, findclass=True)
+    return LocalPath(text)

uripath/_rfc3986.py

@@ -0,0 +1,67 @@
+"""The RFC 3986 pieces uripath needs, on the standard library: the split of a
+URI reference into its five components, and percent-decoding and -encoding."""
+
+from __future__ import annotations
+
+import re as _re
+import typing as _ty
+from urllib.parse import quote_from_bytes as _quote_from_bytes
+from urllib.parse import unquote_to_bytes as _unquote_to_bytes
+
+__all__ = ["Split", "decode", "encode", "split"]
+
+
+class Split(_ty.NamedTuple):
+    """The components of a URI reference as written: `None` for one that is
+    absent, `""` for one that is present and empty (`http://h/x?` has the query
+    `""`, `http://h/x` has none), the path always a `str`."""
+
+    scheme: "str | None"
+    authority: "str | None"
+    path: str
+    query: "str | None"
+    fragment: "str | None"
+
+
+# RFC 3986 appendix B. Like the reference expression, `.` does not match a
+# newline, so a fragment ends at one and what follows is dropped.
+_SPLIT = _re.compile(
+    r"(?:([A-Za-z][A-Za-z0-9+.-]*):)?"  # scheme (3.1)
+    r"(?://([^/?#]*))?"  # authority (3.2)
+    r"([^?#]*)"  # path (3.3)
+    r"(?:\?([^#]*))?"  # query (3.4)
+    r"(?:#(.*))?"  # fragment (3.5)
+)
+
+
+def split(text: str) -> Split:
+    """The five components of `text`. Any text splits: nothing is validated."""
+    return Split(*_SPLIT.match(text).groups())
+
+
+def decode(text: str, errors: str = "strict", encoding: str = "utf-8") -> str:
+    """`text` with each `%XX` escape replaced by its byte, the bytes read as
+    `encoding`.
+
+    Text that is not an escape stays as it is: a `%` that is not followed by two
+    hexadecimal digits (in either case) is a `%`, and non-ASCII characters are
+    kept. Bytes that are not valid in `encoding` are handled by `errors`, and
+    `surrogateescape` carries each through the result."""
+    if encoding == "utf-8" and "%" not in text and text.isascii():
+        return text
+    return _unquote_to_bytes(text.encode(encoding, errors)).decode(encoding, errors)
+
+
+def encode(
+    data: "str | bytes",
+    safe: str = "",
+    *,
+    encoding: str = "utf-8",
+    errors: str = "strict",
+) -> str:
+    """`data` percent-encoded: every byte that is not unreserved (RFC 3986 2.3)
+    or in `safe` becomes `%XX` with upper-case digits. A `str` is first encoded
+    as `encoding` with `errors`; `bytes` are used as they are. `safe` is ASCII."""
+    if not isinstance(data, bytes):
+        data = data.encode(encoding, errors)
+    return _quote_from_bytes(data, safe)

uripath/_transfer.py

@@ -0,0 +1,108 @@
+"""`copy_stream`: the bytes of a path or a binary stream into another."""
+
+from __future__ import annotations
+
+import errno as _errno
+import os as _os
+import shutil as _shutil
+import typing as _ty
+
+from pathapi import Path
+
+from ._pathtext import as_path
+from .exceptions import OutputClosedError
+
+__all__ = ["copy_stream"]
+
+#: Bytes read and written at a time.
+_CHUNK_SIZE = 1024 * 1024
+
+#: Windows reports a write to a pipe whose reader has gone as EINVAL.
+_WINDOWS = _os.name == "nt"
+
+_Endpoint = _ty.Union[str, "_os.PathLike[str]", Path, _ty.BinaryIO]
+
+
+def _is_closed_pipe(error: OSError) -> bool:
+    return isinstance(error, BrokenPipeError) or (
+        _WINDOWS and error.errno == _errno.EINVAL
+    )
+
+
+class _PipeWriter:
+    """Writes to a stream the caller owns, turning a closed pipe there into
+    `OutputClosedError`."""
+
+    __slots__ = ("_stream",)
+
+    def __init__(self, stream):
+        self._stream = stream
+
+    def write(self, data):
+        try:
+            return self._stream.write(data)
+        except OSError as error:
+            if _is_closed_pipe(error):
+                raise OutputClosedError(error.errno, "output closed") from error
+            raise
+
+    def flush(self):
+        try:
+            self._stream.flush()
+        except OSError as error:
+            if _is_closed_pipe(error):
+                raise OutputClosedError(error.errno, "output closed") from error
+            raise
+
+
+def _path_of(endpoint) -> Path:
+    if isinstance(endpoint, Path):
+        return endpoint
+    if isinstance(endpoint, str):
+        return as_path(endpoint)
+    if isinstance(endpoint, _os.PathLike):
+        return as_path(_os.fspath(endpoint))
+    raise TypeError(
+        "expected a path, path text or a binary stream, "
+        f"not {type(endpoint).__name__!r}"
+    )
+
+
+def _copy(reader, target, exclusive: bool, chunk_size: int) -> None:
+    if callable(getattr(target, "write", None)):
+        writer = _PipeWriter(target)
+        _shutil.copyfileobj(reader, writer, chunk_size)
+        writer.flush()
+        return
+    with _path_of(target).open("xb" if exclusive else "wb") as writer:
+        _shutil.copyfileobj(reader, writer, chunk_size)
+
+
+def copy_stream(
+    source: _Endpoint,
+    target: _Endpoint,
+    *,
+    exclusive: bool = False,
+    chunk_size: int = _CHUNK_SIZE,
+) -> None:
+    """Copy the bytes of `source` to `target`, `chunk_size` at a time, never
+    holding the whole object in memory.
+
+    Each end is a binary stream (anything with `read()` or `write()`), a path
+    object, or text that `as_path()` turns into a path. A path is opened here
+    and closed when the copy ends; a stream stays open and is the caller's.
+    The target stream is flushed at the end.
+
+    `exclusive` refuses a target path that exists (`FileExistsError`), as an
+    exclusive create does: there is no check-then-write race. A target stream
+    has nothing to refuse.
+
+    A target stream whose reader went away (a pipe whose other end closed,
+    which Windows reports as `OSError(EINVAL)`) raises `OutputClosedError`.
+    The same error from the connection of a path is not translated.
+    """
+    if callable(getattr(source, "read", None)):
+        _copy(source, target, exclusive, chunk_size)
+        return
+    with _path_of(source).open("rb") as reader:
+        _copy(reader, target, exclusive, chunk_size)