mfatsftp 0.3.0__tar.gz

mfatsftp-0.3.0/.gitignore

@@ -0,0 +1,17 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+build/
+dist/
+.venv/
+venv/
+
+# Tooling
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.coverage
+htmlcov/
+docs/_build/

mfatsftp-0.3.0/CHANGELOG.md

@@ -0,0 +1,138 @@
+# Changelog
+
+All notable changes to this project are documented here. The format is based on
+[Keep a Changelog](https://keepachangelog.com/), and this project adheres to
+[Semantic Versioning](https://semver.org/).
+
+## [Unreleased]
+
+## [0.3.0] - 2026-10-18
+
+paramiko parity for the SFTP client, a documentation site, strict typing, and
+a security fix.
+
+### Added
+- Reply timeouts: `SFTPClient.settimeout()`/`gettimeout()` (also on
+  `SFTPFile`, with `setblocking`), raising the new `SFTPTimeoutError`
+  (a `TimeoutError`).
+- Line I/O on `SFTPFile`: `readline`, `readlines`, iteration, `writelines`.
+- Write buffering via `open(..., bufsize=N)` (`1` = line-buffered) and
+  `SFTPFile.set_pipelined()`; `flush()` now sends and confirms buffered data.
+- `chdir`/`getcwd` (an emulated working directory, as in paramiko).
+- `listdir_iter` (streaming directory listing).
+- `readv` on `SFTPClient` and `SFTPFile` (many ranges, one pipelined batch).
+- `SFTPClient.max_concurrent_requests` and per-call
+  `max_concurrent_prefetch_requests` on `get`/`getfo` to cap requests in flight.
+- paramiko compatibility: `prefetch=` on `get`/`getfo`, `SFTPFile.prefetch()`
+  (a no-op), `SFTPFile.chown()`, `SFTPAttributes.attr` (extended attributes,
+  previously discarded) and `SFTPAttributes.from_stat()`.
+- A garbage-collected `SFTPFile` sends its buffered writes and closes its
+  handle without waiting for replies.
+- Documentation site (Sphinx, with a reference generated from the
+  docstrings), ready for Read the Docs; `CONTRIBUTING.md` and `SECURITY.md`.
+- Tests that fail when a public member lacks a docstring or an entry in
+  `docs/api.md`, or when a code sample in the docs does not compile.
+- A release workflow that publishes to PyPI when a `v*` tag is pushed.
+
+### Changed
+- Type hints pass `mypy --strict`; CI checks them, along with the docs build.
+  Stream arguments are typed `IO[bytes]` and `env` as `Dict[str, str]`.
+
+### Fixed
+- `put_dir` now gives small files exactly their local permission bits. The
+  server's umask could strip bits from them before, for example group write.
+- The docs gave OpenSSH 9.6 as the first version with
+  `users-groups-by-id@openssh.com`; it is 9.1.
+
+### Security
+- `connect()` / `build_ssh_argv()` refuse a host that is empty or starts with
+  `-`. Before, a host such as `-oProxyCommand=…` from untrusted input was read
+  by ssh as an option and could run a local command.
+
+## [0.2.0] - 2026-10-18
+
+Brings the client up to date with the SFTP code in sshPilot, adds the
+convenience API a standalone library needs, and documents all of it.
+
+### Added
+- Pipelined transfers: reads and writes keep a window of requests in flight
+  that grows from the measured round-trip time. Chunk sizes come from the
+  server's `limits@openssh.com`. A server that returns less than it
+  advertised gets the missing pieces fetched together.
+- Batch operations that cost about one round trip per 100 paths:
+  `stat_many`, `listdir_many`, `mkdir_many`, `remove_many`, `rmdir_many`,
+  `read_small_files`, `atomic_upload_many` (with `AtomicUploadItem`).
+  Directory listings keep several READDIRs in flight.
+- `rmtree` (like `rm -r`, never follows symlinks) and `walk` (like `os.walk`).
+- `getfo` / `putfo`, a `confirm` option on `put`/`putfo`, and `copy`
+  (server-side with `copy-data`, read + write otherwise).
+- Metadata: `fstat`, `setstat`, `fsetstat`, `chown`, `utime`, `truncate`,
+  `symlink`, `readlink`.
+- OpenSSH extensions: `atomic_rename` (falls back when `posix-rename` is
+  missing or rejected), `hardlink`, `copy_data`, `statvfs` (`SFTPStatVFS`),
+  `users_groups_by_id`, and `supports_*()` checks for each.
+- `SFTPFile`: `seek`, `tell`, `truncate`, `stat`, `chmod`, `utime`,
+  `readinto` (so `io.BufferedReader`/`io.TextIOWrapper` can wrap it),
+  `readable`/`writable`/`seekable`, `closed`, `handle`.
+- `open(..., create_mode=0o600)` sets a new file's permissions in the open
+  request.
+- `SFTPClient.closed`, `SFTPClient.extensions`, `max_read_length`,
+  `max_write_length`, `max_open_handles`.
+- `SFTPAttributes.is_file()` and an `ls -l`-style `str()`.
+- Exception subclasses that match Python's built-ins: `SFTPNoSuchFileError`
+  (`FileNotFoundError`), `SFTPPermissionError` (`PermissionError`),
+  `SFTPConnectionError` (`ConnectionError`).
+- `PipelinedWriter` (via `pipelined_writer`) with an `on_ack` progress hook.
+- Full API reference (`docs/api.md`), an expanded README, and
+  `examples/transfer.py`.
+- Tests against the real OpenSSH `sftp-server` binary (including simulated
+  high-latency links), and an opt-in end-to-end test through `ssh`
+  (`MFATSFTP_TEST_HOST`).
+
+### Changed
+- `get`/`put`/`get_dir`/`put_dir` use the pipelined and batched paths. A
+  tree of many small files now costs a few round trips per directory level
+  instead of several per file.
+- `put_dir` keeps local permission bits and modification times, and writes
+  small files through a temporary name and a rename.
+- `makedirs` now raises if something other than a directory is in the way,
+  instead of silently ignoring every error.
+- `open()` modes now follow Python: `r+` no longer creates a missing file,
+  `w+` truncates, `x` fails if the file exists, `a` starts at the end, and an
+  unknown mode raises `ValueError`.
+- `SFTPFile.write` returns the number of bytes written.
+- `from_command`/`connect`: `timeout` now limits login plus handshake and
+  raises `TimeoutError`. The default is `None` (no limit). Before, the
+  handshake could hang indefinitely whatever the value.
+- Packaging: SPDX license expression (`GPL-3.0-or-later`) per PEP 639,
+  Python 3.14 classifier, ruff configuration. CI lints, builds and checks
+  the package, and runs the real-server tests.
+- License changed from MIT to GPL-3.0-or-later.
+
+### Fixed
+- Reads no longer stop short on large files, and writes larger than one
+  packet no longer drop the session. OpenSSH caps a READ at about 255 KiB and
+  closes the connection on a message over 256 KiB.
+- Filenames that are not valid UTF-8 are kept intact (`surrogateescape`)
+  instead of being replaced, so such files can be opened, renamed and
+  deleted.
+- `get_dir` refuses server-listed names containing a path separator or equal
+  to `.`/`..`, so a hostile server cannot make it write outside the target
+  directory.
+- A failed connection attempt now closes the ssh process's pipes.
+
+## [0.1.0] - initial extraction
+### Added
+- Pure-Python SFTP v3 wire-protocol codec (`mfatsftp.protocol`).
+- `SFTPClient` / `SFTPFile` over a stream pair, with a paramiko-compatible
+  surface (`open`/`file`, `stat`/`lstat`, `listdir`/`listdir_attr`,
+  `mkdir`/`makedirs`/`rmdir`, `remove`/`unlink`, `rename`/`posix_rename`,
+  `chmod`, `realpath`/`normalize`, `get`/`put`/`get_dir`/`put_dir`).
+- `connect()` / `from_command()` / `build_ssh_argv()` transport over the system
+  OpenSSH (`ssh -s <host> sftp`), with clean subprocess teardown.
+- In-memory SFTP server test fixture; protocol, client and transport test suites.
+
+[Unreleased]: https://github.com/mfat/mfatsftp/compare/v0.3.0...HEAD
+[0.3.0]: https://github.com/mfat/mfatsftp/releases/tag/v0.3.0
+[0.2.0]: https://github.com/mfat/mfatsftp/commit/09978e8
+[0.1.0]: https://github.com/mfat/mfatsftp/commit/aa26356

mfatsftp-0.3.0/CONTRIBUTING.md

@@ -0,0 +1,75 @@
+# Contributing
+
+Thanks for helping improve mfatsftp. Bug reports, fixes and documentation
+improvements are all welcome.
+
+## Setting up
+
+```sh
+git clone https://github.com/mfat/mfatsftp
+cd mfatsftp
+python -m venv .venv && . .venv/bin/activate
+pip install -e ".[dev]"
+```
+
+Most tests drive the real OpenSSH `sftp-server` binary over pipes. Install it
+if it's missing:
+
+- Debian/Ubuntu: `sudo apt install openssh-sftp-server`
+- Fedora: `sudo dnf install openssh-server`
+- macOS: already present at `/usr/libexec/sftp-server`
+
+Tests that need it are skipped, not failed, when it isn't there.
+
+## Checks
+
+These are the checks CI runs. Please run them before opening a pull request:
+
+```sh
+pytest                                   # tests
+ruff check src tests examples            # lint
+mypy --strict src/mfatsftp               # types (the package ships py.typed)
+sphinx-build -W -b html docs docs/_build/html   # docs (pip install -r docs/requirements.txt)
+```
+
+To test a real ssh login as well, point `MFATSFTP_TEST_HOST` at a host you
+can reach without a password prompt (for example `localhost` with your key in
+`~/.ssh/authorized_keys`):
+
+```sh
+MFATSFTP_TEST_HOST=localhost pytest tests/test_live_ssh.py
+```
+
+## Where things are
+
+| Path | What |
+|---|---|
+| `src/mfatsftp/protocol.py` | SFTP v3 wire codec: pure functions, no I/O |
+| `src/mfatsftp/client.py` | `SFTPClient`, `SFTPFile`, `PipelinedWriter`: requests, pipelining, transfers |
+| `src/mfatsftp/transport.py` | `connect` / `from_command`: running `ssh -s host sftp` |
+| `tests/real_server.py` | helpers for tests against `sftp-server`, including a simulated slow link |
+| `tests/fake_server.py` | in-memory SFTP server for protocol-level tests |
+| `docs/` | Sphinx site; `docs/api.md` is the hand-written reference |
+
+## Guidelines
+
+- **Tests:** add a test for every fix or feature. Prefer the real-server
+  fixture (`client` in `tests/conftest.py`) over the fake server.
+- **Speed:** if you change how requests are sent, check the round trips with
+  the simulated slow link (`start_client(delay)`), as in
+  `tests/test_large_files.py`.
+- **Docs:** public methods need a docstring and an entry in `docs/api.md`.
+  `tests/test_docs.py` fails otherwise. User-visible changes go under
+  "Unreleased" in `CHANGELOG.md`.
+- **Dependencies:** the package has no runtime dependencies beyond the
+  standard library, and it should stay that way.
+- **Compatibility:** Python 3.9+ on Linux and macOS.
+
+## Reporting bugs
+
+Open an issue with your Python version, OS, `ssh -V` output, the server
+software if known, and a minimal script that shows the problem. For security
+problems, see [SECURITY.md](https://github.com/mfat/mfatsftp/blob/main/SECURITY.md) instead.
+
+By contributing you agree that your contributions are licensed under the
+project's license (GPL-3.0-or-later).