safeuploads 2.2.0__tar.gz → 4.0.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.agents/skills/safeuploads-library/references/code-standards.md +1 -1
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/workflows/test-matrix-safeuploads.yml +2 -2
- {safeuploads-2.2.0 → safeuploads-4.0.0}/AGENTS.md +3 -3
- {safeuploads-2.2.0 → safeuploads-4.0.0}/CHANGELOG.md +59 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/CONTRIBUTING.md +1 -1
- {safeuploads-2.2.0 → safeuploads-4.0.0}/PKG-INFO +25 -10
- {safeuploads-2.2.0 → safeuploads-4.0.0}/README.md +23 -8
- {safeuploads-2.2.0 → safeuploads-4.0.0}/docs/index.md +20 -7
- {safeuploads-2.2.0 → safeuploads-4.0.0}/docs/security/architecture.md +49 -20
- {safeuploads-2.2.0 → safeuploads-4.0.0}/docs/security/integration-checklist.md +46 -10
- {safeuploads-2.2.0 → safeuploads-4.0.0}/docs/security/threat-model.md +120 -6
- {safeuploads-2.2.0 → safeuploads-4.0.0}/examples/README.md +2 -2
- {safeuploads-2.2.0 → safeuploads-4.0.0}/examples/fastapi_example.py +27 -19
- {safeuploads-2.2.0 → safeuploads-4.0.0}/pyproject.toml +5 -5
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/__init__.py +6 -12
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/config.py +206 -144
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/exceptions.py +48 -47
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/file_validator.py +198 -296
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/inspectors/zip_inspector.py +56 -4
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/utils.py +18 -2
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/__init__.py +4 -2
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/extension_validator.py +8 -4
- safeuploads-4.0.0/safeuploads/validators/image_dimension_validator.py +336 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/mime_validator.py +44 -15
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/signature_validator.py +9 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/conftest.py +18 -29
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/fuzz/test_fuzz_images.py +52 -1
- safeuploads-4.0.0/tests/images.py +188 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/inspectors/test_content_inspector.py +5 -17
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/inspectors/test_zip_inspector.py +114 -1
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_config.py +138 -81
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_config_validation.py +122 -22
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_exceptions.py +55 -32
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_file_validator.py +220 -157
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_performance.py +4 -34
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_sync_validation.py +171 -19
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_validate_activity_file.py +3 -2
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/test_extension_validator.py +53 -8
- safeuploads-4.0.0/tests/validators/test_image_dimension_validator.py +291 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/test_mime_validator.py +75 -7
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/test_signature_validator.py +37 -1
- {safeuploads-2.2.0 → safeuploads-4.0.0}/uv.lock +239 -5
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.agents/skills/conventional-commits/SKILL.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.agents/skills/conventional-commits/references/pull-requests.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.agents/skills/debugging/SKILL.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.agents/skills/safeuploads-library/SKILL.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.agents/skills/safeuploads-library/references/security.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.agents/skills/safeuploads-library/references/testing.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.agents/skills/skill-authoring/SKILL.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/CODEOWNERS +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/FUNDING.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/ISSUE_TEMPLATE/bug_report.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/ISSUE_TEMPLATE/config.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/ISSUE_TEMPLATE/feature_request.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/PULL_REQUEST_TEMPLATE.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/copilot-instructions.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/dependabot.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/scripts/check_conventional_commits.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/scripts/smoke_import.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/workflows/audit-safeuploads.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/workflows/conventional-commits.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/workflows/docs.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/workflows/fuzz.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/workflows/lint-and-test.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/workflows/mirror.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.github/workflows/publish-safeuploads.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/.gitignore +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/CODE_OF_CONDUCT.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/LICENSE.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/SECURITY.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/docs/api.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/docs/rate-limiting.md +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/mkdocs.yml +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/audit.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/enums.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/inspectors/__init__.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/inspectors/base.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/inspectors/content_inspector.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/inspectors/gzip_inspector.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/protocols.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/py.typed +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/base.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/compression_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/filename_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/fit_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/unicode_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/windows_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/safeuploads/validators/xml_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/__init__.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/fuzz/__init__.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/fuzz/helpers.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/fuzz/test_fuzz_config.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/fuzz/test_fuzz_filenames.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/fuzz/test_fuzz_zip.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/inspectors/__init__.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/inspectors/test_gzip_inspector.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_audit.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_enums.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_fastapi_integration.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_utils.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/test_validate_gzip_file.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/__init__.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/test_compression_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/test_filename_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/test_fit_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/test_unicode_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/test_windows_validator.py +0 -0
- {safeuploads-2.2.0 → safeuploads-4.0.0}/tests/validators/test_xml_validator.py +0 -0
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Code standards
|
|
2
2
|
|
|
3
3
|
## Python
|
|
4
|
-
- Python 3.
|
|
4
|
+
- Python 3.11 syntax and standard library (`requires-python = ">=3.11"`): `int | None`, `list[str]`, `dict[str, Any]`; no `typing.Optional/List/Dict`, and no 3.12+ features such as PEP 695 type parameters, `type` aliases or `typing.override`. The 3.11 CI job is the check.
|
|
5
5
|
- Lines up to 79 characters, enforced by `ruff format` (`line-length = 79`); keep comments and docstrings within it too.
|
|
6
6
|
- Imports grouped stdlib / third-party / local; `TYPE_CHECKING` to avoid cycles.
|
|
7
7
|
- Every module has a top-level docstring.
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
name: Test matrix safeuploads
|
|
2
2
|
|
|
3
3
|
# Portability matrix on top of the fast default `Lint & Test` workflow:
|
|
4
|
-
# * every supported Python version (3.
|
|
4
|
+
# * every supported Python version (3.11 to 3.15) with the full
|
|
5
5
|
# dependency set (dev group + fastapi extra), proving the suite
|
|
6
6
|
# passes on every Python the package declares support for; and
|
|
7
7
|
# * a bare `import safeuploads` with only the base dependencies
|
|
@@ -40,7 +40,7 @@ jobs:
|
|
|
40
40
|
strategy:
|
|
41
41
|
fail-fast: false
|
|
42
42
|
matrix:
|
|
43
|
-
python: ["3.13", "3.14", "3.15"]
|
|
43
|
+
python: ["3.11", "3.12", "3.13", "3.14", "3.15"]
|
|
44
44
|
steps:
|
|
45
45
|
- name: Checkout code
|
|
46
46
|
uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# safeuploads
|
|
2
2
|
|
|
3
|
-
Secure file upload validation for Python 3.
|
|
3
|
+
Secure file upload validation for Python 3.11+ applications.
|
|
4
4
|
Catches dangerous filenames, malicious extensions, Windows reserved names, compression bombs, polyglots and unsafe XML before an upload is accepted.
|
|
5
5
|
Runtime dependencies are minimal (`python-magic`, `defusedxml`; FastAPI behind the `fastapi` extra).
|
|
6
6
|
Published to PyPI; the public API is governed by SemVer.
|
|
@@ -33,7 +33,7 @@ docs/ Documentation site (mkdocs)
|
|
|
33
33
|
|
|
34
34
|
## Commands
|
|
35
35
|
|
|
36
|
-
From the repository root (Python 3.
|
|
36
|
+
From the repository root (Python 3.11+, `uv`, system `libmagic`):
|
|
37
37
|
|
|
38
38
|
```bash
|
|
39
39
|
uv sync --dev --extra fastapi
|
|
@@ -97,4 +97,4 @@ uv run pytest
|
|
|
97
97
|
- Never build a filesystem path from an uploaded filename.
|
|
98
98
|
- Logging uses `logging.getLogger(__name__)` and the `SecurityAuditLogger`; never log file contents.
|
|
99
99
|
- Comments and docstrings state what the code does now, not what it used to do.
|
|
100
|
-
- Python 3.
|
|
100
|
+
- Python 3.11 syntax and standard library (no 3.12+ features), 79-character lines (`pyproject.toml`), PEP 257 docstrings with `Args`, `Returns`, `Raises` where applicable.
|
|
@@ -4,6 +4,63 @@ All notable changes to this project are documented in this file.
|
|
|
4
4
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
6
|
|
|
7
|
+
## [4.0.0] - 2026-11-06
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
- WebP image support. `FileSignatureValidator` accepts a RIFF container whose form type at bytes 8-11 is `WEBP` as an image signature; WAV, AVI and other RIFF files are still rejected. `ALLOWED_IMAGE_TYPES` maps `image/webp` to `.webp`.
|
|
12
|
+
- `ValidationResult`, exported from `safeuploads`: a frozen dataclass with the sanitized `filename` and the `mime_type` libmagic detected from the content. A ZIP or gzip that libmagic reports only as `application/octet-stream` (libmagic 5.46 reports every ZIP this way) carries `application/zip` or `application/gzip`, the type its magic signature confirmed. Use `mime_type` for the `Content-Type` of a stored or served copy, and the extension of `filename`, which is checked against `mime_type`, for a stored file's extension, as the integration checklist recommends. `mime_type` alone cannot choose the extension, since GPX and TCX share their XML types.
|
|
13
|
+
- `ErrorCode.EXTENSION_MIME_MISMATCH`, raised with `MimeTypeError` when the content's MIME type is allowed but the filename's extension is not one listed for it. `allowed_mimes` on the error lists the MIME types that extension permits.
|
|
14
|
+
- Image dimension limit against decompression (pixel) bombs. `ImageDimensionValidator`, exported from `safeuploads`, reads width and height from the PNG IHDR chunk, the first JPEG frame header or the first WebP chunk (VP8, VP8L or the VP8X canvas) without decoding pixel data, and rejects an image whose width times height exceeds the new `SecurityLimits.max_image_pixels` (default 89,478,485, Pillow's `Image.MAX_IMAGE_PIXELS`). Configuration validation rejects a limit of 0 or less (`invalid_pixel_limit`) and warns above twice the default (`excessive_pixel_limit`).
|
|
15
|
+
- `ImageDimensionError`, exported from `safeuploads`: a `FileValidationError` with `width`, `height` and `max_pixels`, raised with the new `ErrorCode.IMAGE_DIMENSIONS_EXCEEDED`.
|
|
16
|
+
- `ErrorCode.IMAGE_INVALID_STRUCTURE`, raised with `FileSignatureError` when an image's dimension header is missing, truncated or invalid.
|
|
17
|
+
- `FileSecurityConfig.ALLOWED_IMAGE_TYPES`, `ALLOWED_ZIP_TYPES`, `ALLOWED_ACTIVITY_TYPES` and `ALLOWED_GZIP_TYPES`: read-only mappings from each allowed MIME type to the extensions its content may carry. Configuration validation reports an empty mapping (`empty_mime_set`), a MIME type with no extensions (`empty_extension_set`), and the existing extension format, case and block-list conflict checks on every listed extension.
|
|
18
|
+
|
|
19
|
+
### Changed
|
|
20
|
+
|
|
21
|
+
- **Breaking:** every `validate_*_file()` method returns a `ValidationResult` instead of `None`, and every `validate_*_file_sync()` method returns a `ValidationResult` instead of the sanitized filename. Replace `name = validator.validate_image_file_sync(f, name)` with `name = validator.validate_image_file_sync(f, name).filename`.
|
|
22
|
+
- **Breaking:** an upload is rejected with `MimeTypeError` (`EXTENSION_MIME_MISMATCH`) when its extension is not one listed for the MIME type detected from its content, for example a PNG or WebP named `photo.jpg`, or a JPEG named `photo.png`. 3.0.0 accepted any allowed extension with any allowed MIME type of the same category. To accept such a pair, list the extension under that MIME type, for example `".jpg"` under `"image/webp"`.
|
|
23
|
+
- **Breaking:** `validate_image_file()` and `validate_image_file_sync()` accept `.webp` uploads with WebP content under the default configuration, which 3.0.0 rejected. A host that cannot handle WebP after validation removes `image/webp` from `ALLOWED_IMAGE_TYPES`.
|
|
24
|
+
- **Breaking:** `validate_image_file()` and `validate_image_file_sync()` reject an image whose header declares more than `max_image_pixels` pixels with `ImageDimensionError` (`IMAGE_DIMENSIONS_EXCEEDED`), and an image whose dimension header is missing or malformed with `FileSignatureError` (`IMAGE_INVALID_STRUCTURE`): a PNG whose first chunk is not IHDR, a JPEG that reaches scan data or its end without a frame header or declares a zero width or height, or a WebP whose first chunk is not VP8, VP8L or VP8X. 3.0.0 accepted any image with a valid signature. To accept larger images, raise `max_image_pixels`.
|
|
25
|
+
- Image uploads are streamed once to a spooled temporary file, like ZIP, activity and gzip uploads, instead of being read once for the size check and again for content analysis. The client's declared `size` is no longer checked before reading: an oversized image is rejected with the same `FileSizeError` (`FILE_TOO_LARGE`) once the bytes read exceed `max_image_size`, and `size` on the error is the byte count read at that point.
|
|
26
|
+
- **Breaking:** FIT uploads get the MIME check the other formats get: a `.fit` file must be detected as `application/octet-stream`. A fake FIT that libmagic reports as text is now rejected with `EXTENSION_MIME_MISMATCH` before the FIT structure check.
|
|
27
|
+
- **Breaking:** `MimeTypeValidator.enforce()` takes an allowed-types mapping instead of a set of MIME types and checks the filename's extension against it.
|
|
28
|
+
- **Breaking:** `image/jpg` is no longer an allowed image MIME type. It is not a registered type and libmagic reports JPEG content as `image/jpeg`.
|
|
29
|
+
|
|
30
|
+
### Removed
|
|
31
|
+
|
|
32
|
+
- **Breaking:** `FileSecurityConfig.ALLOWED_IMAGE_MIMES`, `ALLOWED_ZIP_MIMES`, `ALLOWED_ACTIVITY_MIMES`, `ALLOWED_GZIP_MIMES`, `ALLOWED_IMAGE_EXTENSIONS`, `ALLOWED_ZIP_EXTENSIONS`, `ALLOWED_ACTIVITY_EXTENSIONS` and `ALLOWED_GZIP_EXTENSIONS`. Each pair is replaced by one `ALLOWED_*_TYPES` mapping, so a MIME type and its extensions can no longer be changed apart. A configuration that still sets one of the removed names, on a subclass or an instance, fails validation with `removed_setting`, so `FileValidator` raises `FileSecurityConfigurationError` instead of silently applying the default allow-list. Migrate a narrowed allow-list such as
|
|
33
|
+
|
|
34
|
+
```python
|
|
35
|
+
config.ALLOWED_IMAGE_MIMES = frozenset({"image/png"})
|
|
36
|
+
config.ALLOWED_IMAGE_EXTENSIONS = frozenset({".png"})
|
|
37
|
+
```
|
|
38
|
+
|
|
39
|
+
to
|
|
40
|
+
|
|
41
|
+
```python
|
|
42
|
+
config.ALLOWED_IMAGE_TYPES = {"image/png": frozenset({".png"})}
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
## [3.0.0] - 2026-11-05
|
|
46
|
+
|
|
47
|
+
### Added
|
|
48
|
+
|
|
49
|
+
- Support for Python 3.11 and 3.12; `requires-python` is now `>=3.11` and CI tests 3.11 to 3.15. A Python version stays supported until its CPython end of life and is dropped in the next minor release after that. The Unicode checks use the running Python's Unicode database, so a character assigned after that database's version is rejected as unassigned (for example U+1FAE8 on 3.11).
|
|
50
|
+
|
|
51
|
+
### Removed
|
|
52
|
+
|
|
53
|
+
- **Breaking:** the `ConfigValidationError` alias, deprecated in 2.2.0, is removed from `safeuploads` and `safeuploads.exceptions`. Use `ConfigValidationIssue`.
|
|
54
|
+
|
|
55
|
+
### Security
|
|
56
|
+
|
|
57
|
+
- ZIP entry names now get the filename checks a top-level upload gets ([GHSA-jjqv-c38c-fjf3](https://github.com/endurain-project/safeuploads/security/advisories/GHSA-jjqv-c38c-fjf3)). `ZipContentInspector` checked entries only against `ZipThreatCategory.EXECUTABLE_FILES`, `SCRIPT_FILES` and `SYSTEM_FILES`, so an entry such as `invoice.docm`, `a.hta`, `CON.txt` or a name with a right-to-left override was accepted although the same name is rejected as an upload. Each entry name is now checked against `DANGEROUS_UNICODE_CHARS` and the rejected Unicode categories, each path component against `WINDOWS_RESERVED_NAMES`, and a file's base name against `BLOCKED_EXTENSIONS` and `COMPOUND_BLOCKED_EXTENSIONS`, on the NFC and NFKC forms, all taken from `FileSecurityConfig`. A failing entry raises `ZipContentError` with `ZIP_CONTENT_THREAT`.
|
|
58
|
+
- The blocked-extension check ignores trailing dots and spaces, as Windows drops them when it saves a file. A filename such as `run.exe ` or `invoice.docm` followed by U+3000 or U+00A0 (which NFKC folds to a space) was accepted as an upload or a ZIP entry; it is now rejected with `EXTENSION_BLOCKED`.
|
|
59
|
+
|
|
60
|
+
### Changed
|
|
61
|
+
|
|
62
|
+
- **Breaking:** the ZIP entry name checks under Security reject archives that 2.2.0 accepted under the default configuration, for example archives with `.docm`, `.xlsm`, `.hta`, `.min.css` or `CON.txt` entries. Uploads and entries whose name ends in a blocked extension followed by spaces, such as `run.exe `, are rejected as well. An entry named `.jar`, `.war` or `.ear` whose content is a ZIP now reports `ZIP_CONTENT_THREAT` first instead of `ZIP_NESTED_ARCHIVE`.
|
|
63
|
+
|
|
7
64
|
## [2.2.0] - 2026-11-05
|
|
8
65
|
|
|
9
66
|
### Added
|
|
@@ -223,6 +280,8 @@ Initial release.
|
|
|
223
280
|
- **MIME type verification:** magic-number validation for common file types.
|
|
224
281
|
- Framework-agnostic async validation, a rich exception hierarchy with machine-readable error codes, secure defaults, and full type hints.
|
|
225
282
|
|
|
283
|
+
[3.0.0]: https://github.com/endurain-project/safeuploads/compare/v2.2.0...v3.0.0
|
|
284
|
+
[2.2.0]: https://github.com/endurain-project/safeuploads/compare/v2.1.0...v2.2.0
|
|
226
285
|
[2.1.0]: https://github.com/endurain-project/safeuploads/compare/v2.0.0...v2.1.0
|
|
227
286
|
[2.0.0]: https://github.com/endurain-project/safeuploads/compare/v1.1.3...v2.0.0
|
|
228
287
|
[1.1.3]: https://github.com/endurain-project/safeuploads/compare/v1.1.2...v1.1.3
|
|
@@ -72,7 +72,7 @@ PRs that are too large to review efficiently will be asked to be split before th
|
|
|
72
72
|
docs: update the integration checklist
|
|
73
73
|
```
|
|
74
74
|
Commit subjects and PR titles are validated in CI; see `.agents/skills/conventional-commits/SKILL.md` for the allowed types and scopes.
|
|
75
|
-
5. **Run the checks** from the repository root (Python 3.
|
|
75
|
+
5. **Run the checks** from the repository root (Python 3.11+, [uv](https://docs.astral.sh/uv/) and the system `libmagic`):
|
|
76
76
|
```bash
|
|
77
77
|
uv sync --dev --extra fastapi
|
|
78
78
|
uv run ruff check safeuploads tests examples
|
|
@@ -1,11 +1,11 @@
|
|
|
1
1
|
Metadata-Version: 2.5
|
|
2
2
|
Name: safeuploads
|
|
3
|
-
Version:
|
|
3
|
+
Version: 4.0.0
|
|
4
4
|
Summary: A comprehensive file security system for validating uploads and preventing attacks
|
|
5
5
|
Author-email: João Vitória Silva <joao@endurain.com>
|
|
6
6
|
License: MIT
|
|
7
7
|
License-File: LICENSE.md
|
|
8
|
-
Requires-Python: >=3.
|
|
8
|
+
Requires-Python: >=3.11
|
|
9
9
|
Requires-Dist: defusedxml<1.0.0,>=0.7.1
|
|
10
10
|
Requires-Dist: python-magic<0.5.0,>=0.4.27
|
|
11
11
|
Provides-Extra: fastapi
|
|
@@ -18,11 +18,11 @@ Description-Content-Type: text/markdown
|
|
|
18
18
|
[](https://github.com/endurain-project/safeuploads/releases)
|
|
19
19
|
[](https://pypi.org/project/safeuploads/)
|
|
20
20
|
[](https://pypi.org/project/safeuploads/)
|
|
21
|
-
[](https://pypi.org/project/safeuploads/)
|
|
22
22
|
[](https://safeuploads.endurain.com/)
|
|
23
23
|
[](https://github.com/endurain-project/safeuploads)
|
|
24
24
|
|
|
25
|
-
Secure file upload validation for Python 3.
|
|
25
|
+
Secure file upload validation for Python 3.11+ applications. Catches dangerous filenames, malicious extensions, Windows reserved names, and compression-based attacks before you accept an upload.
|
|
26
26
|
|
|
27
27
|
## Features
|
|
28
28
|
|
|
@@ -46,6 +46,8 @@ Secure file upload validation for Python 3.13+ applications. Catches dangerous f
|
|
|
46
46
|
pip install safeuploads
|
|
47
47
|
```
|
|
48
48
|
|
|
49
|
+
safeuploads supports Python 3.11 and newer. A Python version stays supported until its CPython end of life and is dropped in the next minor release after that.
|
|
50
|
+
|
|
49
51
|
For FastAPI integration:
|
|
50
52
|
```bash
|
|
51
53
|
pip install safeuploads[fastapi]
|
|
@@ -64,14 +66,17 @@ validator = FileValidator()
|
|
|
64
66
|
@app.post("/upload")
|
|
65
67
|
async def upload_image(file: UploadFile):
|
|
66
68
|
try:
|
|
67
|
-
await validator.validate_image_file(file)
|
|
69
|
+
result = await validator.validate_image_file(file)
|
|
68
70
|
except FileValidationError as e:
|
|
69
71
|
raise HTTPException(status_code=400, detail=str(e))
|
|
70
|
-
|
|
71
|
-
|
|
72
|
+
|
|
73
|
+
# Store under a generated name; serve with the detected type
|
|
74
|
+
return {"filename": result.filename, "content_type": result.mime_type}
|
|
72
75
|
```
|
|
73
76
|
|
|
74
|
-
|
|
77
|
+
Every validation method returns a `ValidationResult` with the sanitized `filename` and the `mime_type` detected from the content. A file whose extension is not one allowed for its detected type, such as a WebP named `photo.jpg`, is rejected with `MimeTypeError` (`EXTENSION_MIME_MISMATCH`).
|
|
78
|
+
|
|
79
|
+
For synchronous code or worker threads, use the `*_sync` methods. They block the calling thread and return the same result:
|
|
75
80
|
|
|
76
81
|
```python
|
|
77
82
|
from safeuploads import FileValidator
|
|
@@ -79,7 +84,7 @@ from safeuploads import FileValidator
|
|
|
79
84
|
validator = FileValidator()
|
|
80
85
|
|
|
81
86
|
with open("activity.gpx", "rb") as fh:
|
|
82
|
-
|
|
87
|
+
result = validator.validate_activity_file_sync(fh, "activity.gpx")
|
|
83
88
|
```
|
|
84
89
|
|
|
85
90
|
Calling a `*_sync` method on a thread running an asyncio event loop raises `RuntimeError`; await the async method there instead.
|
|
@@ -95,8 +100,16 @@ validator = FileValidator()
|
|
|
95
100
|
# Or customize limits
|
|
96
101
|
config = FileSecurityConfig()
|
|
97
102
|
config.limits.max_image_size = 10 * 1024 * 1024 # 10 MiB
|
|
103
|
+
config.limits.max_image_pixels = 40_000_000 # 40 megapixels
|
|
98
104
|
config.limits.max_compression_ratio = 50
|
|
99
105
|
|
|
106
|
+
# Allowed content: each MIME type maps to the extensions its
|
|
107
|
+
# content may carry
|
|
108
|
+
config.ALLOWED_IMAGE_TYPES = {
|
|
109
|
+
"image/jpeg": frozenset({".jpg", ".jpeg"}),
|
|
110
|
+
"image/png": frozenset({".png"}),
|
|
111
|
+
}
|
|
112
|
+
|
|
100
113
|
# Opt in to strict ZIP checking: decompress every entry to
|
|
101
114
|
# reject archives with forged central-directory metadata
|
|
102
115
|
config.limits.verify_zip_decompression = True
|
|
@@ -142,9 +155,10 @@ except FileValidationError as err:
|
|
|
142
155
|
- **Extension Validation**: Allow/block lists with configurable rules, dangerous extension detection
|
|
143
156
|
- **Compression Security**: ZIP bomb detection, nested archive inspection, recursive structure and quine detection, size and ratio limits, optional strict decompression verification
|
|
144
157
|
- **Content Inspection**: Deep ZIP content analysis with configurable depth and entry limits
|
|
145
|
-
- **MIME Type Verification**: Magic number validation for images, ZIP, activity files, and gzip
|
|
158
|
+
- **MIME Type Verification**: Magic number validation for images, ZIP, activity files, and gzip, with the extension required to match the detected type
|
|
146
159
|
- **Streaming Validation**: Memory-efficient processing via `SpooledTemporaryFile` for large files
|
|
147
160
|
- **Resource Monitoring**: wall-clock time and memory limits enforced via `ResourceMonitor` during the work
|
|
161
|
+
- **Image Dimension Limits**: Header-only pixel count check for PNG, JPEG and WebP that rejects decompression (pixel) bombs before the host decodes them
|
|
148
162
|
- **Activity File Support**: GPX, TCX, and FIT file validation with XXE-safe XML parsing and FIT structure checks
|
|
149
163
|
- **Gzip Support**: Gzip archive validation with decompression bomb detection
|
|
150
164
|
- **Content Analysis**: Optional malware signature, web shell, and polyglot file detection
|
|
@@ -157,6 +171,7 @@ except FileValidationError as err:
|
|
|
157
171
|
|
|
158
172
|
- No built-in rate limiting (application-level concern — see documentation)
|
|
159
173
|
- MIME detection covers first 8 KB; advanced polyglot attacks may require `enable_content_analysis`
|
|
174
|
+
- Images are not decoded: only their dimension headers are checked, so decode and re-encode uploads in an isolated process (see the integration checklist)
|
|
160
175
|
- `SpooledTemporaryFile` uses the system default temp directory
|
|
161
176
|
|
|
162
177
|
## Documentation
|
|
@@ -4,11 +4,11 @@
|
|
|
4
4
|
[](https://github.com/endurain-project/safeuploads/releases)
|
|
5
5
|
[](https://pypi.org/project/safeuploads/)
|
|
6
6
|
[](https://pypi.org/project/safeuploads/)
|
|
7
|
-
[](https://pypi.org/project/safeuploads/)
|
|
8
8
|
[](https://safeuploads.endurain.com/)
|
|
9
9
|
[](https://github.com/endurain-project/safeuploads)
|
|
10
10
|
|
|
11
|
-
Secure file upload validation for Python 3.
|
|
11
|
+
Secure file upload validation for Python 3.11+ applications. Catches dangerous filenames, malicious extensions, Windows reserved names, and compression-based attacks before you accept an upload.
|
|
12
12
|
|
|
13
13
|
## Features
|
|
14
14
|
|
|
@@ -32,6 +32,8 @@ Secure file upload validation for Python 3.13+ applications. Catches dangerous f
|
|
|
32
32
|
pip install safeuploads
|
|
33
33
|
```
|
|
34
34
|
|
|
35
|
+
safeuploads supports Python 3.11 and newer. A Python version stays supported until its CPython end of life and is dropped in the next minor release after that.
|
|
36
|
+
|
|
35
37
|
For FastAPI integration:
|
|
36
38
|
```bash
|
|
37
39
|
pip install safeuploads[fastapi]
|
|
@@ -50,14 +52,17 @@ validator = FileValidator()
|
|
|
50
52
|
@app.post("/upload")
|
|
51
53
|
async def upload_image(file: UploadFile):
|
|
52
54
|
try:
|
|
53
|
-
await validator.validate_image_file(file)
|
|
55
|
+
result = await validator.validate_image_file(file)
|
|
54
56
|
except FileValidationError as e:
|
|
55
57
|
raise HTTPException(status_code=400, detail=str(e))
|
|
56
|
-
|
|
57
|
-
|
|
58
|
+
|
|
59
|
+
# Store under a generated name; serve with the detected type
|
|
60
|
+
return {"filename": result.filename, "content_type": result.mime_type}
|
|
58
61
|
```
|
|
59
62
|
|
|
60
|
-
|
|
63
|
+
Every validation method returns a `ValidationResult` with the sanitized `filename` and the `mime_type` detected from the content. A file whose extension is not one allowed for its detected type, such as a WebP named `photo.jpg`, is rejected with `MimeTypeError` (`EXTENSION_MIME_MISMATCH`).
|
|
64
|
+
|
|
65
|
+
For synchronous code or worker threads, use the `*_sync` methods. They block the calling thread and return the same result:
|
|
61
66
|
|
|
62
67
|
```python
|
|
63
68
|
from safeuploads import FileValidator
|
|
@@ -65,7 +70,7 @@ from safeuploads import FileValidator
|
|
|
65
70
|
validator = FileValidator()
|
|
66
71
|
|
|
67
72
|
with open("activity.gpx", "rb") as fh:
|
|
68
|
-
|
|
73
|
+
result = validator.validate_activity_file_sync(fh, "activity.gpx")
|
|
69
74
|
```
|
|
70
75
|
|
|
71
76
|
Calling a `*_sync` method on a thread running an asyncio event loop raises `RuntimeError`; await the async method there instead.
|
|
@@ -81,8 +86,16 @@ validator = FileValidator()
|
|
|
81
86
|
# Or customize limits
|
|
82
87
|
config = FileSecurityConfig()
|
|
83
88
|
config.limits.max_image_size = 10 * 1024 * 1024 # 10 MiB
|
|
89
|
+
config.limits.max_image_pixels = 40_000_000 # 40 megapixels
|
|
84
90
|
config.limits.max_compression_ratio = 50
|
|
85
91
|
|
|
92
|
+
# Allowed content: each MIME type maps to the extensions its
|
|
93
|
+
# content may carry
|
|
94
|
+
config.ALLOWED_IMAGE_TYPES = {
|
|
95
|
+
"image/jpeg": frozenset({".jpg", ".jpeg"}),
|
|
96
|
+
"image/png": frozenset({".png"}),
|
|
97
|
+
}
|
|
98
|
+
|
|
86
99
|
# Opt in to strict ZIP checking: decompress every entry to
|
|
87
100
|
# reject archives with forged central-directory metadata
|
|
88
101
|
config.limits.verify_zip_decompression = True
|
|
@@ -128,9 +141,10 @@ except FileValidationError as err:
|
|
|
128
141
|
- **Extension Validation**: Allow/block lists with configurable rules, dangerous extension detection
|
|
129
142
|
- **Compression Security**: ZIP bomb detection, nested archive inspection, recursive structure and quine detection, size and ratio limits, optional strict decompression verification
|
|
130
143
|
- **Content Inspection**: Deep ZIP content analysis with configurable depth and entry limits
|
|
131
|
-
- **MIME Type Verification**: Magic number validation for images, ZIP, activity files, and gzip
|
|
144
|
+
- **MIME Type Verification**: Magic number validation for images, ZIP, activity files, and gzip, with the extension required to match the detected type
|
|
132
145
|
- **Streaming Validation**: Memory-efficient processing via `SpooledTemporaryFile` for large files
|
|
133
146
|
- **Resource Monitoring**: wall-clock time and memory limits enforced via `ResourceMonitor` during the work
|
|
147
|
+
- **Image Dimension Limits**: Header-only pixel count check for PNG, JPEG and WebP that rejects decompression (pixel) bombs before the host decodes them
|
|
134
148
|
- **Activity File Support**: GPX, TCX, and FIT file validation with XXE-safe XML parsing and FIT structure checks
|
|
135
149
|
- **Gzip Support**: Gzip archive validation with decompression bomb detection
|
|
136
150
|
- **Content Analysis**: Optional malware signature, web shell, and polyglot file detection
|
|
@@ -143,6 +157,7 @@ except FileValidationError as err:
|
|
|
143
157
|
|
|
144
158
|
- No built-in rate limiting (application-level concern — see documentation)
|
|
145
159
|
- MIME detection covers first 8 KB; advanced polyglot attacks may require `enable_content_analysis`
|
|
160
|
+
- Images are not decoded: only their dimension headers are checked, so decode and re-encode uploads in an isolated process (see the integration checklist)
|
|
146
161
|
- `SpooledTemporaryFile` uses the system default temp directory
|
|
147
162
|
|
|
148
163
|
## Documentation
|
|
@@ -12,7 +12,7 @@
|
|
|
12
12
|
</a>
|
|
13
13
|
</div>
|
|
14
14
|
|
|
15
|
-
Secure file upload validation for Python 3.
|
|
15
|
+
Secure file upload validation for Python 3.11+ applications. Catches dangerous filenames, malicious extensions, Windows reserved names, and compression-based attacks before you accept an upload.
|
|
16
16
|
|
|
17
17
|
## Features
|
|
18
18
|
|
|
@@ -54,14 +54,17 @@ validator = FileValidator()
|
|
|
54
54
|
@app.post("/upload")
|
|
55
55
|
async def upload_image(file: UploadFile):
|
|
56
56
|
try:
|
|
57
|
-
await validator.validate_image_file(file)
|
|
57
|
+
result = await validator.validate_image_file(file)
|
|
58
58
|
except FileValidationError as e:
|
|
59
59
|
raise HTTPException(status_code=400, detail=str(e))
|
|
60
|
-
|
|
61
|
-
|
|
60
|
+
|
|
61
|
+
# Store under a generated name; serve with the detected type
|
|
62
|
+
return {"filename": result.filename, "content_type": result.mime_type}
|
|
62
63
|
```
|
|
63
64
|
|
|
64
|
-
|
|
65
|
+
Every validation method returns a `ValidationResult` with the sanitized `filename` and the `mime_type` detected from the content. A file whose extension is not one allowed for its detected type, such as a WebP named `photo.jpg`, is rejected with `MimeTypeError` (`EXTENSION_MIME_MISMATCH`).
|
|
66
|
+
|
|
67
|
+
For synchronous code or worker threads, use the `*_sync` methods. They block the calling thread and return the same result:
|
|
65
68
|
|
|
66
69
|
```python
|
|
67
70
|
from safeuploads import FileValidator
|
|
@@ -69,7 +72,7 @@ from safeuploads import FileValidator
|
|
|
69
72
|
validator = FileValidator()
|
|
70
73
|
|
|
71
74
|
with open("activity.gpx", "rb") as fh:
|
|
72
|
-
|
|
75
|
+
result = validator.validate_activity_file_sync(fh, "activity.gpx")
|
|
73
76
|
```
|
|
74
77
|
|
|
75
78
|
Calling a `*_sync` method on a thread running an asyncio event loop raises `RuntimeError`; await the async method there instead.
|
|
@@ -85,8 +88,16 @@ validator = FileValidator()
|
|
|
85
88
|
# Or customize limits
|
|
86
89
|
config = FileSecurityConfig()
|
|
87
90
|
config.limits.max_image_size = 10 * 1024 * 1024 # 10 MiB
|
|
91
|
+
config.limits.max_image_pixels = 40_000_000 # 40 megapixels
|
|
88
92
|
config.limits.max_compression_ratio = 50
|
|
89
93
|
|
|
94
|
+
# Allowed content: each MIME type maps to the extensions its
|
|
95
|
+
# content may carry
|
|
96
|
+
config.ALLOWED_IMAGE_TYPES = {
|
|
97
|
+
"image/jpeg": frozenset({".jpg", ".jpeg"}),
|
|
98
|
+
"image/png": frozenset({".png"}),
|
|
99
|
+
}
|
|
100
|
+
|
|
90
101
|
# Opt in to strict ZIP checking: decompress every entry to
|
|
91
102
|
# reject archives with forged central-directory metadata
|
|
92
103
|
config.limits.verify_zip_decompression = True
|
|
@@ -132,9 +143,10 @@ except FileValidationError as err:
|
|
|
132
143
|
- **Extension Validation**: Allow/block lists with configurable rules, dangerous extension detection
|
|
133
144
|
- **Compression Security**: ZIP bomb detection, nested archive inspection, recursive structure and quine detection, size and ratio limits, optional strict decompression verification
|
|
134
145
|
- **Content Inspection**: Deep ZIP content analysis with configurable depth and entry limits
|
|
135
|
-
- **MIME Type Verification**: Magic number validation for images, ZIP, activity files, and gzip
|
|
146
|
+
- **MIME Type Verification**: Magic number validation for images, ZIP, activity files, and gzip, with the extension required to match the detected type
|
|
136
147
|
- **Streaming Validation**: Memory-efficient processing via `SpooledTemporaryFile` for large files
|
|
137
148
|
- **Resource Monitoring**: wall-clock time and memory limits enforced via `ResourceMonitor` during the work
|
|
149
|
+
- **Image Dimension Limits**: Header-only pixel count check for PNG, JPEG and WebP that rejects decompression (pixel) bombs before the host decodes them
|
|
138
150
|
- **Activity File Support**: GPX, TCX, and FIT file validation with XXE-safe XML parsing and FIT structure checks
|
|
139
151
|
- **Gzip Support**: Gzip archive validation with decompression bomb detection
|
|
140
152
|
- **Content Analysis**: Optional malware signature, web shell, and polyglot file detection
|
|
@@ -147,6 +159,7 @@ except FileValidationError as err:
|
|
|
147
159
|
|
|
148
160
|
- No built-in rate limiting (application-level concern — see [Rate Limiting](rate-limiting.md) guide)
|
|
149
161
|
- MIME detection covers first 8 KB; advanced polyglot attacks may require `enable_content_analysis`
|
|
162
|
+
- Images are not decoded: only their dimension headers are checked, so decode and re-encode uploads in an isolated process (see the integration checklist)
|
|
150
163
|
- `SpooledTemporaryFile` uses the system default temp directory
|
|
151
164
|
|
|
152
165
|
## Documentation
|
|
@@ -25,6 +25,7 @@ safeuploads/
|
|
|
25
25
|
│ ├── mime_validator.py # Content MIME detection and enforcement
|
|
26
26
|
│ ├── windows_validator.py # Windows reserved name checks
|
|
27
27
|
│ ├── fit_validator.py # FIT container structure checks
|
|
28
|
+
│ ├── image_dimension_validator.py # Pixel count from image headers
|
|
28
29
|
│ ├── compression_validator.py # ZIP bomb detection
|
|
29
30
|
│ └── xml_validator.py # XXE-safe XML parsing
|
|
30
31
|
└── inspectors/
|
|
@@ -39,19 +40,20 @@ safeuploads/
|
|
|
39
40
|
| Component | Responsibility |
|
|
40
41
|
|---|---|
|
|
41
42
|
| `FileValidator` | Orchestrates validation for each file type. Manages streaming, resource monitoring and audit events, and delegates to validators/inspectors. The async methods offload blocking inspection to a worker thread (or a configured `executor`); the `*_sync` methods run the same pipeline in the calling thread and refuse to run on an event-loop thread. Exposes them as `filename_validator`, `signature_validator`, `mime_validator` and the other validator attributes; `magic_mime` and `magic_available` are properties that delegate to `mime_validator`. |
|
|
42
|
-
| `FileSecurityConfig` | Centralizes all configuration: allowed MIME types
|
|
43
|
+
| `FileSecurityConfig` | Centralizes all configuration: allowed MIME types mapped to the extensions each permits, blocked extensions, Unicode characters, and Windows reserved names. |
|
|
43
44
|
| `SecurityLimits` | Holds numeric thresholds: file sizes, compression ratios, timeouts, entry limits, resource caps. |
|
|
44
45
|
| `BaseValidator` | Abstract base class; validators inherit and implement `validate()`. |
|
|
45
46
|
| `UnicodeSecurityValidator` | NFC normalization and dangerous character detection (rejects with `UnicodeSecurityError`). |
|
|
46
47
|
| `ExtensionSecurityValidator` | Blocked, dangerous and compound extension checks. `validate_allowed` checks an extension against the allowlist and the blocked list. |
|
|
47
48
|
| `WindowsSecurityValidator` | Rejects Windows reserved device names. |
|
|
48
49
|
| `FilenameSecurityValidator` | Strips path components and dangerous characters from a filename and limits its length to `max_sanitized_name_length`. |
|
|
49
|
-
| `FileSignatureValidator` | Checks that the file header matches the signatures of the expected type (image, ZIP, gzip, activity or FIT). |
|
|
50
|
-
| `MimeTypeValidator` | Detects the MIME type from content with libmagic (serialized by a lock, never falling back to the filename)
|
|
50
|
+
| `FileSignatureValidator` | Checks that the file header matches the signatures of the expected type (image: JPEG, PNG or WebP; ZIP, gzip, activity or FIT). |
|
|
51
|
+
| `MimeTypeValidator` | Detects the MIME type from content with libmagic (serialized by a lock, never falling back to the filename), enforces the allowed MIME types and requires the filename's extension to be one listed for the detected type. |
|
|
51
52
|
| `CompressionSecurityValidator` | Checks compression ratio, uncompressed size, entry count, and nested archive limits, and that the listed entries fill the file exactly. |
|
|
52
53
|
| `XmlSecurityValidator` | Parses XML with `defusedxml`, blocking DTDs and external entities (raises `XmlSecurityError`). |
|
|
53
54
|
| `FitSecurityValidator` | Checks the FIT container: headers, header CRC, data size, CRC-16 per section, and that the sections cover the file exactly. |
|
|
54
|
-
| `
|
|
55
|
+
| `ImageDimensionValidator` | Reads width and height from the PNG IHDR chunk, the first JPEG frame header (SOF) or the first WebP chunk (VP8, VP8L or VP8X canvas) without decoding pixels, and rejects an image whose pixel count exceeds `max_image_pixels` (raises `ImageDimensionError`). A missing or malformed dimension header raises `FileSignatureError` with `IMAGE_INVALID_STRUCTURE`. |
|
|
56
|
+
| `ZipContentInspector` | Iterates ZIP entries checking for traversal, executables, scripts, symlinks, and recursive structures, and runs the top-level Unicode, Windows reserved-name and blocked-extension checks on each entry name. With `allow_nested_archives`, inspects every inner ZIP like the top level. |
|
|
55
57
|
| `GzipContentInspector` | Streams gzip decompression, checking ratio and size progressively. |
|
|
56
58
|
| `ContentSecurityInspector` | Scans raw bytes for structurally verified executable headers, web shells, and polyglot patterns (image and activity uploads). |
|
|
57
59
|
| `ResourceMonitor` | Context manager enforcing wall-clock time and peak-RSS memory limits at loop boundaries and on exit. |
|
|
@@ -61,7 +63,7 @@ safeuploads/
|
|
|
61
63
|
|
|
62
64
|
## Validation Pipelines
|
|
63
65
|
|
|
64
|
-
The `validate_*_file_sync` methods run the identical steps listed below in the calling thread, without the worker-thread offload.
|
|
66
|
+
The `validate_*_file_sync` methods run the identical steps listed below in the calling thread, without the worker-thread offload. Every method returns a `ValidationResult` with the sanitized filename and the detected MIME type.
|
|
65
67
|
|
|
66
68
|
### Image Validation (`validate_image_file`)
|
|
67
69
|
|
|
@@ -91,14 +93,18 @@ UploadFile
|
|
|
91
93
|
│
|
|
92
94
|
▼
|
|
93
95
|
┌─────────────────────────────┐
|
|
94
|
-
│ 4.
|
|
95
|
-
│ (
|
|
96
|
+
│ 4. Stream to SpooledTempFile│
|
|
97
|
+
│ (size checked per chunk; │
|
|
98
|
+
│ memory < 10MB → disk) │
|
|
96
99
|
└─────────────┬───────────────┘
|
|
97
100
|
│
|
|
98
101
|
▼
|
|
99
102
|
┌─────────────────────────────┐
|
|
100
|
-
│ 5. MIME Type
|
|
103
|
+
│ 5. MIME Type Check │
|
|
101
104
|
│ (python-magic, first 8KB)│
|
|
105
|
+
│ ├── Type allowed │
|
|
106
|
+
│ └── Extension listed for │
|
|
107
|
+
│ the detected type │
|
|
102
108
|
└─────────────┬───────────────┘
|
|
103
109
|
│
|
|
104
110
|
▼
|
|
@@ -109,7 +115,18 @@ UploadFile
|
|
|
109
115
|
│
|
|
110
116
|
▼
|
|
111
117
|
┌─────────────────────────────┐
|
|
112
|
-
│ 7.
|
|
118
|
+
│ 7. Dimension Check │
|
|
119
|
+
│ (header only, no decode) │
|
|
120
|
+
│ ├── PNG IHDR │
|
|
121
|
+
│ ├── JPEG frame header │
|
|
122
|
+
│ ├── WebP VP8/VP8L/VP8X │
|
|
123
|
+
│ └── width × height ≤ │
|
|
124
|
+
│ max_image_pixels │
|
|
125
|
+
└─────────────┬───────────────┘
|
|
126
|
+
│
|
|
127
|
+
▼
|
|
128
|
+
┌─────────────────────────────┐
|
|
129
|
+
│ 8. Content Analysis │
|
|
113
130
|
│ (if enabled) │
|
|
114
131
|
│ ├── Executable signatures│
|
|
115
132
|
│ ├── Script injection │
|
|
@@ -118,13 +135,13 @@ UploadFile
|
|
|
118
135
|
│
|
|
119
136
|
▼
|
|
120
137
|
┌─────────────────────────────┐
|
|
121
|
-
│
|
|
138
|
+
│ 9. ResourceMonitor END │
|
|
122
139
|
└─────────────┬───────────────┘
|
|
123
140
|
│
|
|
124
141
|
▼
|
|
125
|
-
|
|
126
|
-
│
|
|
127
|
-
|
|
142
|
+
┌──────────────────────────────┐
|
|
143
|
+
│ 10. Audit: VALIDATION_SUCCESS│
|
|
144
|
+
└──────────────────────────────┘
|
|
128
145
|
```
|
|
129
146
|
|
|
130
147
|
### ZIP Validation (`validate_zip_file`)
|
|
@@ -162,6 +179,8 @@ UploadFile
|
|
|
162
179
|
▼
|
|
163
180
|
┌─────────────────────────────┐
|
|
164
181
|
│ 6. MIME + Signature Check │
|
|
182
|
+
│ (extension listed for │
|
|
183
|
+
│ the detected type) │
|
|
165
184
|
└─────────────┬───────────────┘
|
|
166
185
|
│
|
|
167
186
|
▼
|
|
@@ -181,7 +200,11 @@ UploadFile
|
|
|
181
200
|
│ │ ├── Null bytes │
|
|
182
201
|
│ │ ├── Path traversal │
|
|
183
202
|
│ │ ├── Absolute paths │
|
|
184
|
-
│ │
|
|
203
|
+
│ │ ├── Symlinks │
|
|
204
|
+
│ │ └── Top-level name │
|
|
205
|
+
│ │ checks (Unicode, │
|
|
206
|
+
│ │ Windows names, │
|
|
207
|
+
│ │ blocked exts) │
|
|
185
208
|
│ ├── Dangerous entry types│
|
|
186
209
|
│ ├── Binary signatures │
|
|
187
210
|
│ ├── Nested archive magic │
|
|
@@ -241,8 +264,10 @@ UploadFile (.gpx, .tcx, .fit)
|
|
|
241
264
|
│ 5. MIME + Signature Check │
|
|
242
265
|
│ ├── FIT: binary signature │
|
|
243
266
|
│ │ (.FIT at bytes 8-11) │
|
|
244
|
-
│
|
|
245
|
-
│
|
|
267
|
+
│ ├── GPX/TCX: XML header │
|
|
268
|
+
│ │ (<?xml) │
|
|
269
|
+
│ └── Extension listed for │
|
|
270
|
+
│ the detected type │
|
|
246
271
|
└──────────────┬───────────────┘
|
|
247
272
|
│
|
|
248
273
|
▼
|
|
@@ -306,6 +331,8 @@ UploadFile (.gz)
|
|
|
306
331
|
┌──────────────────────────────┐
|
|
307
332
|
│ 5. MIME + Signature Check │
|
|
308
333
|
│ (gzip magic: \x1f\x8b) │
|
|
334
|
+
│ (extension listed for │
|
|
335
|
+
│ the detected type) │
|
|
309
336
|
└──────────────┬───────────────┘
|
|
310
337
|
│
|
|
311
338
|
▼
|
|
@@ -333,8 +360,7 @@ UploadFile (.gz)
|
|
|
333
360
|
|
|
334
361
|
| Stage | What is read | Buffer size |
|
|
335
362
|
|---|---|---|
|
|
336
|
-
|
|
|
337
|
-
| Stream to temp | Chunked reads to `SpooledTemporaryFile` | In-memory up to `max_memory_buffer_size` (10 MB), then disk |
|
|
363
|
+
| Stream to temp (size check) | Chunked reads (`chunk_size`, 64 KB default) to `SpooledTemporaryFile`, counted against the type's size limit | In-memory up to `max_memory_buffer_size` (10 MB), then disk |
|
|
338
364
|
| MIME detection | First 8 KB via `python-magic` | 8 KB |
|
|
339
365
|
| Signature check | First 4-12 bytes depending on format | Minimal |
|
|
340
366
|
| Compression validation | ZIP central directory, local headers and data descriptors | Metadata; entries with a data descriptor are inflated in `chunk_size` chunks |
|
|
@@ -343,6 +369,7 @@ UploadFile (.gz)
|
|
|
343
369
|
| Gzip inspection | Chunked streaming decompression | `chunk_size` at a time |
|
|
344
370
|
| XML validation | Chunked feed to `defusedxml`'s parser, no element tree built | `chunk_size` at a time |
|
|
345
371
|
| FIT validation | Chunked CRC-16 over each section | `chunk_size` at a time |
|
|
372
|
+
| Image dimension check | PNG and WebP: first 30 bytes. JPEG: the marker and length of each segment up to the frame header, skipping segment payloads by seek | Minimal; a run of 0xFF fill bytes is read `chunk_size` at a time |
|
|
346
373
|
| Content analysis | Byte scan of image and activity uploads (not ZIP) | Bounded by `content_scan_max_size` (50 MB); about twice the sample while matching |
|
|
347
374
|
|
|
348
375
|
---
|
|
@@ -361,6 +388,7 @@ Exception
|
|
|
361
388
|
│ ├── FileSizeError
|
|
362
389
|
│ ├── MimeTypeError
|
|
363
390
|
│ ├── FileSignatureError
|
|
391
|
+
│ ├── ImageDimensionError
|
|
364
392
|
│ ├── CompressionSecurityError
|
|
365
393
|
│ │ ├── ZipBombError
|
|
366
394
|
│ │ └── ZipContentError
|
|
@@ -384,8 +412,9 @@ All `FileSecurityError` subclasses carry an `error_code` from
|
|
|
384
412
|
| `ExtensionSecurityError` | `EXTENSION_BLOCKED` |
|
|
385
413
|
| `WindowsReservedNameError` | `WINDOWS_RESERVED_NAME` |
|
|
386
414
|
| `FileSizeError` | `FILE_TOO_LARGE` |
|
|
387
|
-
| `MimeTypeError` | `MIME_TYPE_INVALID` (uploads whose detected MIME is not allowed are reported as `MIME_TYPE_MISMATCH`) |
|
|
388
|
-
| `FileSignatureError` | `FILE_SIGNATURE_MISMATCH` |
|
|
415
|
+
| `MimeTypeError` | `MIME_TYPE_INVALID` (uploads whose detected MIME is not allowed are reported as `MIME_TYPE_MISMATCH`, and an extension not listed for the detected MIME as `EXTENSION_MIME_MISMATCH`) |
|
|
416
|
+
| `FileSignatureError` | `FILE_SIGNATURE_MISMATCH` (a missing or invalid FIT structure is reported as `FIT_INVALID_STRUCTURE`, and a missing or invalid image dimension header as `IMAGE_INVALID_STRUCTURE`) |
|
|
417
|
+
| `ImageDimensionError` | `IMAGE_DIMENSIONS_EXCEEDED` |
|
|
389
418
|
| `CompressionSecurityError` | `ZIP_INVALID_STRUCTURE` |
|
|
390
419
|
| `ZipBombError` | `ZIP_BOMB_DETECTED` |
|
|
391
420
|
| `ZipContentError` | `ZIP_CONTENT_THREAT` (the ZIP inspector passes the code of the first threat found) |
|