upload-guard 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- upload_guard-0.1.0/CHANGELOG.md +33 -0
- upload_guard-0.1.0/LICENSE +21 -0
- upload_guard-0.1.0/MANIFEST.in +5 -0
- upload_guard-0.1.0/PKG-INFO +419 -0
- upload_guard-0.1.0/README.md +361 -0
- upload_guard-0.1.0/SECURITY.md +48 -0
- upload_guard-0.1.0/examples/django_app/conftest.py +7 -0
- upload_guard-0.1.0/examples/django_app/manage.py +10 -0
- upload_guard-0.1.0/examples/django_app/pytest.ini +2 -0
- upload_guard-0.1.0/examples/django_app/requirements.txt +2 -0
- upload_guard-0.1.0/examples/django_app/samples.py +25 -0
- upload_guard-0.1.0/examples/django_app/settings.py +19 -0
- upload_guard-0.1.0/examples/django_app/tests/test_upload.py +52 -0
- upload_guard-0.1.0/examples/django_app/urls.py +8 -0
- upload_guard-0.1.0/examples/django_app/views.py +52 -0
- upload_guard-0.1.0/examples/e2e_demo.py +203 -0
- upload_guard-0.1.0/examples/fastapi_app/main.py +42 -0
- upload_guard-0.1.0/examples/fastapi_app/pytest.ini +2 -0
- upload_guard-0.1.0/examples/fastapi_app/requirements.txt +2 -0
- upload_guard-0.1.0/examples/fastapi_app/samples.py +25 -0
- upload_guard-0.1.0/examples/fastapi_app/test_main.py +57 -0
- upload_guard-0.1.0/examples/flask_app/app.py +42 -0
- upload_guard-0.1.0/examples/flask_app/pytest.ini +2 -0
- upload_guard-0.1.0/examples/flask_app/requirements.txt +2 -0
- upload_guard-0.1.0/examples/flask_app/samples.py +25 -0
- upload_guard-0.1.0/examples/flask_app/test_app.py +56 -0
- upload_guard-0.1.0/pyproject.toml +87 -0
- upload_guard-0.1.0/setup.cfg +4 -0
- upload_guard-0.1.0/src/upload_guard/__init__.py +41 -0
- upload_guard-0.1.0/src/upload_guard/_version.py +1 -0
- upload_guard-0.1.0/src/upload_guard/adapters.py +214 -0
- upload_guard-0.1.0/src/upload_guard/cli.py +97 -0
- upload_guard-0.1.0/src/upload_guard/exceptions.py +33 -0
- upload_guard-0.1.0/src/upload_guard/extension.py +179 -0
- upload_guard-0.1.0/src/upload_guard/guard.py +266 -0
- upload_guard-0.1.0/src/upload_guard/integrations/__init__.py +9 -0
- upload_guard-0.1.0/src/upload_guard/integrations/_http.py +31 -0
- upload_guard-0.1.0/src/upload_guard/integrations/django.py +79 -0
- upload_guard-0.1.0/src/upload_guard/integrations/fastapi.py +73 -0
- upload_guard-0.1.0/src/upload_guard/integrations/flask.py +50 -0
- upload_guard-0.1.0/src/upload_guard/py.typed +0 -0
- upload_guard-0.1.0/src/upload_guard/security/__init__.py +12 -0
- upload_guard-0.1.0/src/upload_guard/security/archive.py +614 -0
- upload_guard-0.1.0/src/upload_guard/security/filename.py +169 -0
- upload_guard-0.1.0/src/upload_guard/security/polyglot.py +185 -0
- upload_guard-0.1.0/src/upload_guard/security/svg.py +411 -0
- upload_guard-0.1.0/src/upload_guard/sniff/__init__.py +4 -0
- upload_guard-0.1.0/src/upload_guard/sniff/containers.py +471 -0
- upload_guard-0.1.0/src/upload_guard/sniff/detector.py +56 -0
- upload_guard-0.1.0/src/upload_guard/sniff/signatures.py +293 -0
- upload_guard-0.1.0/src/upload_guard/sniff/text.py +173 -0
- upload_guard-0.1.0/src/upload_guard/types.py +182 -0
- upload_guard-0.1.0/src/upload_guard.egg-info/PKG-INFO +419 -0
- upload_guard-0.1.0/src/upload_guard.egg-info/SOURCES.txt +67 -0
- upload_guard-0.1.0/src/upload_guard.egg-info/dependency_links.txt +1 -0
- upload_guard-0.1.0/src/upload_guard.egg-info/entry_points.txt +2 -0
- upload_guard-0.1.0/src/upload_guard.egg-info/requires.txt +31 -0
- upload_guard-0.1.0/src/upload_guard.egg-info/top_level.txt +1 -0
- upload_guard-0.1.0/tests/conftest.py +155 -0
- upload_guard-0.1.0/tests/test_archive.py +165 -0
- upload_guard-0.1.0/tests/test_cli.py +36 -0
- upload_guard-0.1.0/tests/test_extension_filename.py +95 -0
- upload_guard-0.1.0/tests/test_guard.py +261 -0
- upload_guard-0.1.0/tests/test_integrations.py +119 -0
- upload_guard-0.1.0/tests/test_polyglot.py +75 -0
- upload_guard-0.1.0/tests/test_readme.py +19 -0
- upload_guard-0.1.0/tests/test_sniff.py +161 -0
- upload_guard-0.1.0/tests/test_svg.py +103 -0
- upload_guard-0.1.0/tools/readme_examples.py +81 -0
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here. The project follows Semantic Versioning; finding
|
|
4
|
+
codes (`svg.script`, `archive.bomb_size`, ...) and the `ScanResult` / `Policy` fields are part of the
|
|
5
|
+
compatibility contract.
|
|
6
|
+
|
|
7
|
+
## 0.1.0 — 2026-10-02
|
|
8
|
+
|
|
9
|
+
First release.
|
|
10
|
+
|
|
11
|
+
- Pure-Python magic-number sniffer (`upload_guard.detect`) covering ~120 formats with
|
|
12
|
+
second-stage refinement for RIFF, ISO-BMFF (`ftyp`), EBML, ZIP-based (OOXML/ODF/EPUB/JAR/APK)
|
|
13
|
+
and OLE2 (DOC/XLS/PPT/MSG/MSI) containers, plus browser-style text sniffing
|
|
14
|
+
(SVG/HTML/XML/JSON/PHP/shell/PEM/vCard/iCalendar).
|
|
15
|
+
- Extension consistency checks with alias handling and allow/deny rules
|
|
16
|
+
(`image/*`, `application/pdf`, `.docx`, `category:archive`).
|
|
17
|
+
- Filename safety checks (path traversal, null bytes, control and bidi-override characters,
|
|
18
|
+
reserved names, dangerous/double extensions) and `sanitize_filename()`.
|
|
19
|
+
- SVG scanner/sanitiser (`scan_svg`, `sanitize_svg`) removing scripts, event handlers,
|
|
20
|
+
script URIs, foreign objects, external references, dangerous CSS, entity declarations and
|
|
21
|
+
processing instructions.
|
|
22
|
+
- Archive inspection for ZIP (including nested archives, overlapping entries, size verification,
|
|
23
|
+
symlinks, Zip Slip) and gzip/bzip2/xz/tar via bounded streaming decompression (Tar Slip,
|
|
24
|
+
symlink escapes, device nodes).
|
|
25
|
+
- Polyglot detection: embedded secondary signatures and trailing data after PNG/JPEG/GIF/PDF
|
|
26
|
+
end markers.
|
|
27
|
+
- `UploadGuard` / `Policy` orchestrator with severity-based rejection and `ScanResult`.
|
|
28
|
+
- Duck-typed adapters for bytes, paths, file objects, FastAPI/Starlette `UploadFile`,
|
|
29
|
+
Django `UploadedFile` and Werkzeug/Flask `FileStorage`; stream positions are restored.
|
|
30
|
+
- Framework helpers: `integrations.fastapi` (`validate_upload`, `Guarded`),
|
|
31
|
+
`integrations.django` (`UploadGuardValidator`, `validate_upload`),
|
|
32
|
+
`integrations.flask` (`validate_upload`).
|
|
33
|
+
- `upload_guard` CLI (`scan`, `detect`).
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Aaron-lab-c
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,419 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: upload_guard
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Pure-Python upload security validator: magic-number MIME sniffing, extension consistency, SVG XSS sanitizing, zip-bomb and path-traversal protection. Zero system dependencies.
|
|
5
|
+
Author: Aaron-lab-c
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/Aaron-lab-c/upload_guard
|
|
8
|
+
Keywords: upload,security,mime,magic,libmagic,file-type,sniff,svg,xss,zip-bomb,path-traversal,fastapi,django,flask
|
|
9
|
+
Classifier: Development Status :: 4 - Beta
|
|
10
|
+
Classifier: Intended Audience :: Developers
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
19
|
+
Classifier: Framework :: Django
|
|
20
|
+
Classifier: Framework :: Django :: 4.2
|
|
21
|
+
Classifier: Framework :: Django :: 5.2
|
|
22
|
+
Classifier: Framework :: Flask
|
|
23
|
+
Classifier: Framework :: FastAPI
|
|
24
|
+
Classifier: Topic :: Security
|
|
25
|
+
Classifier: Topic :: Internet :: WWW/HTTP
|
|
26
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
27
|
+
Classifier: Typing :: Typed
|
|
28
|
+
Requires-Python: >=3.9
|
|
29
|
+
Description-Content-Type: text/markdown
|
|
30
|
+
License-File: LICENSE
|
|
31
|
+
Provides-Extra: fastapi
|
|
32
|
+
Requires-Dist: fastapi>=0.100; extra == "fastapi"
|
|
33
|
+
Requires-Dist: python-multipart; extra == "fastapi"
|
|
34
|
+
Requires-Dist: httpx; extra == "fastapi"
|
|
35
|
+
Provides-Extra: flask
|
|
36
|
+
Requires-Dist: flask>=2.0; extra == "flask"
|
|
37
|
+
Provides-Extra: django
|
|
38
|
+
Requires-Dist: Django>=3.2; extra == "django"
|
|
39
|
+
Provides-Extra: all
|
|
40
|
+
Requires-Dist: fastapi>=0.100; extra == "all"
|
|
41
|
+
Requires-Dist: python-multipart; extra == "all"
|
|
42
|
+
Requires-Dist: httpx; extra == "all"
|
|
43
|
+
Requires-Dist: flask>=2.0; extra == "all"
|
|
44
|
+
Requires-Dist: Django>=3.2; extra == "all"
|
|
45
|
+
Provides-Extra: dev
|
|
46
|
+
Requires-Dist: pytest>=7; extra == "dev"
|
|
47
|
+
Requires-Dist: fastapi>=0.100; extra == "dev"
|
|
48
|
+
Requires-Dist: python-multipart; extra == "dev"
|
|
49
|
+
Requires-Dist: httpx; extra == "dev"
|
|
50
|
+
Requires-Dist: flask>=2.0; extra == "dev"
|
|
51
|
+
Requires-Dist: Django>=3.2; extra == "dev"
|
|
52
|
+
Requires-Dist: mypy; extra == "dev"
|
|
53
|
+
Requires-Dist: ruff; extra == "dev"
|
|
54
|
+
Requires-Dist: bandit; extra == "dev"
|
|
55
|
+
Requires-Dist: build; extra == "dev"
|
|
56
|
+
Requires-Dist: twine; extra == "dev"
|
|
57
|
+
Dynamic: license-file
|
|
58
|
+
|
|
59
|
+
# upload_guard
|
|
60
|
+
|
|
61
|
+
**Pure-Python upload validation: real file-type detection, extension consistency, SVG XSS sanitising, zip-bomb and path-traversal protection — with zero system dependencies.**
|
|
62
|
+
|
|
63
|
+
[](https://pypi.org/project/upload_guard/)
|
|
64
|
+

|
|
65
|
+

|
|
66
|
+
|
|
67
|
+
`python-magic` needs `libmagic` (`.so` / `.dll`) and breaks on Alpine, distroless images and Windows dev boxes. Most back-ends also forget that an "image" can be a PHP web-shell, that SVGs run JavaScript, and that a 40 KB zip can expand to 4 PB. `upload_guard` fixes all of that in one call, using only the standard library.
|
|
68
|
+
|
|
69
|
+
> 中文摘要在[最後一節](#繁體中文簡介)。可直接執行的 FastAPI / Flask / Django 範例在 [examples/](examples/),威脅模型見 [SECURITY.md](SECURITY.md)。
|
|
70
|
+
|
|
71
|
+
## Contents
|
|
72
|
+
|
|
73
|
+
1. [Quick start](#quick-start-30-seconds)
|
|
74
|
+
2. [What it checks](#what-it-checks)
|
|
75
|
+
3. [Install](#install)
|
|
76
|
+
4. [Usage](#usage)
|
|
77
|
+
5. [Framework integration](#framework-integration) — [FastAPI](#fastapi) · [Django](#django) · [Flask](#flask)
|
|
78
|
+
6. [CLI](#cli)
|
|
79
|
+
7. [Design notes](#design-notes)
|
|
80
|
+
8. [Limitations](#limitations)
|
|
81
|
+
9. [繁體中文簡介](#繁體中文簡介)
|
|
82
|
+
|
|
83
|
+
## Quick start (30 seconds)
|
|
84
|
+
|
|
85
|
+
```bash
|
|
86
|
+
pip install upload_guard
|
|
87
|
+
```
|
|
88
|
+
|
|
89
|
+
<!-- run -->
|
|
90
|
+
```python
|
|
91
|
+
from upload_guard import UploadGuard, UploadRejected
|
|
92
|
+
|
|
93
|
+
guard = UploadGuard(allowed=["image/*", "application/pdf", ".docx"], max_size=10 * 1024 * 1024)
|
|
94
|
+
|
|
95
|
+
png = b"\x89PNG\r\n\x1a\n" + bytes(64) # any upload: bytes, path, file object, UploadFile...
|
|
96
|
+
result = guard.check(png, filename="cat.png")
|
|
97
|
+
print(result.mime, result.safe_filename) # image/png cat.png
|
|
98
|
+
|
|
99
|
+
try:
|
|
100
|
+
guard.check(b"MZ" + bytes(200), filename="cat.png") # a Windows executable renamed to .png
|
|
101
|
+
except UploadRejected as exc:
|
|
102
|
+
print("REJECTED", exc.codes) # ['extension.mismatch', 'type.blocked', 'type.not_allowed']
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
In a request handler you would then store `result.sanitized or data` under `result.safe_filename`.
|
|
106
|
+
|
|
107
|
+
## What it checks
|
|
108
|
+
|
|
109
|
+
| Module | What it catches |
|
|
110
|
+
|---|---|
|
|
111
|
+
| **Magic-number sniffer** | Reads the first 2 KiB and identifies ~120 formats (PNG/JPEG/WebP/HEIC/AVIF, PDF, legacy and OOXML Office, ODF, EPUB, ZIP/7z/RAR/tar/gz/xz, PE/ELF/Mach-O, fonts, audio/video, SQLite, Parquet…). Container formats are refined: `RIFF`→WAV/AVI/WebP, `ftyp`→MP4/MOV/HEIC, ZIP→DOCX/XLSX/PPTX/JAR/APK/EPUB, OLE2→DOC/XLS/PPT/MSG/MSI. Text is classified as SVG/HTML/XML/JSON/PHP/shell/PEM/vCard… the way a browser would sniff it. |
|
|
112
|
+
| **Extension consistency** | Declared extension vs. detected type (with aliases `jpg`/`jpeg`, `tif`/`tiff`…); allow/deny rules like `"image/*"`, `"application/pdf"`, `".docx"`, `"category:archive"`; declared `Content-Type` vs. reality. |
|
|
113
|
+
| **Filename safety** | `../` traversal, absolute paths, null bytes, control characters, **RTLO / bidi-override spoofing** (`invoice_gnp.exe`), zero-width characters, Windows reserved names (`CON`, `LPT1`), `.htaccess`/`web.config`, dangerous and double extensions (`shell.php.jpg`). Plus `sanitize_filename()` that returns something safe to store. |
|
|
114
|
+
| **SVG XSS** | `<script>`, `on*` handlers, `javascript:`/`data:text/html` URIs (including obfuscated `java\tscript:`), `<foreignObject>`, HTML elements, external `<use>`/`<image>` references, CSS `expression()`/`@import`/`url()`, `<!ENTITY>` (XXE / billion laughs), `<?xml-stylesheet?>`. Either **report** or **sanitise** (returns clean bytes). |
|
|
115
|
+
| **Archive bombs** | ZIP: total uncompressed size, compression ratio, entry count, nested archives (recursively, bounded), overlapping-entry bombs, lying headers (optional real decompression with a cap), encrypted entries, symlinks, **Zip Slip** paths. gzip/bzip2/xz/tar: bounded streaming decompression that never buffers more than 64 KiB, tar header walk (GNU long names, PAX), **Tar Slip**, symlink escapes, device nodes. Also applies to DOCX/XLSX/PPTX/ODF (they are zips). |
|
|
116
|
+
| **Polyglots** | Secondary signatures inside images/documents (`<?php` in a GIF comment, ZIP after a GIF = GIFAR, PDF inside PNG, `<script>` in JPEG) and data appended after the format's EOF marker (PNG `IEND`, JPEG `FFD9`, GIF `;`, PDF `%%EOF`) — the classic "image with a web-shell appended" trick. |
|
|
117
|
+
|
|
118
|
+
Everything runs with hard limits on bytes read, elements parsed and decompressed output, so the validator itself cannot be used as a DoS vector.
|
|
119
|
+
|
|
120
|
+
## Install
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
pip install upload_guard
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
No compiled extensions, no `libmagic`, no third-party runtime dependencies. Python 3.9+ (the setuptools>=77 build backend needs 3.9; the wheel itself is pure Python).
|
|
127
|
+
|
|
128
|
+
## Usage
|
|
129
|
+
|
|
130
|
+
### One-shot functions
|
|
131
|
+
|
|
132
|
+
```python
|
|
133
|
+
from upload_guard import check, scan, detect_type, sanitize_svg, sanitize_filename, inspect_archive
|
|
134
|
+
|
|
135
|
+
detect_type(b"\x89PNG\r\n\x1a\n...").mime # 'image/png'
|
|
136
|
+
detect_type("/tmp/upload.bin") # DetectedType(mime=..., extensions=(...), category=...)
|
|
137
|
+
|
|
138
|
+
result = scan(data, filename="cat.png") # never raises
|
|
139
|
+
result.ok, result.mime, result.findings
|
|
140
|
+
|
|
141
|
+
check(data, filename="cat.png", allowed=["image/*"]) # raises UploadRejected
|
|
142
|
+
|
|
143
|
+
clean = sanitize_svg(svg_bytes).sanitized # bytes without scripts/handlers/external refs
|
|
144
|
+
sanitize_filename("../../etc/passwd\x00.png") # 'passwd.png'
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
### Policy
|
|
148
|
+
|
|
149
|
+
```python
|
|
150
|
+
from upload_guard import UploadGuard, Policy, Severity, ArchiveLimits, SvgPolicy
|
|
151
|
+
|
|
152
|
+
guard = UploadGuard(
|
|
153
|
+
allowed=["image/*", "application/pdf", ".docx", ".xlsx"], # evaluated against the *detected* type
|
|
154
|
+
blocked=["category:executable", "category:script"], # default
|
|
155
|
+
max_size=20 * 1024 * 1024,
|
|
156
|
+
require_extension=True, # filename must have an extension
|
|
157
|
+
strict_extension=True, # ...and it must match the detected type
|
|
158
|
+
check_content_type=True, # declared Content-Type mismatch → LOW finding
|
|
159
|
+
allow_unknown=False, # unidentifiable binary → rejected
|
|
160
|
+
sanitize_svg=True, # clean SVGs instead of rejecting them
|
|
161
|
+
svg_policy=SvgPolicy(allow_external_references=False, allow_data_images=True),
|
|
162
|
+
inspect_archives=True,
|
|
163
|
+
archive_limits=ArchiveLimits(
|
|
164
|
+
max_total_uncompressed=512 * 1024 * 1024,
|
|
165
|
+
max_ratio=100.0,
|
|
166
|
+
max_entries=10_000,
|
|
167
|
+
max_nesting=1,
|
|
168
|
+
verify_sizes=False, # True: actually decompress ZIP entries (bounded) to catch lying headers
|
|
169
|
+
allow_symlinks=False,
|
|
170
|
+
),
|
|
171
|
+
polyglot_scan_limit=1024 * 1024,
|
|
172
|
+
check_trailing_data=True,
|
|
173
|
+
reject_threshold=Severity.MEDIUM, # findings at or above this severity reject the upload
|
|
174
|
+
)
|
|
175
|
+
```
|
|
176
|
+
|
|
177
|
+
Every `UploadGuard` keyword is a field of `Policy`; you can also pass a `Policy` object and override individual fields: `UploadGuard(policy, max_size=5_000_000)`.
|
|
178
|
+
|
|
179
|
+
### Reading the result
|
|
180
|
+
|
|
181
|
+
```python
|
|
182
|
+
result = guard.scan(upload)
|
|
183
|
+
|
|
184
|
+
result.ok # bool
|
|
185
|
+
result.detected # DetectedType(mime, extensions, description, category)
|
|
186
|
+
result.mime # 'image/png'
|
|
187
|
+
result.extension # extension to store the file with (detected, else declared)
|
|
188
|
+
result.safe_filename # sanitised filename, extension added if missing
|
|
189
|
+
result.size
|
|
190
|
+
result.findings # list[Finding(code, severity, message, detail, remediated)]
|
|
191
|
+
result.errors # findings that caused rejection
|
|
192
|
+
result.warnings # everything else (including remediated SVG problems)
|
|
193
|
+
result.archive # ArchiveReport(entry_count, total_uncompressed, ratio, nested_archives, unsafe_paths…)
|
|
194
|
+
result.sanitized # cleaned SVG bytes when sanitising was enabled, else None
|
|
195
|
+
result.to_dict() # JSON-serialisable
|
|
196
|
+
```
|
|
197
|
+
|
|
198
|
+
Finding codes are stable strings, grouped by prefix: `size.*`, `type.*`, `extension.*`, `content_type.*`, `filename.*`, `svg.*`, `archive.*`, `polyglot.*`.
|
|
199
|
+
|
|
200
|
+
Severity semantics:
|
|
201
|
+
|
|
202
|
+
| Severity | Meaning | Examples |
|
|
203
|
+
|---|---|---|
|
|
204
|
+
| `CRITICAL` | Definitely hostile | `svg.script`, `archive.bomb_size`, `filename.path_traversal`, `polyglot.embedded_php` |
|
|
205
|
+
| `HIGH` | Violates policy or strongly suspicious | `extension.mismatch`, `type.blocked`, `size.too_large`, `svg.foreign_object` |
|
|
206
|
+
| `MEDIUM` | Suspicious, usually worth rejecting (default threshold) | `polyglot.trailing_data`, `svg.external_reference`, `filename.double_extension` (executable inner ext) |
|
|
207
|
+
| `LOW` | Informational / lying browsers | `content_type.mismatch`, `filename.hidden`, `archive.encrypted` |
|
|
208
|
+
| `INFO` | Notes | `filename.double_extension` (`.tar.gz`) |
|
|
209
|
+
|
|
210
|
+
## Framework integration
|
|
211
|
+
|
|
212
|
+
Inputs are duck-typed, so plain `bytes`, `io.BytesIO`, paths, open files, Starlette/FastAPI `UploadFile`, Django `UploadedFile` and Werkzeug/Flask `FileStorage` all work directly. The stream position is always restored, so you can still `save()` / `read()` afterwards.
|
|
213
|
+
|
|
214
|
+
### FastAPI
|
|
215
|
+
|
|
216
|
+
Full runnable app with tests: [examples/fastapi_app](examples/fastapi_app). `validate_upload` raises
|
|
217
|
+
`HTTPException` 413 / 415 / 422 with a JSON body listing the findings; `Guarded(...)` is the same check as a
|
|
218
|
+
dependency.
|
|
219
|
+
|
|
220
|
+
<!-- include: examples/fastapi_app/main.py -->
|
|
221
|
+
```python
|
|
222
|
+
# examples/fastapi_app/main.py
|
|
223
|
+
import os
|
|
224
|
+
from pathlib import Path
|
|
225
|
+
|
|
226
|
+
from fastapi import Depends, FastAPI, File, UploadFile
|
|
227
|
+
|
|
228
|
+
from upload_guard import UploadGuard
|
|
229
|
+
from upload_guard.integrations.fastapi import Guarded, validate_upload
|
|
230
|
+
|
|
231
|
+
# ---- 1. 設定一次:允許的類型以「偵測到的真實類型」為準 -------------------------------
|
|
232
|
+
guard = UploadGuard(
|
|
233
|
+
allowed=["image/*", "application/pdf", ".docx"],
|
|
234
|
+
max_size=5 * 1024 * 1024,
|
|
235
|
+
sanitize_svg=True, # SVG 會被消毒後接受,而不是整個拒絕
|
|
236
|
+
)
|
|
237
|
+
|
|
238
|
+
app = FastAPI()
|
|
239
|
+
|
|
240
|
+
|
|
241
|
+
def upload_dir() -> Path:
|
|
242
|
+
path = Path(os.environ.get("UPLOAD_DIR", "uploads"))
|
|
243
|
+
path.mkdir(parents=True, exist_ok=True)
|
|
244
|
+
return path
|
|
245
|
+
|
|
246
|
+
|
|
247
|
+
# ---- 2. 在 handler 內驗證:失敗時自動回 413 / 415 / 422 --------------------------------
|
|
248
|
+
@app.post("/upload")
|
|
249
|
+
async def upload(file: UploadFile = File(...)):
|
|
250
|
+
result = validate_upload(file, guard) # raises HTTPException on rejection
|
|
251
|
+
data = result.sanitized or await file.read() # 消毒後的 SVG,否則原始內容(位置已還原)
|
|
252
|
+
(upload_dir() / result.safe_filename).write_bytes(data)
|
|
253
|
+
return {
|
|
254
|
+
"mime": result.mime,
|
|
255
|
+
"filename": result.safe_filename,
|
|
256
|
+
"warnings": [f.code for f in result.warnings],
|
|
257
|
+
}
|
|
258
|
+
|
|
259
|
+
|
|
260
|
+
# ---- 3. 或者當成 dependency:欄位名由 field= 決定 -----------------------------------------
|
|
261
|
+
@app.post("/avatar")
|
|
262
|
+
async def avatar(result=Depends(Guarded(guard, field="avatar"))):
|
|
263
|
+
return {"mime": result.mime, "size": result.size}
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
### Django
|
|
267
|
+
|
|
268
|
+
Full runnable project with tests: [examples/django_app](examples/django_app). `UploadGuardValidator` works on
|
|
269
|
+
`forms.FileField` and on model `FileField` / `ImageField` (it is `@deconstructible`, so migrations are fine);
|
|
270
|
+
`validate_upload` raises `ValidationError` with one error per finding.
|
|
271
|
+
|
|
272
|
+
<!-- include: examples/django_app/views.py -->
|
|
273
|
+
```python
|
|
274
|
+
# examples/django_app/views.py
|
|
275
|
+
from pathlib import Path
|
|
276
|
+
|
|
277
|
+
from django import forms
|
|
278
|
+
from django.conf import settings
|
|
279
|
+
from django.core.exceptions import ValidationError
|
|
280
|
+
from django.http import JsonResponse
|
|
281
|
+
from django.views.decorators.csrf import csrf_exempt
|
|
282
|
+
from django.views.decorators.http import require_POST
|
|
283
|
+
|
|
284
|
+
from upload_guard import UploadGuard
|
|
285
|
+
from upload_guard.integrations.django import UploadGuardValidator, validate_upload
|
|
286
|
+
|
|
287
|
+
# ---- 1. 設定一次 --------------------------------------------------------------------
|
|
288
|
+
guard = UploadGuard(allowed=["image/*", "application/pdf"], max_size=5 * 1024 * 1024)
|
|
289
|
+
|
|
290
|
+
|
|
291
|
+
# ---- 2a. 表單 / Model 欄位:掛 validator,錯誤進 form.errors ------------------------------
|
|
292
|
+
class UploadForm(forms.Form):
|
|
293
|
+
file = forms.FileField(validators=[UploadGuardValidator(guard)])
|
|
294
|
+
|
|
295
|
+
|
|
296
|
+
@csrf_exempt # 範例用;真實專案請走 CSRF token 或 API key 驗證
|
|
297
|
+
@require_POST
|
|
298
|
+
def upload_form(request):
|
|
299
|
+
form = UploadForm(request.POST, request.FILES)
|
|
300
|
+
if not form.is_valid():
|
|
301
|
+
return JsonResponse({"errors": form.errors.get_json_data()}, status=400)
|
|
302
|
+
f = form.cleaned_data["file"]
|
|
303
|
+
result = guard.scan(f) # 取得 ScanResult(消毒結果、安全檔名)
|
|
304
|
+
_store(result, f)
|
|
305
|
+
return JsonResponse({"mime": result.mime, "filename": result.safe_filename})
|
|
306
|
+
|
|
307
|
+
|
|
308
|
+
# ---- 2b. 直接在 view 內驗證:失敗時拋 ValidationError -------------------------------------
|
|
309
|
+
@csrf_exempt
|
|
310
|
+
@require_POST
|
|
311
|
+
def upload_api(request):
|
|
312
|
+
f = request.FILES["file"]
|
|
313
|
+
try:
|
|
314
|
+
result = validate_upload(f, guard)
|
|
315
|
+
except ValidationError as exc:
|
|
316
|
+
return JsonResponse({"errors": [{"code": e.code, "message": e.message} for e in exc.error_list]}, status=400)
|
|
317
|
+
_store(result, f)
|
|
318
|
+
return JsonResponse({"mime": result.mime, "filename": result.safe_filename, "warnings": [w.code for w in result.warnings]})
|
|
319
|
+
|
|
320
|
+
|
|
321
|
+
def _store(result, uploaded):
|
|
322
|
+
root = Path(settings.MEDIA_ROOT)
|
|
323
|
+
root.mkdir(parents=True, exist_ok=True)
|
|
324
|
+
data = result.sanitized if result.sanitized is not None else uploaded.read() # 位置已還原
|
|
325
|
+
(root / result.safe_filename).write_bytes(data)
|
|
326
|
+
```
|
|
327
|
+
|
|
328
|
+
### Flask
|
|
329
|
+
|
|
330
|
+
Full runnable app with tests: [examples/flask_app](examples/flask_app). `validate_upload` aborts with the
|
|
331
|
+
matching Werkzeug `HTTPException` (413 / 415 / 422); use `guard.scan()` when you want to shape the response
|
|
332
|
+
yourself.
|
|
333
|
+
|
|
334
|
+
<!-- include: examples/flask_app/app.py -->
|
|
335
|
+
```python
|
|
336
|
+
# examples/flask_app/app.py
|
|
337
|
+
import os
|
|
338
|
+
from pathlib import Path
|
|
339
|
+
|
|
340
|
+
from flask import Flask, jsonify, request
|
|
341
|
+
|
|
342
|
+
from upload_guard import UploadGuard
|
|
343
|
+
from upload_guard.integrations.flask import validate_upload
|
|
344
|
+
|
|
345
|
+
# ---- 1. 設定一次 --------------------------------------------------------------------
|
|
346
|
+
guard = UploadGuard(allowed=["image/*", "application/pdf"], max_size=5 * 1024 * 1024)
|
|
347
|
+
|
|
348
|
+
|
|
349
|
+
def create_app(upload_dir=None):
|
|
350
|
+
app = Flask(__name__)
|
|
351
|
+
app.config["UPLOAD_DIR"] = Path(upload_dir or os.environ.get("UPLOAD_DIR", "uploads"))
|
|
352
|
+
app.config["MAX_CONTENT_LENGTH"] = 6 * 1024 * 1024 # 第一道防線:Werkzeug 直接擋掉超大請求
|
|
353
|
+
|
|
354
|
+
# ---- 2. 驗證失敗時 validate_upload 會 abort(413/415/422),Flask 回對應錯誤頁 -------------
|
|
355
|
+
@app.post("/upload")
|
|
356
|
+
def upload():
|
|
357
|
+
f = request.files["file"]
|
|
358
|
+
result = validate_upload(f, guard)
|
|
359
|
+
app.config["UPLOAD_DIR"].mkdir(parents=True, exist_ok=True)
|
|
360
|
+
target = app.config["UPLOAD_DIR"] / result.safe_filename # 永遠用消毒後的檔名
|
|
361
|
+
if result.sanitized is not None:
|
|
362
|
+
target.write_bytes(result.sanitized) # 消毒後的 SVG
|
|
363
|
+
else:
|
|
364
|
+
f.save(target) # 串流位置已還原,可直接存
|
|
365
|
+
return jsonify(mime=result.mime, filename=result.safe_filename, warnings=[w.code for w in result.warnings])
|
|
366
|
+
|
|
367
|
+
# ---- 3. 想自己決定回應格式:用 scan() 取得完整結果 -------------------------------------------
|
|
368
|
+
@app.post("/check")
|
|
369
|
+
def check():
|
|
370
|
+
result = guard.scan(request.files["file"])
|
|
371
|
+
return jsonify(result.to_dict()), (200 if result.ok else 400)
|
|
372
|
+
|
|
373
|
+
return app
|
|
374
|
+
|
|
375
|
+
|
|
376
|
+
if __name__ == "__main__":
|
|
377
|
+
create_app().run(debug=True)
|
|
378
|
+
```
|
|
379
|
+
|
|
380
|
+
## CLI
|
|
381
|
+
|
|
382
|
+
```bash
|
|
383
|
+
upload_guard scan photo.png report.pdf --allow "image/*" --allow application/pdf
|
|
384
|
+
upload_guard scan logo.svg --write-sanitized clean.svg
|
|
385
|
+
upload_guard scan suspicious.zip --json
|
|
386
|
+
upload_guard detect *.bin
|
|
387
|
+
```
|
|
388
|
+
|
|
389
|
+
Exit code `0` = all accepted, `1` = at least one rejected, `2` = I/O error.
|
|
390
|
+
|
|
391
|
+
## Design notes
|
|
392
|
+
|
|
393
|
+
* **Detected, not declared.** Allow-lists and extension checks are evaluated against the sniffed type. A `.png` that is really a PE file fails `extension.mismatch` *and* `type.blocked` even if `image/*` is allowed.
|
|
394
|
+
* **Browsers sniff too.** Text starting with `<html`, `<script`, `<svg` is classified as HTML/SVG regardless of extension, because that is what a browser will render — which is exactly how stored-XSS via "text" uploads happens.
|
|
395
|
+
* **Bounded everything.** 2 KiB for sniffing, 1 MiB (configurable) for polyglot scanning, 64 KiB tail for trailing-data checks, streaming decompression with a hard output cap, element/depth caps for SVG parsing. The validator cannot be turned into the bomb.
|
|
396
|
+
* **Remediation is tracked.** When an SVG is sanitised, the findings stay in `result.findings` with `remediated=True` for logging, but they don't reject the upload.
|
|
397
|
+
* **Lossless for callers.** Stream positions are restored; owned streams (paths, bytes) are closed; framework upload objects are left open.
|
|
398
|
+
|
|
399
|
+
## Limitations
|
|
400
|
+
|
|
401
|
+
* Encrypted ZIP entries, 7z, RAR and Zstandard archives are identified but their contents are not inspected (they produce `archive.encrypted` / no report).
|
|
402
|
+
* Detection is signature-based, not a parser: a valid-looking header followed by garbage is still "a PNG". Pair with an image decoder (Pillow `Image.verify()`) when you need structural validation.
|
|
403
|
+
* SVG sanitising uses an allow-leaning deny-list; if you need a strict allow-list of elements use `SvgPolicy.blocked_elements` to tighten further, or reject SVGs outright by excluding `image/svg+xml` from `allowed`.
|
|
404
|
+
|
|
405
|
+
## 繁體中文簡介
|
|
406
|
+
|
|
407
|
+
`upload_guard` 是不依賴 `libmagic` 的純 Python 上傳檔案檢驗套件:
|
|
408
|
+
|
|
409
|
+
* **Magic Number 鑑定**:讀前 2048 bytes 判斷真實 MIME(含 docx/xlsx/pptx、doc/xls/ppt、HEIC、WebM 等容器格式細分)。
|
|
410
|
+
* **副檔名一致性**:宣告副檔名/Content-Type 與偵測結果比對;`allowed=["image/*", ".pdf"]` 以「真實類型」為準。
|
|
411
|
+
* **檔名防護**:`../`、null byte、RTLO 字元偽裝、Windows 保留名、雙重副檔名;`sanitize_filename()` 產生安全檔名。
|
|
412
|
+
* **SVG 消毒**:移除 `<script>`、`on*`、`javascript:`、`<foreignObject>`、外部參照、危險 CSS、XML 實體宣告。
|
|
413
|
+
* **壓縮炸彈 / Zip Slip**:ZIP/TAR/gzip/bz2/xz 解壓大小與比率預估、巢狀壓縮、重疊項目、路徑穿越、symlink,全程有界串流,不落地。
|
|
414
|
+
* **Polyglot**:圖片中夾帶 PHP/ZIP/PDF/HTML 簽章、EOF 後尾隨資料。
|
|
415
|
+
* **框架友好**:直接接受 FastAPI `UploadFile`、Django `UploadedFile`、Flask `FileStorage`、`bytes`、`BytesIO`、路徑。
|
|
416
|
+
|
|
417
|
+
## License
|
|
418
|
+
|
|
419
|
+
MIT
|