zipremove 0.7.1__tar.gz → 0.8.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {zipremove-0.7.1/src/zipremove.egg-info → zipremove-0.8.1}/PKG-INFO +53 -25
- {zipremove-0.7.1 → zipremove-0.8.1}/README.md +47 -19
- zipremove-0.8.1/pyproject.toml +118 -0
- zipremove-0.8.1/setup.cfg +4 -0
- {zipremove-0.7.1 → zipremove-0.8.1}/src/zipremove/__init__.py +30 -11
- {zipremove-0.7.1 → zipremove-0.8.1/src/zipremove.egg-info}/PKG-INFO +53 -25
- {zipremove-0.7.1 → zipremove-0.8.1}/src/zipremove.egg-info/SOURCES.txt +0 -2
- {zipremove-0.7.1 → zipremove-0.8.1}/tests/test_zipfile.py +198 -88
- {zipremove-0.7.1 → zipremove-0.8.1}/tests/test_zipfile64.py +6 -2
- zipremove-0.7.1/pyproject.toml +0 -3
- zipremove-0.7.1/setup.cfg +0 -55
- zipremove-0.7.1/setup.py +0 -2
- {zipremove-0.7.1 → zipremove-0.8.1}/LICENSE.txt +0 -0
- {zipremove-0.7.1 → zipremove-0.8.1}/src/zipremove.egg-info/dependency_links.txt +0 -0
- {zipremove-0.7.1 → zipremove-0.8.1}/src/zipremove.egg-info/top_level.txt +0 -0
|
@@ -1,12 +1,11 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: zipremove
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.8.1
|
|
4
4
|
Summary: Extend `zipfile` with `remove`-related functionalities
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
Classifier: Development Status :: 4 - Beta
|
|
5
|
+
Author-email: Danny Lin <danny0838@gmail.com>
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/danny0838/zipremove
|
|
8
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
10
9
|
Classifier: Intended Audience :: Developers
|
|
11
10
|
Classifier: Topic :: System :: Archiving :: Compression
|
|
12
11
|
Classifier: Programming Language :: Python :: 3
|
|
@@ -17,6 +16,7 @@ Classifier: Programming Language :: Python :: 3.11
|
|
|
17
16
|
Classifier: Programming Language :: Python :: 3.12
|
|
18
17
|
Classifier: Programming Language :: Python :: 3.13
|
|
19
18
|
Classifier: Programming Language :: Python :: 3.14
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.15
|
|
20
20
|
Classifier: Operating System :: OS Independent
|
|
21
21
|
Requires-Python: ~=3.9
|
|
22
22
|
Description-Content-Type: text/markdown
|
|
@@ -38,8 +38,9 @@ This package extends `zipfile` with `remove`-related functionalities.
|
|
|
38
38
|
|
|
39
39
|
Removes a member entry from the archive's central directory.
|
|
40
40
|
*zinfo_or_arcname* may be the full path of the member or a `ZipInfo`
|
|
41
|
-
instance. If multiple members share the same
|
|
42
|
-
provided, only one
|
|
41
|
+
instance. If multiple members share the same path and a string is
|
|
42
|
+
provided, only one unspecified entry is removed; pass a specific
|
|
43
|
+
`ZipInfo` instance to guarantee which is removed.
|
|
43
44
|
|
|
44
45
|
The archive must be opened with mode ``'w'``, ``'x'`` or ``'a'``.
|
|
45
46
|
|
|
@@ -51,26 +52,44 @@ This package extends `zipfile` with `remove`-related functionalities.
|
|
|
51
52
|
> This method only removes the member's entry from the central directory,
|
|
52
53
|
> making it inaccessible to most tools. The member's local file entry,
|
|
53
54
|
> including content and metadata, remains in the archive and is still
|
|
54
|
-
> recoverable
|
|
55
|
-
>
|
|
55
|
+
> forensically recoverable. To completely delete the data and reclaim
|
|
56
|
+
> space, call `repack` afterwards (preferably passing the returned `ZipInfo`
|
|
57
|
+
> instance).
|
|
56
58
|
|
|
57
|
-
* `ZipFile.repack(removed=None, *, strict_descriptor=
|
|
59
|
+
* `ZipFile.repack(removed=None, *, strict_descriptor=True[, chunk_size])`
|
|
58
60
|
|
|
59
61
|
Rewrites the archive to remove unreferenced local file entries, shrinking
|
|
60
62
|
its file size. The archive must be opened with mode ``'a'``.
|
|
61
63
|
|
|
62
64
|
If *removed* is provided, it must be a sequence of `ZipInfo` objects
|
|
63
65
|
representing the recently removed members, and only their corresponding
|
|
64
|
-
local file entries will be removed.
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
66
|
+
local file entries will be removed. This is the most efficient and reliable
|
|
67
|
+
way to reclaim space. For example:
|
|
68
|
+
|
|
69
|
+
```python
|
|
70
|
+
with ZipFile('spam.zip', 'a') as myzip:
|
|
71
|
+
removed = [myzip.remove(name) for name in ('ham.txt', 'eggs.txt')]
|
|
72
|
+
myzip.repack(removed)
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
If *removed* is omitted, the archive is scanned to locate and remove local
|
|
76
|
+
file entries that are no longer referenced in the central directory.
|
|
77
|
+
|
|
78
|
+
When scanning, *strict_descriptor* controls how entries written with an
|
|
79
|
+
unsigned *data descriptor* are handled. A data descriptor is an optional
|
|
80
|
+
record holding an entry's CRC and sizes, stored just after the entry's
|
|
81
|
+
data; it is used when the archive is written to a non-seekable stream, and
|
|
82
|
+
is *signed* when it begins with a marker signature or *unsigned* otherwise.
|
|
83
|
+
Unsigned descriptors have been deprecated by the [PKZIP Application Note]
|
|
84
|
+
since version 6.3.0 (released in 2006) and are written only by some legacy
|
|
85
|
+
tools; signed descriptors—written by Python and other modern tools—are
|
|
86
|
+
always detected. When *strict_descriptor* is true (the default), unsigned
|
|
87
|
+
descriptors are not detected, and related unreferenced entries are not
|
|
88
|
+
removed. Setting `strict_descriptor=False` additionally detects unsigned
|
|
89
|
+
descriptors, at the cost of a significantly slower scan—around 100 to 1000
|
|
90
|
+
times in the worst case—which may be exploitable as a denial-of-service
|
|
91
|
+
vector on untrusted input. This does not affect entries without a data
|
|
92
|
+
descriptor, and is not needed when *removed* is provided.
|
|
74
93
|
|
|
75
94
|
*chunk_size* may be specified to control the buffer size when moving
|
|
76
95
|
entry data (default is 1 MiB).
|
|
@@ -106,6 +125,17 @@ This package extends `zipfile` with `remove`-related functionalities.
|
|
|
106
125
|
|
|
107
126
|
Calling `copy` on a closed ZipFile will raise a `ValueError`.
|
|
108
127
|
|
|
128
|
+
> **Note:**
|
|
129
|
+
> Renaming a member in a ZIP file requires rewriting its data, as the
|
|
130
|
+
> filename is stored within its local file entry.
|
|
131
|
+
>
|
|
132
|
+
> To rename a member and reclaim the space occupied by the old entry,
|
|
133
|
+
> combine `copy`, `remove`, and `repack` like:
|
|
134
|
+
>
|
|
135
|
+
> ```python
|
|
136
|
+
> with ZipFile('spam.zip', 'a') as myzip:
|
|
137
|
+
> myzip.repack([myzip.remove(myzip.copy('old.txt', 'new.txt'))])
|
|
138
|
+
> ```
|
|
109
139
|
|
|
110
140
|
## Examples
|
|
111
141
|
|
|
@@ -161,10 +191,6 @@ print(os.path.getsize('archive.zip')) # 116 (would be 245 without `repack`)
|
|
|
161
191
|
|
|
162
192
|
### Move entries under a folder and reclaim space
|
|
163
193
|
|
|
164
|
-
Moving entries in a ZIP file must be done as a combination of `copy`, `remove`,
|
|
165
|
-
and optionally `repack`, because every local file entry contains the filename
|
|
166
|
-
and requires rewriting.
|
|
167
|
-
|
|
168
194
|
```python
|
|
169
195
|
import os
|
|
170
196
|
import zipremove as zipfile
|
|
@@ -213,3 +239,5 @@ with zipfile.ZipFile('archive.zip', 'a') as zh:
|
|
|
213
239
|
|
|
214
240
|
print(os.path.getsize('archive.zip')) # 446 (would be 599 without `repack`)
|
|
215
241
|
```
|
|
242
|
+
|
|
243
|
+
[PKZIP Application Note]: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
|
|
@@ -13,8 +13,9 @@ This package extends `zipfile` with `remove`-related functionalities.
|
|
|
13
13
|
|
|
14
14
|
Removes a member entry from the archive's central directory.
|
|
15
15
|
*zinfo_or_arcname* may be the full path of the member or a `ZipInfo`
|
|
16
|
-
instance. If multiple members share the same
|
|
17
|
-
provided, only one
|
|
16
|
+
instance. If multiple members share the same path and a string is
|
|
17
|
+
provided, only one unspecified entry is removed; pass a specific
|
|
18
|
+
`ZipInfo` instance to guarantee which is removed.
|
|
18
19
|
|
|
19
20
|
The archive must be opened with mode ``'w'``, ``'x'`` or ``'a'``.
|
|
20
21
|
|
|
@@ -26,26 +27,44 @@ This package extends `zipfile` with `remove`-related functionalities.
|
|
|
26
27
|
> This method only removes the member's entry from the central directory,
|
|
27
28
|
> making it inaccessible to most tools. The member's local file entry,
|
|
28
29
|
> including content and metadata, remains in the archive and is still
|
|
29
|
-
> recoverable
|
|
30
|
-
>
|
|
30
|
+
> forensically recoverable. To completely delete the data and reclaim
|
|
31
|
+
> space, call `repack` afterwards (preferably passing the returned `ZipInfo`
|
|
32
|
+
> instance).
|
|
31
33
|
|
|
32
|
-
* `ZipFile.repack(removed=None, *, strict_descriptor=
|
|
34
|
+
* `ZipFile.repack(removed=None, *, strict_descriptor=True[, chunk_size])`
|
|
33
35
|
|
|
34
36
|
Rewrites the archive to remove unreferenced local file entries, shrinking
|
|
35
37
|
its file size. The archive must be opened with mode ``'a'``.
|
|
36
38
|
|
|
37
39
|
If *removed* is provided, it must be a sequence of `ZipInfo` objects
|
|
38
40
|
representing the recently removed members, and only their corresponding
|
|
39
|
-
local file entries will be removed.
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
41
|
+
local file entries will be removed. This is the most efficient and reliable
|
|
42
|
+
way to reclaim space. For example:
|
|
43
|
+
|
|
44
|
+
```python
|
|
45
|
+
with ZipFile('spam.zip', 'a') as myzip:
|
|
46
|
+
removed = [myzip.remove(name) for name in ('ham.txt', 'eggs.txt')]
|
|
47
|
+
myzip.repack(removed)
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
If *removed* is omitted, the archive is scanned to locate and remove local
|
|
51
|
+
file entries that are no longer referenced in the central directory.
|
|
52
|
+
|
|
53
|
+
When scanning, *strict_descriptor* controls how entries written with an
|
|
54
|
+
unsigned *data descriptor* are handled. A data descriptor is an optional
|
|
55
|
+
record holding an entry's CRC and sizes, stored just after the entry's
|
|
56
|
+
data; it is used when the archive is written to a non-seekable stream, and
|
|
57
|
+
is *signed* when it begins with a marker signature or *unsigned* otherwise.
|
|
58
|
+
Unsigned descriptors have been deprecated by the [PKZIP Application Note]
|
|
59
|
+
since version 6.3.0 (released in 2006) and are written only by some legacy
|
|
60
|
+
tools; signed descriptors—written by Python and other modern tools—are
|
|
61
|
+
always detected. When *strict_descriptor* is true (the default), unsigned
|
|
62
|
+
descriptors are not detected, and related unreferenced entries are not
|
|
63
|
+
removed. Setting `strict_descriptor=False` additionally detects unsigned
|
|
64
|
+
descriptors, at the cost of a significantly slower scan—around 100 to 1000
|
|
65
|
+
times in the worst case—which may be exploitable as a denial-of-service
|
|
66
|
+
vector on untrusted input. This does not affect entries without a data
|
|
67
|
+
descriptor, and is not needed when *removed* is provided.
|
|
49
68
|
|
|
50
69
|
*chunk_size* may be specified to control the buffer size when moving
|
|
51
70
|
entry data (default is 1 MiB).
|
|
@@ -81,6 +100,17 @@ This package extends `zipfile` with `remove`-related functionalities.
|
|
|
81
100
|
|
|
82
101
|
Calling `copy` on a closed ZipFile will raise a `ValueError`.
|
|
83
102
|
|
|
103
|
+
> **Note:**
|
|
104
|
+
> Renaming a member in a ZIP file requires rewriting its data, as the
|
|
105
|
+
> filename is stored within its local file entry.
|
|
106
|
+
>
|
|
107
|
+
> To rename a member and reclaim the space occupied by the old entry,
|
|
108
|
+
> combine `copy`, `remove`, and `repack` like:
|
|
109
|
+
>
|
|
110
|
+
> ```python
|
|
111
|
+
> with ZipFile('spam.zip', 'a') as myzip:
|
|
112
|
+
> myzip.repack([myzip.remove(myzip.copy('old.txt', 'new.txt'))])
|
|
113
|
+
> ```
|
|
84
114
|
|
|
85
115
|
## Examples
|
|
86
116
|
|
|
@@ -136,10 +166,6 @@ print(os.path.getsize('archive.zip')) # 116 (would be 245 without `repack`)
|
|
|
136
166
|
|
|
137
167
|
### Move entries under a folder and reclaim space
|
|
138
168
|
|
|
139
|
-
Moving entries in a ZIP file must be done as a combination of `copy`, `remove`,
|
|
140
|
-
and optionally `repack`, because every local file entry contains the filename
|
|
141
|
-
and requires rewriting.
|
|
142
|
-
|
|
143
169
|
```python
|
|
144
170
|
import os
|
|
145
171
|
import zipremove as zipfile
|
|
@@ -188,3 +214,5 @@ with zipfile.ZipFile('archive.zip', 'a') as zh:
|
|
|
188
214
|
|
|
189
215
|
print(os.path.getsize('archive.zip')) # 446 (would be 599 without `repack`)
|
|
190
216
|
```
|
|
217
|
+
|
|
218
|
+
[PKZIP Application Note]: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
|
|
@@ -0,0 +1,118 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=77.0.3", "wheel"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "zipremove"
|
|
7
|
+
version = "0.8.1"
|
|
8
|
+
requires-python = "~=3.9"
|
|
9
|
+
description = "Extend `zipfile` with `remove`-related functionalities"
|
|
10
|
+
readme = "README.md"
|
|
11
|
+
license = "MIT"
|
|
12
|
+
license-files = ["LICENSE.*"]
|
|
13
|
+
authors = [
|
|
14
|
+
{name = "Danny Lin", email = "danny0838@gmail.com"},
|
|
15
|
+
]
|
|
16
|
+
classifiers = [
|
|
17
|
+
"Development Status :: 5 - Production/Stable",
|
|
18
|
+
"Intended Audience :: Developers",
|
|
19
|
+
"Topic :: System :: Archiving :: Compression",
|
|
20
|
+
"Programming Language :: Python :: 3",
|
|
21
|
+
"Programming Language :: Python :: 3 :: Only",
|
|
22
|
+
"Programming Language :: Python :: 3.9",
|
|
23
|
+
"Programming Language :: Python :: 3.10",
|
|
24
|
+
"Programming Language :: Python :: 3.11",
|
|
25
|
+
"Programming Language :: Python :: 3.12",
|
|
26
|
+
"Programming Language :: Python :: 3.13",
|
|
27
|
+
"Programming Language :: Python :: 3.14",
|
|
28
|
+
"Programming Language :: Python :: 3.15",
|
|
29
|
+
"Operating System :: OS Independent",
|
|
30
|
+
]
|
|
31
|
+
|
|
32
|
+
[project.urls]
|
|
33
|
+
Homepage = "https://github.com/danny0838/zipremove"
|
|
34
|
+
|
|
35
|
+
[dependency-groups]
|
|
36
|
+
lint = [
|
|
37
|
+
"flake8>=6.1",
|
|
38
|
+
"Flake8-pyproject>=1.2.3",
|
|
39
|
+
"flake8-comprehensions>=3.12",
|
|
40
|
+
"flake8-bugbear>=22.6.22",
|
|
41
|
+
"flake8-isort>=6.1",
|
|
42
|
+
"isort>=5.9.2",
|
|
43
|
+
]
|
|
44
|
+
build = [
|
|
45
|
+
"tox>=4.21.2",
|
|
46
|
+
]
|
|
47
|
+
dev = [
|
|
48
|
+
{include-group = "lint"},
|
|
49
|
+
{include-group = "build"},
|
|
50
|
+
]
|
|
51
|
+
|
|
52
|
+
[tool.flake8]
|
|
53
|
+
exclude = [
|
|
54
|
+
".git",
|
|
55
|
+
".venv",
|
|
56
|
+
".tox",
|
|
57
|
+
"__pycache__",
|
|
58
|
+
"build",
|
|
59
|
+
]
|
|
60
|
+
max-line-length = 130
|
|
61
|
+
|
|
62
|
+
# Flake8 Rules
|
|
63
|
+
# https://www.flake8rules.com/
|
|
64
|
+
ignore = [
|
|
65
|
+
"E122",
|
|
66
|
+
"E226",
|
|
67
|
+
"E302",
|
|
68
|
+
"E305",
|
|
69
|
+
"F403",
|
|
70
|
+
"F405",
|
|
71
|
+
"W504",
|
|
72
|
+
]
|
|
73
|
+
|
|
74
|
+
[tool.isort]
|
|
75
|
+
multi_line_output = 3
|
|
76
|
+
include_trailing_comma = true
|
|
77
|
+
|
|
78
|
+
# skip directories included by default
|
|
79
|
+
# ref: https://pycqa.github.io/isort/docs/configuration/options.html
|
|
80
|
+
# extend_skip = []
|
|
81
|
+
|
|
82
|
+
[tool.tox]
|
|
83
|
+
env_list = ["py314", "py313", "py312", "py311", "py310", "py39"]
|
|
84
|
+
min_version = "4.22"
|
|
85
|
+
|
|
86
|
+
[tool.tox.env_run_base]
|
|
87
|
+
description = "run unit tests"
|
|
88
|
+
package = "editable"
|
|
89
|
+
pass_env = ["TEST_RESOURCES"]
|
|
90
|
+
commands = [["python", "-m", "unittest", {replace = "posargs", extend = true}]]
|
|
91
|
+
|
|
92
|
+
[tool.tox.env.release]
|
|
93
|
+
description = "build and release to PyPI"
|
|
94
|
+
package = "skip"
|
|
95
|
+
deps = [
|
|
96
|
+
"build",
|
|
97
|
+
"twine>=4.0",
|
|
98
|
+
]
|
|
99
|
+
pass_env = [
|
|
100
|
+
"TWINE_USERNAME",
|
|
101
|
+
"TWINE_PASSWORD",
|
|
102
|
+
]
|
|
103
|
+
commands = [
|
|
104
|
+
["python", "-m", "build", "--sdist", "--wheel"],
|
|
105
|
+
["python", "-m", "twine", "upload", "--skip-existing", "dist/*"],
|
|
106
|
+
]
|
|
107
|
+
|
|
108
|
+
[tool.tox.env.lint]
|
|
109
|
+
description = "lint source code"
|
|
110
|
+
package = "skip"
|
|
111
|
+
dependency_groups = ["lint"]
|
|
112
|
+
commands = [["flake8", {replace = "posargs", default = ["."], extend = true}]]
|
|
113
|
+
|
|
114
|
+
[tool.tox.env.isort]
|
|
115
|
+
description = "sort imports for the source code"
|
|
116
|
+
package = "skip"
|
|
117
|
+
dependency_groups = ["lint"]
|
|
118
|
+
commands = [["isort", {replace = "posargs", default = ["."], extend = true}]]
|
|
@@ -69,9 +69,13 @@ except AttributeError:
|
|
|
69
69
|
LZMADecompressor.unused_data = unused_data
|
|
70
70
|
|
|
71
71
|
|
|
72
|
+
_REPACK_CHUNK_SIZE = 2**20
|
|
73
|
+
|
|
74
|
+
|
|
72
75
|
class _ZipRepacker:
|
|
73
76
|
"""Class for ZipFile repacking."""
|
|
74
|
-
def __init__(self, *, strict_descriptor=
|
|
77
|
+
def __init__(self, *, strict_descriptor=True,
|
|
78
|
+
chunk_size=_REPACK_CHUNK_SIZE, debug=0):
|
|
75
79
|
self.debug = debug # Level of printing: 0 through 3
|
|
76
80
|
self.chunk_size = chunk_size
|
|
77
81
|
self.strict_descriptor = strict_descriptor
|
|
@@ -122,7 +126,7 @@ class _ZipRepacker:
|
|
|
122
126
|
|
|
123
127
|
2. Data before the first referenced entry is stripped only when it
|
|
124
128
|
appears to be a sequence of consecutive entries with no extra
|
|
125
|
-
following bytes; extra
|
|
129
|
+
following bytes; extra preceding bytes are preserved.
|
|
126
130
|
|
|
127
131
|
3. Data between referenced entries is stripped only when it appears to
|
|
128
132
|
be a sequence of consecutive entries with no extra preceding bytes;
|
|
@@ -407,13 +411,20 @@ class _ZipRepacker:
|
|
|
407
411
|
|
|
408
412
|
zinfo.CRC, zinfo.compress_size, zinfo.file_size, dd_size = dd
|
|
409
413
|
|
|
410
|
-
|
|
414
|
+
entry_size = (
|
|
411
415
|
sizeFileHeader +
|
|
412
416
|
fheader[_FH_FILENAME_LENGTH] + fheader[_FH_EXTRA_FIELD_LENGTH] +
|
|
413
417
|
zinfo.compress_size +
|
|
414
418
|
dd_size
|
|
415
419
|
)
|
|
416
420
|
|
|
421
|
+
# Treat as a false positive if the entry would extend past end_offset,
|
|
422
|
+
# so callers never strip more bytes than the gap actually holds.
|
|
423
|
+
if offset + entry_size > end_offset:
|
|
424
|
+
return None
|
|
425
|
+
|
|
426
|
+
return entry_size
|
|
427
|
+
|
|
417
428
|
def _read_local_file_header(self, fp):
|
|
418
429
|
fheader = fp.read(sizeFileHeader)
|
|
419
430
|
if len(fheader) != sizeFileHeader:
|
|
@@ -446,7 +457,8 @@ class _ZipRepacker:
|
|
|
446
457
|
|
|
447
458
|
return None
|
|
448
459
|
|
|
449
|
-
def _scan_data_descriptor_no_sig(self, fp, offset, end_offset, zip64,
|
|
460
|
+
def _scan_data_descriptor_no_sig(self, fp, offset, end_offset, zip64,
|
|
461
|
+
chunk_size=io.DEFAULT_BUFFER_SIZE):
|
|
450
462
|
dd_fmt = '<LQQ' if zip64 else '<LLL'
|
|
451
463
|
dd_size = struct.calcsize(dd_fmt)
|
|
452
464
|
|
|
@@ -461,10 +473,7 @@ class _ZipRepacker:
|
|
|
461
473
|
mv = memoryview(chunk)
|
|
462
474
|
for i in range(len(chunk) - dd_size + 1):
|
|
463
475
|
dd = mv[i:i + dd_size]
|
|
464
|
-
|
|
465
|
-
crc, compress_size, file_size = struct.unpack(dd_fmt, dd)
|
|
466
|
-
except struct.error:
|
|
467
|
-
continue
|
|
476
|
+
crc, compress_size, file_size = struct.unpack(dd_fmt, dd)
|
|
468
477
|
if delta + i != compress_size:
|
|
469
478
|
continue
|
|
470
479
|
|
|
@@ -556,6 +565,11 @@ class _ZipRepacker:
|
|
|
556
565
|
while read_size < size:
|
|
557
566
|
fp.seek(old_offset + read_size)
|
|
558
567
|
data = fp.read(min(size - read_size, self.chunk_size))
|
|
568
|
+
if not data:
|
|
569
|
+
raise BadZipFile(
|
|
570
|
+
"Truncated data while repacking "
|
|
571
|
+
f"(expected {size} bytes at offset {old_offset})"
|
|
572
|
+
)
|
|
559
573
|
fp.seek(new_offset + read_size)
|
|
560
574
|
fp.write(data)
|
|
561
575
|
fp.flush()
|
|
@@ -637,11 +651,12 @@ class ZipFile(ZipFile):
|
|
|
637
651
|
|
|
638
652
|
return zinfo
|
|
639
653
|
|
|
640
|
-
def repack(self, removed=None,
|
|
654
|
+
def repack(self, removed=None, *, strict_descriptor=True,
|
|
655
|
+
chunk_size=_REPACK_CHUNK_SIZE):
|
|
641
656
|
"""Repack a zip file, removing non-referenced file entries.
|
|
642
657
|
|
|
643
658
|
The archive must be opened with mode 'a', as mode 'w'/'x' do not
|
|
644
|
-
truncate the file when closed. This cannot be
|
|
659
|
+
truncate the file when closed. This cannot be simply changed as
|
|
645
660
|
they may be used on an unseekable file buffer, which disallows
|
|
646
661
|
truncation."""
|
|
647
662
|
if self.mode != 'a':
|
|
@@ -657,6 +672,10 @@ class ZipFile(ZipFile):
|
|
|
657
672
|
with self._lock:
|
|
658
673
|
self._writing = True
|
|
659
674
|
try:
|
|
660
|
-
_ZipRepacker(
|
|
675
|
+
repacker = _ZipRepacker(
|
|
676
|
+
strict_descriptor=strict_descriptor,
|
|
677
|
+
chunk_size=chunk_size,
|
|
678
|
+
)
|
|
679
|
+
repacker.repack(self, removed)
|
|
661
680
|
finally:
|
|
662
681
|
self._writing = False
|
|
@@ -1,12 +1,11 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: zipremove
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.8.1
|
|
4
4
|
Summary: Extend `zipfile` with `remove`-related functionalities
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
Classifier: Development Status :: 4 - Beta
|
|
5
|
+
Author-email: Danny Lin <danny0838@gmail.com>
|
|
6
|
+
License-Expression: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/danny0838/zipremove
|
|
8
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
10
9
|
Classifier: Intended Audience :: Developers
|
|
11
10
|
Classifier: Topic :: System :: Archiving :: Compression
|
|
12
11
|
Classifier: Programming Language :: Python :: 3
|
|
@@ -17,6 +16,7 @@ Classifier: Programming Language :: Python :: 3.11
|
|
|
17
16
|
Classifier: Programming Language :: Python :: 3.12
|
|
18
17
|
Classifier: Programming Language :: Python :: 3.13
|
|
19
18
|
Classifier: Programming Language :: Python :: 3.14
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.15
|
|
20
20
|
Classifier: Operating System :: OS Independent
|
|
21
21
|
Requires-Python: ~=3.9
|
|
22
22
|
Description-Content-Type: text/markdown
|
|
@@ -38,8 +38,9 @@ This package extends `zipfile` with `remove`-related functionalities.
|
|
|
38
38
|
|
|
39
39
|
Removes a member entry from the archive's central directory.
|
|
40
40
|
*zinfo_or_arcname* may be the full path of the member or a `ZipInfo`
|
|
41
|
-
instance. If multiple members share the same
|
|
42
|
-
provided, only one
|
|
41
|
+
instance. If multiple members share the same path and a string is
|
|
42
|
+
provided, only one unspecified entry is removed; pass a specific
|
|
43
|
+
`ZipInfo` instance to guarantee which is removed.
|
|
43
44
|
|
|
44
45
|
The archive must be opened with mode ``'w'``, ``'x'`` or ``'a'``.
|
|
45
46
|
|
|
@@ -51,26 +52,44 @@ This package extends `zipfile` with `remove`-related functionalities.
|
|
|
51
52
|
> This method only removes the member's entry from the central directory,
|
|
52
53
|
> making it inaccessible to most tools. The member's local file entry,
|
|
53
54
|
> including content and metadata, remains in the archive and is still
|
|
54
|
-
> recoverable
|
|
55
|
-
>
|
|
55
|
+
> forensically recoverable. To completely delete the data and reclaim
|
|
56
|
+
> space, call `repack` afterwards (preferably passing the returned `ZipInfo`
|
|
57
|
+
> instance).
|
|
56
58
|
|
|
57
|
-
* `ZipFile.repack(removed=None, *, strict_descriptor=
|
|
59
|
+
* `ZipFile.repack(removed=None, *, strict_descriptor=True[, chunk_size])`
|
|
58
60
|
|
|
59
61
|
Rewrites the archive to remove unreferenced local file entries, shrinking
|
|
60
62
|
its file size. The archive must be opened with mode ``'a'``.
|
|
61
63
|
|
|
62
64
|
If *removed* is provided, it must be a sequence of `ZipInfo` objects
|
|
63
65
|
representing the recently removed members, and only their corresponding
|
|
64
|
-
local file entries will be removed.
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
66
|
+
local file entries will be removed. This is the most efficient and reliable
|
|
67
|
+
way to reclaim space. For example:
|
|
68
|
+
|
|
69
|
+
```python
|
|
70
|
+
with ZipFile('spam.zip', 'a') as myzip:
|
|
71
|
+
removed = [myzip.remove(name) for name in ('ham.txt', 'eggs.txt')]
|
|
72
|
+
myzip.repack(removed)
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
If *removed* is omitted, the archive is scanned to locate and remove local
|
|
76
|
+
file entries that are no longer referenced in the central directory.
|
|
77
|
+
|
|
78
|
+
When scanning, *strict_descriptor* controls how entries written with an
|
|
79
|
+
unsigned *data descriptor* are handled. A data descriptor is an optional
|
|
80
|
+
record holding an entry's CRC and sizes, stored just after the entry's
|
|
81
|
+
data; it is used when the archive is written to a non-seekable stream, and
|
|
82
|
+
is *signed* when it begins with a marker signature or *unsigned* otherwise.
|
|
83
|
+
Unsigned descriptors have been deprecated by the [PKZIP Application Note]
|
|
84
|
+
since version 6.3.0 (released in 2006) and are written only by some legacy
|
|
85
|
+
tools; signed descriptors—written by Python and other modern tools—are
|
|
86
|
+
always detected. When *strict_descriptor* is true (the default), unsigned
|
|
87
|
+
descriptors are not detected, and related unreferenced entries are not
|
|
88
|
+
removed. Setting `strict_descriptor=False` additionally detects unsigned
|
|
89
|
+
descriptors, at the cost of a significantly slower scan—around 100 to 1000
|
|
90
|
+
times in the worst case—which may be exploitable as a denial-of-service
|
|
91
|
+
vector on untrusted input. This does not affect entries without a data
|
|
92
|
+
descriptor, and is not needed when *removed* is provided.
|
|
74
93
|
|
|
75
94
|
*chunk_size* may be specified to control the buffer size when moving
|
|
76
95
|
entry data (default is 1 MiB).
|
|
@@ -106,6 +125,17 @@ This package extends `zipfile` with `remove`-related functionalities.
|
|
|
106
125
|
|
|
107
126
|
Calling `copy` on a closed ZipFile will raise a `ValueError`.
|
|
108
127
|
|
|
128
|
+
> **Note:**
|
|
129
|
+
> Renaming a member in a ZIP file requires rewriting its data, as the
|
|
130
|
+
> filename is stored within its local file entry.
|
|
131
|
+
>
|
|
132
|
+
> To rename a member and reclaim the space occupied by the old entry,
|
|
133
|
+
> combine `copy`, `remove`, and `repack` like:
|
|
134
|
+
>
|
|
135
|
+
> ```python
|
|
136
|
+
> with ZipFile('spam.zip', 'a') as myzip:
|
|
137
|
+
> myzip.repack([myzip.remove(myzip.copy('old.txt', 'new.txt'))])
|
|
138
|
+
> ```
|
|
109
139
|
|
|
110
140
|
## Examples
|
|
111
141
|
|
|
@@ -161,10 +191,6 @@ print(os.path.getsize('archive.zip')) # 116 (would be 245 without `repack`)
|
|
|
161
191
|
|
|
162
192
|
### Move entries under a folder and reclaim space
|
|
163
193
|
|
|
164
|
-
Moving entries in a ZIP file must be done as a combination of `copy`, `remove`,
|
|
165
|
-
and optionally `repack`, because every local file entry contains the filename
|
|
166
|
-
and requires rewriting.
|
|
167
|
-
|
|
168
194
|
```python
|
|
169
195
|
import os
|
|
170
196
|
import zipremove as zipfile
|
|
@@ -213,3 +239,5 @@ with zipfile.ZipFile('archive.zip', 'a') as zh:
|
|
|
213
239
|
|
|
214
240
|
print(os.path.getsize('archive.zip')) # 446 (would be 599 without `repack`)
|
|
215
241
|
```
|
|
242
|
+
|
|
243
|
+
[PKZIP Application Note]: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT
|