lz4ext 1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- lz4ext-1.0/.gitattributes +6 -0
- lz4ext-1.0/CHANGELOG.md +17 -0
- lz4ext-1.0/LICENSE +28 -0
- lz4ext-1.0/MANIFEST.in +8 -0
- lz4ext-1.0/PKG-INFO +130 -0
- lz4ext-1.0/README.rst +109 -0
- lz4ext-1.0/RELEASING.md +60 -0
- lz4ext-1.0/SECURITY.md +12 -0
- lz4ext-1.0/lz4ext.egg-info/PKG-INFO +130 -0
- lz4ext-1.0/lz4ext.egg-info/SOURCES.txt +23 -0
- lz4ext-1.0/lz4ext.egg-info/dependency_links.txt +1 -0
- lz4ext-1.0/lz4ext.egg-info/top_level.txt +1 -0
- lz4ext-1.0/pyproject.toml +32 -0
- lz4ext-1.0/setup.cfg +4 -0
- lz4ext-1.0/setup.py +9 -0
- lz4ext-1.0/src/LICENSE.lz4 +24 -0
- lz4ext-1.0/src/README.lz4.md +14 -0
- lz4ext-1.0/src/lz4.c +2829 -0
- lz4ext-1.0/src/lz4.h +884 -0
- lz4ext-1.0/src/lz4hc.c +2192 -0
- lz4ext-1.0/src/lz4hc.h +414 -0
- lz4ext-1.0/src/python-lz4.c +346 -0
- lz4ext-1.0/src/python-lz4.h +41 -0
- lz4ext-1.0/tests/bench.py +28 -0
- lz4ext-1.0/tests/test.py +192 -0
lz4ext-1.0/CHANGELOG.md
ADDED
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 1.0
|
|
4
|
+
|
|
5
|
+
- Update vendored LZ4 r119 to official v1.10.0.
|
|
6
|
+
- Require CPython 3.11 or newer and contiguous binary input; reject text.
|
|
7
|
+
- Check compression bounds and pass explicit output capacities.
|
|
8
|
+
- Validate prefixed decoded lengths and empty framing; emit canonical empty blocks.
|
|
9
|
+
- Return cleanly from raw decoder errors and check output capacity arithmetic.
|
|
10
|
+
- Default decoder allocation limit to 64 MiB, with keyword-only `max_output_size`
|
|
11
|
+
overrides on every decoder alias.
|
|
12
|
+
- Preserve the four-byte little-endian prefixed format and existing aliases.
|
|
13
|
+
- Use PEP 517/621 packaging and include binding and vendored LZ4 licenses.
|
|
14
|
+
- Expose package/library versions and default output limit.
|
|
15
|
+
|
|
16
|
+
Historical malformed 20-byte empty records require trusted offline migration;
|
|
17
|
+
see the README. Valid nonempty legacy blocks remain readable.
|
lz4ext-1.0/LICENSE
ADDED
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
Copyright (c) 2012-2013, Steeve Morin
|
|
2
|
+
All rights reserved.
|
|
3
|
+
|
|
4
|
+
Redistribution and use in source and binary forms, with or without
|
|
5
|
+
modification, are permitted provided that the following conditions are met:
|
|
6
|
+
|
|
7
|
+
1. Redistributions of source code must retain the above copyright notice,
|
|
8
|
+
this list of conditions and the following disclaimer.
|
|
9
|
+
|
|
10
|
+
2. Redistributions in binary form must reproduce the above copyright notice,
|
|
11
|
+
this list of conditions and the following disclaimer in the documentation
|
|
12
|
+
and/or other materials provided with the distribution.
|
|
13
|
+
|
|
14
|
+
3. Neither the name of Steeve Morin nor the names of its contributors may be
|
|
15
|
+
used to endorse or promote products derived from this software without
|
|
16
|
+
specific prior written permission.
|
|
17
|
+
|
|
18
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
|
|
19
|
+
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
|
20
|
+
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
|
|
21
|
+
ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE
|
|
22
|
+
LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
|
|
23
|
+
CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
|
|
24
|
+
SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
|
|
25
|
+
INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
|
|
26
|
+
CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
|
|
27
|
+
ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
|
|
28
|
+
POSSIBILITY OF SUCH DAMAGE.
|
lz4ext-1.0/MANIFEST.in
ADDED
lz4ext-1.0/PKG-INFO
ADDED
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: lz4ext
|
|
3
|
+
Version: 1.0
|
|
4
|
+
Summary: Native LZ4 block compression bindings for CPython
|
|
5
|
+
Author-email: Steeve Morin <steeve.morin@gmail.com>, Anton Stuk <sigman@ioupg.com>
|
|
6
|
+
License-Expression: BSD-3-Clause AND BSD-2-Clause
|
|
7
|
+
Project-URL: Homepage, https://github.com/sigman78/python-lz4
|
|
8
|
+
Project-URL: Issues, https://github.com/sigman78/python-lz4/issues
|
|
9
|
+
Classifier: Programming Language :: C
|
|
10
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
11
|
+
Classifier: Programming Language :: Python :: Implementation :: CPython
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
16
|
+
Requires-Python: >=3.11
|
|
17
|
+
Description-Content-Type: text/x-rst
|
|
18
|
+
License-File: LICENSE
|
|
19
|
+
License-File: src/LICENSE.lz4
|
|
20
|
+
Dynamic: license-file
|
|
21
|
+
|
|
22
|
+
=============
|
|
23
|
+
python-lz4ext
|
|
24
|
+
=============
|
|
25
|
+
|
|
26
|
+
``lz4ext`` provides native LZ4 block compression for CPython 3.11–3.14,
|
|
27
|
+
using vendored `LZ4 v1.10.0 <https://github.com/lz4/lz4/releases/tag/v1.10.0>`_.
|
|
28
|
+
The distribution and import name are both ``lz4ext``.
|
|
29
|
+
|
|
30
|
+
GitHub Actions tests/builds the supported Python matrix. See ``RELEASING.md``
|
|
31
|
+
for release artifacts and the separate PyPI Trusted Publishing configuration,
|
|
32
|
+
and ``SECURITY.md`` for vulnerability reporting.
|
|
33
|
+
|
|
34
|
+
Install and develop
|
|
35
|
+
===================
|
|
36
|
+
|
|
37
|
+
Install the distribution::
|
|
38
|
+
|
|
39
|
+
python -m pip install lz4ext
|
|
40
|
+
|
|
41
|
+
Build from this checkout (requires a C compiler, such as MSVC on Windows)::
|
|
42
|
+
|
|
43
|
+
python -m pip install .
|
|
44
|
+
python tests/test.py
|
|
45
|
+
python tests/bench.py --loops 1000
|
|
46
|
+
|
|
47
|
+
Build distributable artifacts with an isolated PEP 517 backend::
|
|
48
|
+
|
|
49
|
+
python -m pip install build
|
|
50
|
+
python -m build
|
|
51
|
+
|
|
52
|
+
The extension uses the CPython C API, retains the GIL, and does not declare
|
|
53
|
+
free-threaded Python support. Builds use compiler defaults without
|
|
54
|
+
``-march=native`` flags. No runtime Python dependencies are required.
|
|
55
|
+
|
|
56
|
+
API
|
|
57
|
+
===
|
|
58
|
+
|
|
59
|
+
Input must be contiguous bytes-like data, including ``bytes``, ``bytearray``,
|
|
60
|
+
and contiguous ``memoryview``. Text is rejected; encode it explicitly.
|
|
61
|
+
All output is ``bytes``::
|
|
62
|
+
|
|
63
|
+
import lz4ext
|
|
64
|
+
data = b"hello" * 100
|
|
65
|
+
block = lz4ext.compress(data)
|
|
66
|
+
assert lz4ext.decompress(block) == data
|
|
67
|
+
assert lz4ext.loads(lz4ext.compressHC(data)) == data
|
|
68
|
+
|
|
69
|
+
``compress(data)`` and ``compressHC(data)`` return a four-byte little-endian
|
|
70
|
+
unsigned original size followed by an LZ4 block. This is the historical
|
|
71
|
+
python-lz4ext format, not the LZ4 frame format.
|
|
72
|
+
|
|
73
|
+
``decompress(data, *, max_output_size=67108864)`` validates framing and requires
|
|
74
|
+
an exact match between the decoded length and header. ``max_output_size`` must
|
|
75
|
+
be positive and at most ``INT_MAX`` (2,147,483,647). Declared output above the
|
|
76
|
+
limit is rejected before allocation. To deliberately permit larger output,
|
|
77
|
+
pass an explicit limit appropriate to the data and available memory.
|
|
78
|
+
|
|
79
|
+
``compress_raw(data)`` returns a headerless block.
|
|
80
|
+
``decompress_raw(data, output_size=0, *, max_output_size=67108864)`` allocates
|
|
81
|
+
the supplied capacity, or twice the compressed input length when capacity is
|
|
82
|
+
zero or omitted. Capacity must fit the output limit. High-ratio blocks usually
|
|
83
|
+
need an explicit capacity::
|
|
84
|
+
|
|
85
|
+
raw = lz4ext.compress_raw(data)
|
|
86
|
+
assert lz4ext.decompress_raw(raw, len(data)) == data
|
|
87
|
+
|
|
88
|
+
Malformed raw input and insufficient capacity both raise ``ValueError``;
|
|
89
|
+
the decoder cannot distinguish them. There are no automatic allocation retries.
|
|
90
|
+
Empty raw output is ``b'\x00'``; zero-length raw input is invalid.
|
|
91
|
+
|
|
92
|
+
Compression rejects input above LZ4's supported maximum (2,113,929,216 bytes).
|
|
93
|
+
Limits do not guarantee allocation success; ``MemoryError`` may still occur.
|
|
94
|
+
Output caps bound the allocation for a call, not total process memory or
|
|
95
|
+
cumulative work. This block format provides no checksum or authenticity check.
|
|
96
|
+
|
|
97
|
+
Aliases and constants
|
|
98
|
+
=====================
|
|
99
|
+
|
|
100
|
+
``dumps`` and ``LZ4_compress`` behave like ``compress``.
|
|
101
|
+
``loads``, ``uncompress``, and ``LZ4_uncompress`` behave like ``decompress``.
|
|
102
|
+
``uncompress_raw`` behaves like ``decompress_raw``. All decoder aliases accept
|
|
103
|
+
the keyword-only output limit.
|
|
104
|
+
|
|
105
|
+
``VERSION`` and ``__version__`` contain the package version. ``LZ4_VERSION``
|
|
106
|
+
reports the linked vendored library version. ``DEFAULT_MAX_OUTPUT_SIZE`` is
|
|
107
|
+
64 MiB (67,108,864 bytes).
|
|
108
|
+
|
|
109
|
+
Compatibility and migration
|
|
110
|
+
===========================
|
|
111
|
+
|
|
112
|
+
Version 1.0 requires Python 3.11 or newer and rejects implicit text input.
|
|
113
|
+
Valid r119-era nonempty prefixed and raw blocks remain readable; compression
|
|
114
|
+
may produce different bytes after the library upgrade. No frame API is added.
|
|
115
|
+
Output above 64 MiB needs an explicit limit override. Incorrect size headers
|
|
116
|
+
and detected malformed blocks now raise exceptions.
|
|
117
|
+
|
|
118
|
+
Empty prefixed output is exactly five zero bytes (zero header plus canonical
|
|
119
|
+
``00`` block). The exact four-byte zero-header legacy sentinel is accepted.
|
|
120
|
+
Old releases could emit 20-byte empty records: a zero header followed by 16
|
|
121
|
+
uninitialized bytes. These malformed records are rejected. If trusted records
|
|
122
|
+
are known independently to represent empty data, migrate them offline to the
|
|
123
|
+
canonical five-byte encoding. Arbitrary zero-header padding is not valid data.
|
|
124
|
+
|
|
125
|
+
Licenses
|
|
126
|
+
========
|
|
127
|
+
|
|
128
|
+
The binding uses BSD 3-Clause in ``LICENSE``. The unmodified vendored LZ4
|
|
129
|
+
sources use BSD 2-Clause; see ``src/LICENSE.lz4`` and ``src/README.lz4.md`` for
|
|
130
|
+
upstream provenance. Both licenses accompany source and wheel distributions.
|
lz4ext-1.0/README.rst
ADDED
|
@@ -0,0 +1,109 @@
|
|
|
1
|
+
=============
|
|
2
|
+
python-lz4ext
|
|
3
|
+
=============
|
|
4
|
+
|
|
5
|
+
``lz4ext`` provides native LZ4 block compression for CPython 3.11–3.14,
|
|
6
|
+
using vendored `LZ4 v1.10.0 <https://github.com/lz4/lz4/releases/tag/v1.10.0>`_.
|
|
7
|
+
The distribution and import name are both ``lz4ext``.
|
|
8
|
+
|
|
9
|
+
GitHub Actions tests/builds the supported Python matrix. See ``RELEASING.md``
|
|
10
|
+
for release artifacts and the separate PyPI Trusted Publishing configuration,
|
|
11
|
+
and ``SECURITY.md`` for vulnerability reporting.
|
|
12
|
+
|
|
13
|
+
Install and develop
|
|
14
|
+
===================
|
|
15
|
+
|
|
16
|
+
Install the distribution::
|
|
17
|
+
|
|
18
|
+
python -m pip install lz4ext
|
|
19
|
+
|
|
20
|
+
Build from this checkout (requires a C compiler, such as MSVC on Windows)::
|
|
21
|
+
|
|
22
|
+
python -m pip install .
|
|
23
|
+
python tests/test.py
|
|
24
|
+
python tests/bench.py --loops 1000
|
|
25
|
+
|
|
26
|
+
Build distributable artifacts with an isolated PEP 517 backend::
|
|
27
|
+
|
|
28
|
+
python -m pip install build
|
|
29
|
+
python -m build
|
|
30
|
+
|
|
31
|
+
The extension uses the CPython C API, retains the GIL, and does not declare
|
|
32
|
+
free-threaded Python support. Builds use compiler defaults without
|
|
33
|
+
``-march=native`` flags. No runtime Python dependencies are required.
|
|
34
|
+
|
|
35
|
+
API
|
|
36
|
+
===
|
|
37
|
+
|
|
38
|
+
Input must be contiguous bytes-like data, including ``bytes``, ``bytearray``,
|
|
39
|
+
and contiguous ``memoryview``. Text is rejected; encode it explicitly.
|
|
40
|
+
All output is ``bytes``::
|
|
41
|
+
|
|
42
|
+
import lz4ext
|
|
43
|
+
data = b"hello" * 100
|
|
44
|
+
block = lz4ext.compress(data)
|
|
45
|
+
assert lz4ext.decompress(block) == data
|
|
46
|
+
assert lz4ext.loads(lz4ext.compressHC(data)) == data
|
|
47
|
+
|
|
48
|
+
``compress(data)`` and ``compressHC(data)`` return a four-byte little-endian
|
|
49
|
+
unsigned original size followed by an LZ4 block. This is the historical
|
|
50
|
+
python-lz4ext format, not the LZ4 frame format.
|
|
51
|
+
|
|
52
|
+
``decompress(data, *, max_output_size=67108864)`` validates framing and requires
|
|
53
|
+
an exact match between the decoded length and header. ``max_output_size`` must
|
|
54
|
+
be positive and at most ``INT_MAX`` (2,147,483,647). Declared output above the
|
|
55
|
+
limit is rejected before allocation. To deliberately permit larger output,
|
|
56
|
+
pass an explicit limit appropriate to the data and available memory.
|
|
57
|
+
|
|
58
|
+
``compress_raw(data)`` returns a headerless block.
|
|
59
|
+
``decompress_raw(data, output_size=0, *, max_output_size=67108864)`` allocates
|
|
60
|
+
the supplied capacity, or twice the compressed input length when capacity is
|
|
61
|
+
zero or omitted. Capacity must fit the output limit. High-ratio blocks usually
|
|
62
|
+
need an explicit capacity::
|
|
63
|
+
|
|
64
|
+
raw = lz4ext.compress_raw(data)
|
|
65
|
+
assert lz4ext.decompress_raw(raw, len(data)) == data
|
|
66
|
+
|
|
67
|
+
Malformed raw input and insufficient capacity both raise ``ValueError``;
|
|
68
|
+
the decoder cannot distinguish them. There are no automatic allocation retries.
|
|
69
|
+
Empty raw output is ``b'\x00'``; zero-length raw input is invalid.
|
|
70
|
+
|
|
71
|
+
Compression rejects input above LZ4's supported maximum (2,113,929,216 bytes).
|
|
72
|
+
Limits do not guarantee allocation success; ``MemoryError`` may still occur.
|
|
73
|
+
Output caps bound the allocation for a call, not total process memory or
|
|
74
|
+
cumulative work. This block format provides no checksum or authenticity check.
|
|
75
|
+
|
|
76
|
+
Aliases and constants
|
|
77
|
+
=====================
|
|
78
|
+
|
|
79
|
+
``dumps`` and ``LZ4_compress`` behave like ``compress``.
|
|
80
|
+
``loads``, ``uncompress``, and ``LZ4_uncompress`` behave like ``decompress``.
|
|
81
|
+
``uncompress_raw`` behaves like ``decompress_raw``. All decoder aliases accept
|
|
82
|
+
the keyword-only output limit.
|
|
83
|
+
|
|
84
|
+
``VERSION`` and ``__version__`` contain the package version. ``LZ4_VERSION``
|
|
85
|
+
reports the linked vendored library version. ``DEFAULT_MAX_OUTPUT_SIZE`` is
|
|
86
|
+
64 MiB (67,108,864 bytes).
|
|
87
|
+
|
|
88
|
+
Compatibility and migration
|
|
89
|
+
===========================
|
|
90
|
+
|
|
91
|
+
Version 1.0 requires Python 3.11 or newer and rejects implicit text input.
|
|
92
|
+
Valid r119-era nonempty prefixed and raw blocks remain readable; compression
|
|
93
|
+
may produce different bytes after the library upgrade. No frame API is added.
|
|
94
|
+
Output above 64 MiB needs an explicit limit override. Incorrect size headers
|
|
95
|
+
and detected malformed blocks now raise exceptions.
|
|
96
|
+
|
|
97
|
+
Empty prefixed output is exactly five zero bytes (zero header plus canonical
|
|
98
|
+
``00`` block). The exact four-byte zero-header legacy sentinel is accepted.
|
|
99
|
+
Old releases could emit 20-byte empty records: a zero header followed by 16
|
|
100
|
+
uninitialized bytes. These malformed records are rejected. If trusted records
|
|
101
|
+
are known independently to represent empty data, migrate them offline to the
|
|
102
|
+
canonical five-byte encoding. Arbitrary zero-header padding is not valid data.
|
|
103
|
+
|
|
104
|
+
Licenses
|
|
105
|
+
========
|
|
106
|
+
|
|
107
|
+
The binding uses BSD 3-Clause in ``LICENSE``. The unmodified vendored LZ4
|
|
108
|
+
sources use BSD 2-Clause; see ``src/LICENSE.lz4`` and ``src/README.lz4.md`` for
|
|
109
|
+
upstream provenance. Both licenses accompany source and wheel distributions.
|
lz4ext-1.0/RELEASING.md
ADDED
|
@@ -0,0 +1,60 @@
|
|
|
1
|
+
# Release configuration and procedure
|
|
2
|
+
|
|
3
|
+
The workflow is `.github/workflows/ci.yml`. Pushes, pull requests, and manual
|
|
4
|
+
runs test/build only. A **published GitHub release** starts a fresh build and
|
|
5
|
+
can publish its tested artifacts to PyPI. Creating a tag or pushing a branch
|
|
6
|
+
alone does not publish. The tag must exactly equal `v` plus the project version.
|
|
7
|
+
|
|
8
|
+
The matrix tests CPython 3.11–3.14 on Linux, Windows, and macOS with
|
|
9
|
+
`PYTHONMALLOC=debug`. cibuildwheel tests installed wheels on Linux x86_64
|
|
10
|
+
(manylinux and musllinux), Windows AMD64, and macOS x86_64/arm64. It selects
|
|
11
|
+
normal CPython builds, excluding free-threaded builds. The source distribution
|
|
12
|
+
is installed and tested in a clean environment. Distribution artifacts from
|
|
13
|
+
each workflow run are available for review; remote checks must actually run
|
|
14
|
+
before platform compatibility is considered verified.
|
|
15
|
+
|
|
16
|
+
## One-time external setup
|
|
17
|
+
|
|
18
|
+
These settings must be configured by the repository/PyPI owner; adding this
|
|
19
|
+
workflow does not create them or publish anything.
|
|
20
|
+
|
|
21
|
+
1. Enable GitHub Actions in `sigman78/python-lz4` and create the GitHub
|
|
22
|
+
environment named exactly `pypi` under repository Settings → Environments.
|
|
23
|
+
Restrict deployment tags to release tags such as `v*`. Configure required
|
|
24
|
+
reviewers if the maintainers want an additional release approval gate.
|
|
25
|
+
2. In the `lz4ext` project on PyPI, open Manage → Publishing and add a GitHub
|
|
26
|
+
Trusted Publisher with these exact values:
|
|
27
|
+
- Owner: `sigman78`
|
|
28
|
+
- Repository: `python-lz4`
|
|
29
|
+
- Workflow filename: `ci.yml` (filename only)
|
|
30
|
+
- Environment: `pypi`
|
|
31
|
+
3. If the PyPI project does not exist or is not owned by the maintainer, resolve
|
|
32
|
+
ownership first or register a pending publisher using these same fields.
|
|
33
|
+
Do not substitute the unrelated `lz4` project. Forks must configure their
|
|
34
|
+
own owner/repository and PyPI project before enabling publication.
|
|
35
|
+
|
|
36
|
+
See the [PyPI Trusted Publisher instructions](https://docs.pypi.org/trusted-publishers/adding-a-publisher/).
|
|
37
|
+
No PyPI password or API token is needed. OIDC `id-token: write` is granted only
|
|
38
|
+
to the isolated publish job, which does not check out or execute this source.
|
|
39
|
+
It downloads only this run's `dist-*` artifacts after every required job passes.
|
|
40
|
+
Build jobs have read-only repository permissions; pull requests do not publish.
|
|
41
|
+
|
|
42
|
+
## Release
|
|
43
|
+
|
|
44
|
+
1. Update `project.version` in `pyproject.toml`, the direct-build fallback in
|
|
45
|
+
`src/python-lz4.c`, the version test, and the changelog.
|
|
46
|
+
2. Run `python -m build`, install the built wheel in a clean environment, run
|
|
47
|
+
`python tests/test.py`, and run `python -m twine check dist/*` (install `build`
|
|
48
|
+
and `twine` first). On PowerShell, enumerate artifacts for pip rather than
|
|
49
|
+
assuming shell expansion of `*.whl`.
|
|
50
|
+
3. Push the reviewed commit and confirm the GitHub Actions test/build run passes.
|
|
51
|
+
4. Create the matching tag, for example `v1.0` for version `1.0`, and publish
|
|
52
|
+
its GitHub release. Publishing the release is the action that enables PyPI
|
|
53
|
+
upload. The workflow rebuilds and tests artifacts for that tagged commit.
|
|
54
|
+
5. Inspect the release run and PyPI files. Failed publishing should be diagnosed
|
|
55
|
+
before rerunning; existing files on PyPI cannot be replaced. The workflow
|
|
56
|
+
does not silently skip existing artifacts.
|
|
57
|
+
|
|
58
|
+
Actions are pinned to verified upstream commit IDs with version comments.
|
|
59
|
+
Review new action versions and update pins deliberately. No remote publication,
|
|
60
|
+
tag creation, or external-account configuration is performed by local builds.
|
lz4ext-1.0/SECURITY.md
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
# Reporting a vulnerability
|
|
2
|
+
|
|
3
|
+
Avoid putting sensitive exploit details in a public issue. Check the repository's
|
|
4
|
+
Security tab for a private vulnerability reporting option. Its availability
|
|
5
|
+
depends on the repository owner's configuration; this file does not enable it.
|
|
6
|
+
If unavailable, contact the maintainers privately using the contact details in
|
|
7
|
+
`pyproject.toml` to arrange disclosure.
|
|
8
|
+
|
|
9
|
+
Use the current release. Historical versions before this modernization contain
|
|
10
|
+
known native-wrapper memory handling defects. The README describes decoder
|
|
11
|
+
allocation limits and format compatibility. This block format does not provide
|
|
12
|
+
integrity or authenticity; use application-level verification when needed.
|
|
@@ -0,0 +1,130 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: lz4ext
|
|
3
|
+
Version: 1.0
|
|
4
|
+
Summary: Native LZ4 block compression bindings for CPython
|
|
5
|
+
Author-email: Steeve Morin <steeve.morin@gmail.com>, Anton Stuk <sigman@ioupg.com>
|
|
6
|
+
License-Expression: BSD-3-Clause AND BSD-2-Clause
|
|
7
|
+
Project-URL: Homepage, https://github.com/sigman78/python-lz4
|
|
8
|
+
Project-URL: Issues, https://github.com/sigman78/python-lz4/issues
|
|
9
|
+
Classifier: Programming Language :: C
|
|
10
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
11
|
+
Classifier: Programming Language :: Python :: Implementation :: CPython
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
16
|
+
Requires-Python: >=3.11
|
|
17
|
+
Description-Content-Type: text/x-rst
|
|
18
|
+
License-File: LICENSE
|
|
19
|
+
License-File: src/LICENSE.lz4
|
|
20
|
+
Dynamic: license-file
|
|
21
|
+
|
|
22
|
+
=============
|
|
23
|
+
python-lz4ext
|
|
24
|
+
=============
|
|
25
|
+
|
|
26
|
+
``lz4ext`` provides native LZ4 block compression for CPython 3.11–3.14,
|
|
27
|
+
using vendored `LZ4 v1.10.0 <https://github.com/lz4/lz4/releases/tag/v1.10.0>`_.
|
|
28
|
+
The distribution and import name are both ``lz4ext``.
|
|
29
|
+
|
|
30
|
+
GitHub Actions tests/builds the supported Python matrix. See ``RELEASING.md``
|
|
31
|
+
for release artifacts and the separate PyPI Trusted Publishing configuration,
|
|
32
|
+
and ``SECURITY.md`` for vulnerability reporting.
|
|
33
|
+
|
|
34
|
+
Install and develop
|
|
35
|
+
===================
|
|
36
|
+
|
|
37
|
+
Install the distribution::
|
|
38
|
+
|
|
39
|
+
python -m pip install lz4ext
|
|
40
|
+
|
|
41
|
+
Build from this checkout (requires a C compiler, such as MSVC on Windows)::
|
|
42
|
+
|
|
43
|
+
python -m pip install .
|
|
44
|
+
python tests/test.py
|
|
45
|
+
python tests/bench.py --loops 1000
|
|
46
|
+
|
|
47
|
+
Build distributable artifacts with an isolated PEP 517 backend::
|
|
48
|
+
|
|
49
|
+
python -m pip install build
|
|
50
|
+
python -m build
|
|
51
|
+
|
|
52
|
+
The extension uses the CPython C API, retains the GIL, and does not declare
|
|
53
|
+
free-threaded Python support. Builds use compiler defaults without
|
|
54
|
+
``-march=native`` flags. No runtime Python dependencies are required.
|
|
55
|
+
|
|
56
|
+
API
|
|
57
|
+
===
|
|
58
|
+
|
|
59
|
+
Input must be contiguous bytes-like data, including ``bytes``, ``bytearray``,
|
|
60
|
+
and contiguous ``memoryview``. Text is rejected; encode it explicitly.
|
|
61
|
+
All output is ``bytes``::
|
|
62
|
+
|
|
63
|
+
import lz4ext
|
|
64
|
+
data = b"hello" * 100
|
|
65
|
+
block = lz4ext.compress(data)
|
|
66
|
+
assert lz4ext.decompress(block) == data
|
|
67
|
+
assert lz4ext.loads(lz4ext.compressHC(data)) == data
|
|
68
|
+
|
|
69
|
+
``compress(data)`` and ``compressHC(data)`` return a four-byte little-endian
|
|
70
|
+
unsigned original size followed by an LZ4 block. This is the historical
|
|
71
|
+
python-lz4ext format, not the LZ4 frame format.
|
|
72
|
+
|
|
73
|
+
``decompress(data, *, max_output_size=67108864)`` validates framing and requires
|
|
74
|
+
an exact match between the decoded length and header. ``max_output_size`` must
|
|
75
|
+
be positive and at most ``INT_MAX`` (2,147,483,647). Declared output above the
|
|
76
|
+
limit is rejected before allocation. To deliberately permit larger output,
|
|
77
|
+
pass an explicit limit appropriate to the data and available memory.
|
|
78
|
+
|
|
79
|
+
``compress_raw(data)`` returns a headerless block.
|
|
80
|
+
``decompress_raw(data, output_size=0, *, max_output_size=67108864)`` allocates
|
|
81
|
+
the supplied capacity, or twice the compressed input length when capacity is
|
|
82
|
+
zero or omitted. Capacity must fit the output limit. High-ratio blocks usually
|
|
83
|
+
need an explicit capacity::
|
|
84
|
+
|
|
85
|
+
raw = lz4ext.compress_raw(data)
|
|
86
|
+
assert lz4ext.decompress_raw(raw, len(data)) == data
|
|
87
|
+
|
|
88
|
+
Malformed raw input and insufficient capacity both raise ``ValueError``;
|
|
89
|
+
the decoder cannot distinguish them. There are no automatic allocation retries.
|
|
90
|
+
Empty raw output is ``b'\x00'``; zero-length raw input is invalid.
|
|
91
|
+
|
|
92
|
+
Compression rejects input above LZ4's supported maximum (2,113,929,216 bytes).
|
|
93
|
+
Limits do not guarantee allocation success; ``MemoryError`` may still occur.
|
|
94
|
+
Output caps bound the allocation for a call, not total process memory or
|
|
95
|
+
cumulative work. This block format provides no checksum or authenticity check.
|
|
96
|
+
|
|
97
|
+
Aliases and constants
|
|
98
|
+
=====================
|
|
99
|
+
|
|
100
|
+
``dumps`` and ``LZ4_compress`` behave like ``compress``.
|
|
101
|
+
``loads``, ``uncompress``, and ``LZ4_uncompress`` behave like ``decompress``.
|
|
102
|
+
``uncompress_raw`` behaves like ``decompress_raw``. All decoder aliases accept
|
|
103
|
+
the keyword-only output limit.
|
|
104
|
+
|
|
105
|
+
``VERSION`` and ``__version__`` contain the package version. ``LZ4_VERSION``
|
|
106
|
+
reports the linked vendored library version. ``DEFAULT_MAX_OUTPUT_SIZE`` is
|
|
107
|
+
64 MiB (67,108,864 bytes).
|
|
108
|
+
|
|
109
|
+
Compatibility and migration
|
|
110
|
+
===========================
|
|
111
|
+
|
|
112
|
+
Version 1.0 requires Python 3.11 or newer and rejects implicit text input.
|
|
113
|
+
Valid r119-era nonempty prefixed and raw blocks remain readable; compression
|
|
114
|
+
may produce different bytes after the library upgrade. No frame API is added.
|
|
115
|
+
Output above 64 MiB needs an explicit limit override. Incorrect size headers
|
|
116
|
+
and detected malformed blocks now raise exceptions.
|
|
117
|
+
|
|
118
|
+
Empty prefixed output is exactly five zero bytes (zero header plus canonical
|
|
119
|
+
``00`` block). The exact four-byte zero-header legacy sentinel is accepted.
|
|
120
|
+
Old releases could emit 20-byte empty records: a zero header followed by 16
|
|
121
|
+
uninitialized bytes. These malformed records are rejected. If trusted records
|
|
122
|
+
are known independently to represent empty data, migrate them offline to the
|
|
123
|
+
canonical five-byte encoding. Arbitrary zero-header padding is not valid data.
|
|
124
|
+
|
|
125
|
+
Licenses
|
|
126
|
+
========
|
|
127
|
+
|
|
128
|
+
The binding uses BSD 3-Clause in ``LICENSE``. The unmodified vendored LZ4
|
|
129
|
+
sources use BSD 2-Clause; see ``src/LICENSE.lz4`` and ``src/README.lz4.md`` for
|
|
130
|
+
upstream provenance. Both licenses accompany source and wheel distributions.
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
.gitattributes
|
|
2
|
+
CHANGELOG.md
|
|
3
|
+
LICENSE
|
|
4
|
+
MANIFEST.in
|
|
5
|
+
README.rst
|
|
6
|
+
RELEASING.md
|
|
7
|
+
SECURITY.md
|
|
8
|
+
pyproject.toml
|
|
9
|
+
setup.py
|
|
10
|
+
lz4ext.egg-info/PKG-INFO
|
|
11
|
+
lz4ext.egg-info/SOURCES.txt
|
|
12
|
+
lz4ext.egg-info/dependency_links.txt
|
|
13
|
+
lz4ext.egg-info/top_level.txt
|
|
14
|
+
src/LICENSE.lz4
|
|
15
|
+
src/README.lz4.md
|
|
16
|
+
src/lz4.c
|
|
17
|
+
src/lz4.h
|
|
18
|
+
src/lz4hc.c
|
|
19
|
+
src/lz4hc.h
|
|
20
|
+
src/python-lz4.c
|
|
21
|
+
src/python-lz4.h
|
|
22
|
+
tests/bench.py
|
|
23
|
+
tests/test.py
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
lz4ext
|
|
@@ -0,0 +1,32 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=77.0.3"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "lz4ext"
|
|
7
|
+
version = "1.0"
|
|
8
|
+
description = "Native LZ4 block compression bindings for CPython"
|
|
9
|
+
readme = "README.rst"
|
|
10
|
+
requires-python = ">=3.11"
|
|
11
|
+
license = "BSD-3-Clause AND BSD-2-Clause"
|
|
12
|
+
license-files = ["LICENSE", "src/LICENSE.lz4"]
|
|
13
|
+
authors = [
|
|
14
|
+
{name = "Steeve Morin", email = "steeve.morin@gmail.com"},
|
|
15
|
+
{name = "Anton Stuk", email = "sigman@ioupg.com"},
|
|
16
|
+
]
|
|
17
|
+
classifiers = [
|
|
18
|
+
"Programming Language :: C",
|
|
19
|
+
"Programming Language :: Python :: 3 :: Only",
|
|
20
|
+
"Programming Language :: Python :: Implementation :: CPython",
|
|
21
|
+
"Programming Language :: Python :: 3.11",
|
|
22
|
+
"Programming Language :: Python :: 3.12",
|
|
23
|
+
"Programming Language :: Python :: 3.13",
|
|
24
|
+
"Programming Language :: Python :: 3.14",
|
|
25
|
+
]
|
|
26
|
+
|
|
27
|
+
[project.urls]
|
|
28
|
+
Homepage = "https://github.com/sigman78/python-lz4"
|
|
29
|
+
Issues = "https://github.com/sigman78/python-lz4/issues"
|
|
30
|
+
|
|
31
|
+
[tool.setuptools]
|
|
32
|
+
packages = []
|
lz4ext-1.0/setup.cfg
ADDED
lz4ext-1.0/setup.py
ADDED
|
@@ -0,0 +1,9 @@
|
|
|
1
|
+
from pathlib import Path
|
|
2
|
+
import tomllib
|
|
3
|
+
from setuptools import Extension, setup
|
|
4
|
+
|
|
5
|
+
metadata = tomllib.loads(Path(__file__).with_name("pyproject.toml").read_text(encoding="utf-8"))
|
|
6
|
+
setup(ext_modules=[Extension(
|
|
7
|
+
"lz4ext", ["src/python-lz4.c", "src/lz4.c", "src/lz4hc.c"],
|
|
8
|
+
define_macros=[("LZ4EXT_VERSION", '"%s"' % metadata["project"]["version"])],
|
|
9
|
+
)])
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
LZ4 Library
|
|
2
|
+
Copyright (c) 2011-2020, Yann Collet
|
|
3
|
+
All rights reserved.
|
|
4
|
+
|
|
5
|
+
Redistribution and use in source and binary forms, with or without modification,
|
|
6
|
+
are permitted provided that the following conditions are met:
|
|
7
|
+
|
|
8
|
+
* Redistributions of source code must retain the above copyright notice, this
|
|
9
|
+
list of conditions and the following disclaimer.
|
|
10
|
+
|
|
11
|
+
* Redistributions in binary form must reproduce the above copyright notice, this
|
|
12
|
+
list of conditions and the following disclaimer in the documentation and/or
|
|
13
|
+
other materials provided with the distribution.
|
|
14
|
+
|
|
15
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
|
|
16
|
+
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
|
17
|
+
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
|
18
|
+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR
|
|
19
|
+
ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
|
|
20
|
+
(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
|
|
21
|
+
LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
|
|
22
|
+
ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
|
23
|
+
(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
|
|
24
|
+
SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# Vendored LZ4
|
|
2
|
+
|
|
3
|
+
`lz4.c`, `lz4.h`, `lz4hc.c`, and `lz4hc.h` are unmodified copies of
|
|
4
|
+
the `lib/` files from the official LZ4 v1.10.0 release:
|
|
5
|
+
https://github.com/lz4/lz4/releases/tag/v1.10.0
|
|
6
|
+
|
|
7
|
+
Upstream commit: `ebb370ca83af193212df4dcbadcc5d87bc0de2f0`.
|
|
8
|
+
|
|
9
|
+
Retrieved from https://raw.githubusercontent.com/lz4/lz4/v1.10.0/lib/
|
|
10
|
+
on 2026-10-05. The upstream BSD-2-Clause license is in `LICENSE.lz4`.
|
|
11
|
+
|
|
12
|
+
When updating these files, copy them verbatim from an official release,
|
|
13
|
+
update this provenance and the library version in the README/changelog, and
|
|
14
|
+
retain the license. The extension reports `LZ4_VERSION` from the linked library.
|