pyfastcdc 0.2.0__tar.gz → 0.2.1__tar.gz

{pyfastcdc-0.2.0/pyfastcdc.egg-info → pyfastcdc-0.2.1}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: pyfastcdc
-Version: 0.2.0
-Summary: FastCDC 2020 implementation in Python, with Cython acceleration
+Version: 0.2.1
+Summary: A high-performance FastCDC 2020 implementation written in Python + Cython
 Author: Fallen_Breath
 License: MIT
 Project-URL: Homepage, https://github.com/Fallen-Breath/pyfastcdc
@@ -42,7 +42,7 @@ Dynamic: summary
 [![Issues](https://img.shields.io/github/issues/Fallen-Breath/pyfastcdc.svg)](https://github.com/Fallen-Breath/pyfastcdc/issues)
 [![PyPI Version](https://img.shields.io/pypi/v/pyfastcdc.svg?label=PyPI)](https://pypi.org/project/pyfastcdc)
 
-A FastCDC 2020 implementation written in Python, with [Cython](https://github.com/cython/cython) acceleration
+A high-performance FastCDC 2020 implementation written in Python + [Cython](https://github.com/cython/cython)
 
 Supports Python 3.6+. Provides prebuilt wheels for Python 3.8+
 
@@ -63,14 +63,46 @@ For platforms without prebuilt wheels, a suitable build environment capable of c
 For example, on Debian, you might need to install `gcc` and `python3-dev` via `apt`
 
 If the Cython extension fails to compile, the installation will fall back to a pure Python implementation,
-which is significantly slower (around 0.01× or less in memory chunking speed)
+which is significantly slower (about 0.5% or less in memory chunking speed)
+
+<details>
+
+<summary>I only want to use the Cython implemetion, not the slow pure-Python one</summary>
+
+You can set the environment variable `PYFASTCDC_REQUIRE_CYTHON=true` or `PYFASTCDC_REQUIRE_CYTHON=1` for the `pip` installation command
+to disable the pure-Python fallback on extension compilation error and make the installation fail hard.
+Thus, after a successful installation, you will always have a working Cython extension
+
+Example bash command using `pip`:
+
+```
+$ PYFASTCDC_REQUIRE_CYTHON=true pip install pyfastcdc
+... some pip output ...
+Building wheels for collected packages: pyfastcdc
+  Building wheel for pyfastcdc (pyproject.toml) ... error
+  ... some pip output ...
+  ###########################################################################################################
+  Failed to compile pyfastcdc Cython extension, fail hard since PYFASTCDC_REQUIRE_CYTHON is set to true
+  Unset PYFASTCDC_REQUIRE_CYTHON to allow pure-Python fallback if that's acceptable for your use case.
+  <class 'distutils.compilers.C.errors.CompileError'> command 'gcc' failed: No such file or directory
+  ###########################################################################################################
+  error: command 'gcc' failed: No such file or directory
+  ----------------------------------------
+  ERROR: Failed building wheel for pyfastcdc
+  ... some pip output ...
+```
+
+</details>
 
 ## Usage
 
-The basic usage is simple:
+The usage of PyFastCDC is simple:
 
 1. Construct a `FastCDC` instance with desired parameters
 2. Call `FastCDC.cut_xxx()` function to chunk your data
+    - Call `cut_buf()` to chunk in-memory data buffers
+    - Call `cut_file()` to chunk a regular file using mmap
+    - Call `cut_stream()` to chunk a custom file-like streaming object
 
 Example:
 
@@ -88,8 +120,8 @@ Please only import members from `pyfastcdc` in your application code and avoid i
 Only public APIs inside the `pyfastcdc` module are guaranteed to be stable across releases
 
 ```python
-from pyfastcdc import Chunk         # GOOD
-from pyfastcdc.common import Chunk  # BAD, no API stability guarantee
+from pyfastcdc import NormalizedChunking         # GOOD
+from pyfastcdc.common import NormalizedChunking  # BAD, no API stability guarantee
 ```
 
 ## Performance

{pyfastcdc-0.2.0 → pyfastcdc-0.2.1}/README.md

@@ -4,7 +4,7 @@
 [![Issues](https://img.shields.io/github/issues/Fallen-Breath/pyfastcdc.svg)](https://github.com/Fallen-Breath/pyfastcdc/issues)
 [![PyPI Version](https://img.shields.io/pypi/v/pyfastcdc.svg?label=PyPI)](https://pypi.org/project/pyfastcdc)
 
-A FastCDC 2020 implementation written in Python, with [Cython](https://github.com/cython/cython) acceleration
+A high-performance FastCDC 2020 implementation written in Python + [Cython](https://github.com/cython/cython)
 
 Supports Python 3.6+. Provides prebuilt wheels for Python 3.8+
 
@@ -25,14 +25,46 @@ For platforms without prebuilt wheels, a suitable build environment capable of c
 For example, on Debian, you might need to install `gcc` and `python3-dev` via `apt`
 
 If the Cython extension fails to compile, the installation will fall back to a pure Python implementation,
-which is significantly slower (around 0.01× or less in memory chunking speed)
+which is significantly slower (about 0.5% or less in memory chunking speed)
+
+<details>
+
+<summary>I only want to use the Cython implemetion, not the slow pure-Python one</summary>
+
+You can set the environment variable `PYFASTCDC_REQUIRE_CYTHON=true` or `PYFASTCDC_REQUIRE_CYTHON=1` for the `pip` installation command
+to disable the pure-Python fallback on extension compilation error and make the installation fail hard.
+Thus, after a successful installation, you will always have a working Cython extension
+
+Example bash command using `pip`:
+
+```
+$ PYFASTCDC_REQUIRE_CYTHON=true pip install pyfastcdc
+... some pip output ...
+Building wheels for collected packages: pyfastcdc
+  Building wheel for pyfastcdc (pyproject.toml) ... error
+  ... some pip output ...
+  ###########################################################################################################
+  Failed to compile pyfastcdc Cython extension, fail hard since PYFASTCDC_REQUIRE_CYTHON is set to true
+  Unset PYFASTCDC_REQUIRE_CYTHON to allow pure-Python fallback if that's acceptable for your use case.
+  <class 'distutils.compilers.C.errors.CompileError'> command 'gcc' failed: No such file or directory
+  ###########################################################################################################
+  error: command 'gcc' failed: No such file or directory
+  ----------------------------------------
+  ERROR: Failed building wheel for pyfastcdc
+  ... some pip output ...
+```
+
+</details>
 
 ## Usage
 
-The basic usage is simple:
+The usage of PyFastCDC is simple:
 
 1. Construct a `FastCDC` instance with desired parameters
 2. Call `FastCDC.cut_xxx()` function to chunk your data
+    - Call `cut_buf()` to chunk in-memory data buffers
+    - Call `cut_file()` to chunk a regular file using mmap
+    - Call `cut_stream()` to chunk a custom file-like streaming object
 
 Example:
 
@@ -50,8 +82,8 @@ Please only import members from `pyfastcdc` in your application code and avoid i
 Only public APIs inside the `pyfastcdc` module are guaranteed to be stable across releases
 
 ```python
-from pyfastcdc import Chunk         # GOOD
-from pyfastcdc.common import Chunk  # BAD, no API stability guarantee
+from pyfastcdc import NormalizedChunking         # GOOD
+from pyfastcdc.common import NormalizedChunking  # BAD, no API stability guarantee
 ```
 
 ## Performance

{pyfastcdc-0.2.0 → pyfastcdc-0.2.1}/pyfastcdc/__init__.py

@@ -1,4 +1,4 @@
-__version__ = '0.2.0'
+__version__ = '0.2.1'
 
 __all__ = [
 	'BinaryStreamReader',

{pyfastcdc-0.2.0 → pyfastcdc-0.2.1/pyfastcdc.egg-info}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: pyfastcdc
-Version: 0.2.0
-Summary: FastCDC 2020 implementation in Python, with Cython acceleration
+Version: 0.2.1
+Summary: A high-performance FastCDC 2020 implementation written in Python + Cython
 Author: Fallen_Breath
 License: MIT
 Project-URL: Homepage, https://github.com/Fallen-Breath/pyfastcdc
@@ -42,7 +42,7 @@ Dynamic: summary
 [![Issues](https://img.shields.io/github/issues/Fallen-Breath/pyfastcdc.svg)](https://github.com/Fallen-Breath/pyfastcdc/issues)
 [![PyPI Version](https://img.shields.io/pypi/v/pyfastcdc.svg?label=PyPI)](https://pypi.org/project/pyfastcdc)
 
-A FastCDC 2020 implementation written in Python, with [Cython](https://github.com/cython/cython) acceleration
+A high-performance FastCDC 2020 implementation written in Python + [Cython](https://github.com/cython/cython)
 
 Supports Python 3.6+. Provides prebuilt wheels for Python 3.8+
 
@@ -63,14 +63,46 @@ For platforms without prebuilt wheels, a suitable build environment capable of c
 For example, on Debian, you might need to install `gcc` and `python3-dev` via `apt`
 
 If the Cython extension fails to compile, the installation will fall back to a pure Python implementation,
-which is significantly slower (around 0.01× or less in memory chunking speed)
+which is significantly slower (about 0.5% or less in memory chunking speed)
+
+<details>
+
+<summary>I only want to use the Cython implemetion, not the slow pure-Python one</summary>
+
+You can set the environment variable `PYFASTCDC_REQUIRE_CYTHON=true` or `PYFASTCDC_REQUIRE_CYTHON=1` for the `pip` installation command
+to disable the pure-Python fallback on extension compilation error and make the installation fail hard.
+Thus, after a successful installation, you will always have a working Cython extension
+
+Example bash command using `pip`:
+
+```
+$ PYFASTCDC_REQUIRE_CYTHON=true pip install pyfastcdc
+... some pip output ...
+Building wheels for collected packages: pyfastcdc
+  Building wheel for pyfastcdc (pyproject.toml) ... error
+  ... some pip output ...
+  ###########################################################################################################
+  Failed to compile pyfastcdc Cython extension, fail hard since PYFASTCDC_REQUIRE_CYTHON is set to true
+  Unset PYFASTCDC_REQUIRE_CYTHON to allow pure-Python fallback if that's acceptable for your use case.
+  <class 'distutils.compilers.C.errors.CompileError'> command 'gcc' failed: No such file or directory
+  ###########################################################################################################
+  error: command 'gcc' failed: No such file or directory
+  ----------------------------------------
+  ERROR: Failed building wheel for pyfastcdc
+  ... some pip output ...
+```
+
+</details>
 
 ## Usage
 
-The basic usage is simple:
+The usage of PyFastCDC is simple:
 
 1. Construct a `FastCDC` instance with desired parameters
 2. Call `FastCDC.cut_xxx()` function to chunk your data
+    - Call `cut_buf()` to chunk in-memory data buffers
+    - Call `cut_file()` to chunk a regular file using mmap
+    - Call `cut_stream()` to chunk a custom file-like streaming object
 
 Example:
 
@@ -88,8 +120,8 @@ Please only import members from `pyfastcdc` in your application code and avoid i
 Only public APIs inside the `pyfastcdc` module are guaranteed to be stable across releases
 
 ```python
-from pyfastcdc import Chunk         # GOOD
-from pyfastcdc.common import Chunk  # BAD, no API stability guarantee
+from pyfastcdc import NormalizedChunking         # GOOD
+from pyfastcdc.common import NormalizedChunking  # BAD, no API stability guarantee
 ```
 
 ## Performance

{pyfastcdc-0.2.0 → pyfastcdc-0.2.1}/setup.py

@@ -1,6 +1,7 @@
 import ast
 import contextlib
 import functools
+import os
 import sys
 from pathlib import Path
 from typing import List
@@ -59,12 +60,20 @@ class BuildExt(build_ext):
 	@classmethod
 	@contextlib.contextmanager
 	def __wrap_ext_err(cls):
+		require_cython = os.environ.get('PYFASTCDC_REQUIRE_CYTHON', '').lower() in ['true', '1']
 		try:
 			yield
 		except Exception as e:
+			if require_cython:
+				print("###########################################################################################################", file=sys.stderr)
+				print("Failed to compile pyfastcdc Cython extension, fail hard since PYFASTCDC_REQUIRE_CYTHON is set to true", file=sys.stderr)
+				print("Unset PYFASTCDC_REQUIRE_CYTHON to allow pure-Python fallback if that's acceptable for your use case.", file=sys.stderr)
+				print(type(e), e, file=sys.stderr)
+				print("###########################################################################################################", file=sys.stderr)
+				raise e
 			print("###########################################################################################################", file=sys.stderr)
-			print("Failed to compile pyfastcdc cython extension, fallback to pure python implementation with is a lot slower", file=sys.stderr)
-			print(e)
+			print("Failed to compile pyfastcdc Cython extension, fallback to pure python implementation with is a lot slower", file=sys.stderr)
+			print(type(e), e, file=sys.stderr)
 			print("###########################################################################################################", file=sys.stderr)
 
 	def run(self):
@@ -97,7 +106,7 @@ print(f'Cython version: {cython_version}')
 setup(
 	name='pyfastcdc',
 	version=get_version(),
-	description='FastCDC 2020 implementation in Python, with Cython acceleration',
+	description='A high-performance FastCDC 2020 implementation written in Python + Cython',
 	long_description=read_file('README.md'),
 	long_description_content_type='text/markdown',
 	author='Fallen_Breath',