hashcodecs 1.4.0__tar.gz → 1.4.1__tar.gz

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/BENCHMARK.md

@@ -9,9 +9,8 @@ Build the Python wheel with CPython 3.12 and the full C API. Keep competitor val
 Use `uv run --python 3.12 --no-project python benchmarks/render_charts.py` to render the charts. Read exact values in
 [docs/benchmarks/results.csv](docs/benchmarks/results.csv).
 
-The standard and URL-safe Python decode panels report CPython 3.12.10 measurements from 2026-09-09; the lenient
-decode chart retains its 2026-09-05 measurements. Each value is the median of 15 samples lasting at least 0.2
-seconds each, with one logical CPU pinned.
+Python values use CPython 3.12.10 and report the median of 15 samples lasting at least 0.2 seconds each, with one
+logical CPU pinned. Focused runs refresh only the affected series; other values retain their previous measurements.
 
 ## Timing Controls
 
@@ -31,6 +30,43 @@ GIL-detachment cutoffs. The `--thread-scaling` mode measures aggregate throughpu
 it does not pin the process to one logical CPU. Use `--buffer-inputs` to compare 64-byte and 4 KiB XXH3-64 calls
 across bytes, full and sliced memoryviews, writable and non-contiguous views, and `array('B')`.
 
+## Rust MurmurHash3 x64 Dispatch
+
+Use scalar below 512 bytes of full blocks, then AVX2 when available. The SSE4.1 fallback starts at 512 bytes
+and retains its 8 MiB upper limit. Incremental updates apply these thresholds to each batch of full blocks.
+
+Forced-backend measurements with complete finalization on the Core Ultra 7 265K put scalar and AVX2 near parity
+at 384 bytes (39.99 and 40.20 ns/hash), with AVX2 ahead at 512 bytes (53.98 and 52.76 ns/hash). Use 512 bytes as
+a crossover candidate for this host. The shared minimum also avoids the measured SSE4.1 overhead on small inputs;
+it does not establish an SSE4.1 crossover or an optimum across Intel and AMD CPUs.
+
+The following public Rust API measurements include runtime dispatch and finalization. On 2026-09-13, we collected
+50 Criterion samples per case with one logical CPU pinned, a 300 ms warmup, and a 1 s measurement target. These
+values report Criterion's mean estimate with seed 42; they are separate from the forced-backend measurements.
+
+| Input | ns/hash |
+| --- | ---: |
+| 15 B | 5.32 |
+| 16 B | 5.47 |
+| 31 B | 5.89 |
+| 32 B | 6.03 |
+| 64 B | 7.80 |
+| 255 B | 24.42 |
+| 256 B | 25.04 |
+| 384 B | 37.93 |
+| 511 B | 50.35 |
+| 512 B | 49.79 |
+| 513 B | 50.43 |
+| 1 KiB | 96.89 |
+
+We refreshed the x64 hashcodecs series in the Rust MurmurHash3 throughput chart on the same date, retaining the
+x86 and competitor measurements from the previous comparison run.
+
+```sh
+cargo bench --manifest-path benches/Cargo.toml --bench crossover -- murmur_x64_128_crossover
+cargo bench --manifest-path benches/Cargo.toml --bench murmur3 -- x64_128/hashcodecs
+```
+
 ## XXH3
 
 For the Rust comparison, link hashcodecs with xxHash 0.8.3 through `xxhash-c-sys`. Build the C baseline with AVX2.
@@ -51,14 +87,61 @@ bytearrays, and writable memoryviews. Use `--batch-counts 2 9 32 33 --sizes 64`
 
 The [32-item parent comparison](docs/benchmarks/xxh3-batch-parent-comparison.csv) records CPython 3.12.10 results
 against parent commit `f17ab86`, measured on 2026-09-05. These paired measurements also cover the 256 KiB
-GIL-detachment threshold and 1 MiB items. The Python XXH3 chart uses the candidate's allocating-batch measurements;
-packed-output and upstream values retain their prior measurements.
+GIL-detachment threshold and 1 MiB items. The Python XXH3 batch panels report CPython 3.12.10 measurements
+from 2026-09-13, using 15 samples of at least 0.2 seconds; one-shot and upstream values retain their prior measurements.
 
 Across the 30 paired 32-item cases, branch throughput ranges from 2.59% lower to 2.70% higher than the parent.
 The [stack-boundary comparison](docs/benchmarks/xxh3-batch-boundary-comparison.csv) covers counts 2 and 33 with
 64-byte items: two-item bytearray batches lose 5.40–6.05%, and 33-item bytes batches lose 6.06–7.05%. These
 measurements show residual overhead for some small-input batches; they do not establish zero regression.
 
+### Packed-batch detachment
+
+XXH3 batches release the GIL at 1 MiB of total input or 16,384 items, provided the input buffers permit detachment.
+The item limit accounts for per-item hashing and output work, including empty inputs. One-shot XXH3 keeps its
+256 KiB threshold. Mutable inputs retain their existing synchronization rules.
+
+The detached packed path retains input owners, borrows 64 inputs at a time on the stack, and stages results until
+it reacquires the GIL and rechecks the output size. This removes one allocation and 16 bytes of temporary input
+descriptors per item on 64-bit hosts. Little-endian hosts copy staged results in one operation.
+
+The [candidate measurements](docs/benchmarks/xxh3-packed-candidates.csv) compare 256 KiB, 512 KiB, and 1 MiB
+byte thresholds with the same allocation reduction and 16,384-item limit. Each exploratory value uses five
+samples of at least 0.03 seconds. The 1 MiB policy keeps both 4,096- and 8,192-item batches of 64-byte inputs on
+the direct-output path. This trades longer GIL holds for lower latency; it does not establish an optimal threshold
+for other processors or contended workloads.
+
+The [selected-policy measurements](docs/benchmarks/xxh3-packed-thresholds.csv) use CPython 3.15.0b4 on the
+Intel Core Ultra 7 265K, with one logical CPU pinned, independent input allocations, and 15 samples of at least
+0.2 seconds each, measured on 2026-09-13. They cover both digest widths and item sizes of 64 bytes, 1 KiB, and 64 KiB.
+For 64-byte inputs:
+
+| Items | XXH3-64 packed latency | XXH3-128 packed latency |
+| ---: | ---: | ---: |
+| 4,095 | 13.92 µs | 27.29 µs |
+| 4,096 | 14.48 µs | 27.33 µs |
+| 4,097 | 13.95 µs | 27.44 µs |
+| 8,191 | 28.95 µs | 54.72 µs |
+| 8,192 | 29.14 µs | 54.61 µs |
+| 8,193 | 29.46 µs | 54.57 µs |
+| 16,383 | 58.49 µs | 109.12 µs |
+| 16,384 | 92.78 µs | 144.04 µs |
+| 16,385 | 94.52 µs | 145.62 µs |
+
+A discontinuity remains at the new detachment boundary because retaining owners and staging output still cost work.
+The longest measured attached call near these boundaries takes about 109 microseconds. This is a latency
+measurement on this host, not a bound on thread waiting time; the tests also verify GIL progress for large inputs
+and for 16,384-item batches of empty, one-byte, and 64-byte inputs.
+
+```sh
+uv run --frozen --no-sync python benchmarks/python_xxhash_thresholds.py --output docs/benchmarks/xxh3-packed-thresholds.csv
+uv run --frozen --no-sync python benchmarks/python_xxhash.py --batches-only --hashcodecs-only
+```
+
+`--thresholds-kib` selects benchmark input sizes around each candidate; it does not change the extension's
+compiled policy. To compare candidates, rebuild the wheel after changing `BATCH_DETACH_BYTES` in
+`src/bindings/xxhash/batch.rs`.
+
 The Rust mixed benchmarks use `[1024, 1024, 4096, 4096]`, `[257, 258, 259, 260]`, `[240, 240, 241, 241]`, and the
 reverse boundary order. The 1024/4096 case measures adjacent two-item long runs. The 257–260 case measures a
 four-item run with one shared stripe count and distinct final stripes. The 240/241 cases measure both orders across
@@ -89,6 +172,11 @@ Pass one reusable `bytearray` to each `*_into` call.
 Run `python benchmarks/python_base64.py --lenient`. The MIME cases insert CRLF after each 76-character line. The
 noisy cases insert `!` at the same boundaries. Both cases measure returned bytes and reusable output buffers.
 
+Run `uv run --no-project --python 3.12 python benchmarks/python_base64.py --custom-lenient` to measure clean and noisy
+inputs with `@#` altchars. For a 1 MiB decoded payload, the custom clean case reaches 3.44 GiB/s with returned bytes
+and 13.05 GiB/s with a reusable output buffer. The decoder translates complete symbol runs in a 4 KiB staging buffer
+for SIMD decoding and handles padding and ignored characters through the lenient state machine.
+
 [![Lenient Python Base64 throughput](docs/benchmarks/base64-python-lenient.svg)](docs/benchmarks/base64-python-lenient.svg)
 
 ## Wrapped Python Base64
@@ -99,9 +187,8 @@ Run `python benchmarks/python_base64.py --wrapped` with CPython 3.15 or newer. T
 ## Python Memoryview Inputs
 
 Use `--memoryview-input` for full immutable views and `--sliced-memoryview-input` for equal-length contiguous views
-with a nonzero starting offset. Full views can recover their exact immutable owner at detachment sizes; slices cover
-offset-buffer handling, which borrows under the GIL and stabilizes the input in free-threaded builds. The encoded data
-remains identical.
+with a nonzero starting offset. At detachment sizes, both retain the immutable bytes owner without copying the input.
+Callback-capable decode arguments keep their required snapshots and conversion order. Both modes use identical data.
 
 [![Python Base64 memoryview inputs](docs/benchmarks/base64-python-memoryview.svg)](docs/benchmarks/base64-python-memoryview.svg)
 

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/CHANGELOG.md

@@ -4,6 +4,24 @@ This file records notable user-facing changes to `hashcodecs`. Version 1.0.0 sta
 
 ## [Unreleased]
 
+## [1.4.1] - 2026-09-13
+
+### What's Changed
+* fix: match CPython Base64 and XXH3 buffer semantics by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/114
+* fix: preserve Base64 callback buffer semantics by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/115
+* fix: preserve CPython Base64 observable behavior by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/116
+* fix: resolve repository type diagnostics by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/117
+* fix: release Base64 input buffers before writing output by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/118
+* refactor: centralize buffer callback policy and clarify finalizers by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/119
+* fix: accelerate custom-alphabet lenient Base64 decoding by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/120
+* fix: reduce XXH3 packed batch detachment overhead by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/121
+* fix: delay MurmurHash3 x64 SIMD dispatch until 512 bytes by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/122
+* fix: restore Python buffer and batch throughput by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/123
+* fix: typo in the performance viz image by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/124
+
+
+**Full Changelog**: https://github.com/kozistr/hashcodecs-rs/compare/v1.4.0...v1.4.1
+
 ## [1.4.0] - 2026-09-10
 
 ### What's Changed
@@ -193,7 +211,8 @@ This file records notable user-facing changes to `hashcodecs`. Version 1.0.0 sta
 - Initial Python and Rust APIs for Base64 and MurmurHash3.
 - Runtime SIMD dispatch and platform-specific CPython wheels.
 
-[Unreleased]: https://github.com/kozistr/hashcodecs-rs/compare/v1.4.0...HEAD
+[Unreleased]: https://github.com/kozistr/hashcodecs-rs/compare/v1.4.1...HEAD
+[1.4.1]: https://github.com/kozistr/hashcodecs-rs/compare/v1.4.0...v1.4.1
 [1.4.0]: https://github.com/kozistr/hashcodecs-rs/compare/v1.3.0...v1.4.0
 [1.3.0]: https://github.com/kozistr/hashcodecs-rs/compare/v1.2.1...v1.3.0
 [1.2.1]: https://github.com/kozistr/hashcodecs-rs/compare/v1.2.0...v1.2.1

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/CITATION.cff

@@ -5,8 +5,8 @@ authors:
     given-names: Hyeongchan
     orcid: https://orcid.org/0000-0002-1729-0580
 title: "hashcodecs: SIMD-accelerated Base64, MurmurHash3, and XXH3 for Python and Rust"
-version: 1.4.0
-date-released: 2026-09-10
+version: 1.4.1
+date-released: 2026-09-13
 license: "MIT OR Apache-2.0"
 repository-code: "https://github.com/kozistr/hashcodecs-rs"
 url: "https://github.com/kozistr/hashcodecs-rs"

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/Cargo.lock

@@ -194,7 +194,7 @@ dependencies = [
 
 [[package]]
 name = "hashcodecs"
-version = "1.4.0"
+version = "1.4.1"
 dependencies = [
  "base64",
  "memchr",

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "hashcodecs"
-version = "1.4.0"
+version = "1.4.1"
 edition = "2024"
 rust-version = "1.89"
 autobenches = false

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: hashcodecs
-Version: 1.4.0
+Version: 1.4.1
 Summary: SIMD-accelerated Base64, MurmurHash3, and xxHash codecs
 Project-URL: Documentation, https://hashcodecs-rs.readthedocs.io/
 Project-URL: Repository, https://github.com/kozistr/hashcodecs-rs
@@ -253,11 +253,7 @@ cargo bench --manifest-path benches/Cargo.toml --bench xxhash
 
 ## Performance snapshot
 
-In the focused 2026-09-09 hashcodecs-only run on the benchmark host, `hashcodecs.xxh3_64` processes a 1 MiB input
-at 91.27 GiB/s. In the full 2026-09-02 run, the Base64 batch API reaches 11.84 GiB/s for encode and 6.23 GiB/s for
-decode with 256 B items in batches of 64. Each run pins one logical CPU and uses 15 samples with a 0.2-second
-minimum per sample. Read the
-[benchmark details](BENCHMARK.md) and [raw comparison results](docs/benchmarks/results.csv).
+On the benchmark host, `hashcodecs.xxh3_64` processes a 1 MiB input at 91.62 GiB/s. The Base64 batch API reaches 11.56 GiB/s for encode and 8.28 GiB/s for decode with 256 B items in batches of 64. Each run pins one logical CPU and uses 15 samples with a 0.2-second minimum per sample. Read the [benchmark details](BENCHMARK.md) and [raw comparison results](docs/benchmarks/results.csv).
 
 ## Development
 

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/README.md

@@ -222,11 +222,7 @@ cargo bench --manifest-path benches/Cargo.toml --bench xxhash
 
 ## Performance snapshot
 
-In the focused 2026-09-09 hashcodecs-only run on the benchmark host, `hashcodecs.xxh3_64` processes a 1 MiB input
-at 91.27 GiB/s. In the full 2026-09-02 run, the Base64 batch API reaches 11.84 GiB/s for encode and 6.23 GiB/s for
-decode with 256 B items in batches of 64. Each run pins one logical CPU and uses 15 samples with a 0.2-second
-minimum per sample. Read the
-[benchmark details](BENCHMARK.md) and [raw comparison results](docs/benchmarks/results.csv).
+On the benchmark host, `hashcodecs.xxh3_64` processes a 1 MiB input at 91.62 GiB/s. The Base64 batch API reaches 11.56 GiB/s for encode and 8.28 GiB/s for decode with 256 B items in batches of 64. Each run pins one logical CPU and uses 15 samples with a 0.2-second minimum per sample. Read the [benchmark details](BENCHMARK.md) and [raw comparison results](docs/benchmarks/results.csv).
 
 ## Development
 

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/SAFETY.md

@@ -27,6 +27,14 @@ batches use a fallible vector. Large immutable inputs keep their owners across
 GIL detachment. Subprocess tests on CPython 3.10/3.11 trigger finalizers during
 allocation and reuse freed storage, covering both sides of the stack boundary.
 
+Packed XXH3 batches retain immutable input owners before detaching, then borrow
+64 inputs at a time into stack storage. They stage hashes in native memory;
+after reattaching, they recheck the destination length before copying results.
+The little-endian bulk copy relies on the padding-free layout contract of the
+private `PackedDigest` trait. Big-endian hosts serialize each word. Tests clear
+the source list and shrink the output from another Python thread before hashing
+the retained inputs and checking the output size again.
+
 Run the same checks locally on Linux:
 
 ```sh

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/benches/crossover.rs

@@ -8,6 +8,7 @@ mod support;
 const BASE64_ENCODE_SIZES: [usize; 10] = [15, 16, 31, 32, 47, 48, 51, 52, 103, 104];
 const BASE64_DECODE_SIZES: [usize; 8] = [12, 16, 28, 32, 60, 64, 124, 128];
 const MURMUR_SIZES: [usize; 6] = [15, 16, 31, 32, 255, 256];
+const MURMUR_X64_SIZES: [usize; 12] = [15, 16, 31, 32, 64, 255, 256, 384, 511, 512, 513, 1024];
 
 fn data(size: usize) -> Vec<u8> {
     (0..size)
@@ -41,9 +42,9 @@ fn base64_decode(c: &mut Criterion) {
 }
 
 macro_rules! murmur_group {
-    ($criterion:expr, $name:literal, $function:path) => {{
+    ($criterion:expr, $name:literal, $function:path, $sizes:expr) => {{
         let mut group = $criterion.benchmark_group($name);
-        for size in MURMUR_SIZES {
+        for size in $sizes {
             let input = data(size);
             group.throughput(Throughput::Bytes(size as u64));
             group.bench_with_input(BenchmarkId::from_parameter(size), &input, |bench, input| {
@@ -58,17 +59,20 @@ fn murmur3(c: &mut Criterion) {
     murmur_group!(
         c,
         "murmur_x86_32_crossover",
-        hashcodecs::murmur3::murmur3_x86_32
+        hashcodecs::murmur3::murmur3_x86_32,
+        MURMUR_SIZES
     );
     murmur_group!(
         c,
         "murmur_x86_128_crossover",
-        hashcodecs::murmur3::murmur3_x86_128
+        hashcodecs::murmur3::murmur3_x86_128,
+        MURMUR_SIZES
     );
     murmur_group!(
         c,
         "murmur_x64_128_crossover",
-        hashcodecs::murmur3::murmur3_x64_128
+        hashcodecs::murmur3::murmur3_x64_128,
+        MURMUR_X64_SIZES
     );
 }
 

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "hashcodecs"
-version = "1.4.0"
+version = "1.4.1"
 description = "SIMD-accelerated Base64, MurmurHash3, and xxHash codecs"
 readme = "README.md"
 requires-python = ">=3.10"

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/src/base64/decode/avx2.rs

@@ -14,14 +14,9 @@ use super::tables::{
 };
 use super::x86_contracts::{Decoder, Store};
 
-/// TODO
-/// So my optimization priority would be:
-/// 1. Split bulk and terminal blocks, eliminating the per-iteration padded-store branches.
-/// 2. Cache input_ptr / input_len.
-/// 3. Inspect assembly for spills caused by the 4× unroll.
-/// 4. Benchmark 2× versus 4× unrolling.
-/// 5. Leave the combined error reduction alone unless profiling says otherwise.
-///    I'd inspect cargo asm/Compiler Explorer output for spills and benchmark 2× vs 4×.
+// Optimization candidate: compare two-block and four-block unrolling after
+// checking generated assembly for register spills. Preserve the combined error
+// reduction and the Store policy's exact output boundary for the final block.
 #[target_feature(enable = "avx2")]
 pub(crate) unsafe fn decode_avx2<A: Decoder, S: Store>(
     input: &[u8],

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/src/bindings/arguments.rs

@@ -3,6 +3,7 @@ use std::ptr;
 
 use pyo3::ffi;
 
+#[inline]
 pub(super) unsafe fn parse_raw_arguments<const N: usize>(
     args: *const *mut ffi::PyObject,
     nargs: isize,
@@ -40,8 +41,10 @@ pub(super) unsafe fn parse_raw_arguments<const N: usize>(
     for keyword_index in 0..keyword_count {
         let keyword = unsafe { tuple_item(keywords, keyword_index) };
         let value = unsafe { *args.add(nargs + keyword_index) };
-        let parameter_index = parameter_names.iter().position(|parameter| {
-            (unsafe { ffi::PyUnicode_CompareWithASCIIString(keyword, *parameter) }) == 0
+        // Valid keywords follow the positional arguments. Search those slots
+        // first, then check the filled slots to preserve duplicate diagnostics.
+        let parameter_index = (nargs..N).chain(0..nargs).find(|&index| {
+            (unsafe { ffi::PyUnicode_CompareWithASCIIString(keyword, parameter_names[index]) }) == 0
         });
 
         let Some(parameter_index) = parameter_index else {

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/src/bindings/base64/api.rs

@@ -63,9 +63,7 @@ callback! {
 
 callback! {
     urlsafe_b64encode, |py; s, padded| {
-        let result = padded
-            .truthy(py)
-            .and_then(|padded| encode::urlsafe_b64encode(py, s.raw(py), padded));
+        let result = encode::urlsafe_b64encode(py, s.raw(py), padded);
         return_bound(py, result)
     }
 }
@@ -74,7 +72,7 @@ callback! {
     urlsafe_b64encode_into, |py; s, output, padded| {
         let result = (|| {
             let output = output.raw(py).cast::<PyByteArray>()?;
-            encode::urlsafe_b64encode_into(s.raw(py), output, padded.truthy(py)?)
+            encode::urlsafe_b64encode_into(py, s.raw(py), output, padded)
         })();
         return_usize(py, result)
     }
@@ -169,17 +167,15 @@ callback! {
 
 callback! {
     b64decode, |py; s, altchars, validate, padded, ignorechars, canonical| {
-        let result = (|| {
-            decode::b64decode(
-                py,
-                s.raw(py),
-                altchars.optional(py),
-                validate.optional_truthy(py)?,
-                padded.truthy(py)?,
-                ignorechars.provided(py),
-                canonical.truthy(py)?,
-            )
-        })();
+        let result = decode::b64decode(
+            py,
+            s.raw(py),
+            altchars.optional(py),
+            validate,
+            padded,
+            ignorechars.provided(py),
+            canonical,
+        );
         return_bound(py, result)
     }
 }
@@ -278,10 +274,10 @@ callback! {
                 s.raw(py),
                 output,
                 altchars.optional(py),
-                validate.optional_truthy(py)?,
-                padded.truthy(py)?,
+                validate,
+                padded,
                 ignorechars.provided(py),
-                canonical.truthy(py)?,
+                canonical,
             )
         })();
         return_usize(py, result)
@@ -290,9 +286,7 @@ callback! {
 
 callback! {
     urlsafe_b64decode, |py; s, padded| {
-        let result = padded
-            .truthy(py)
-            .and_then(|padded| decode::urlsafe_b64decode(py, s.raw(py), padded));
+        let result = decode::urlsafe_b64decode(py, s.raw(py), padded);
         return_bound(py, result)
     }
 }
@@ -301,7 +295,7 @@ callback! {
     urlsafe_b64decode_into, |py; s, output, padded| {
         let result = (|| {
             let output = output.raw(py).cast::<PyByteArray>()?;
-            decode::urlsafe_b64decode_into(py, s.raw(py), output, padded.truthy(py)?)
+            decode::urlsafe_b64decode_into(py, s.raw(py), output, padded)
         })();
         return_usize(py, result)
     }

{hashcodecs-1.4.0 → hashcodecs-1.4.1}/src/bindings/base64/batch.rs

@@ -10,8 +10,8 @@ use pyo3::prelude::*;
 use pyo3::types::{PyAny, PyByteArray, PyBytes, PyInt, PyList, PyMemoryView, PyString};
 
 #[cfg(not(Py_GIL_DISABLED))]
-use super::encode::encode_exact;
-use super::encode::{encode_parsed, encode_parsed_into};
+use super::encode::encode_small_padded;
+use super::encode::{PreparedEncoder, encode_into, encode_with_prepared};
 use super::policy::{DecodePolicy, PreparedDecoder};
 use crate::bindings::buffer::{
     BufferRange, BytesLike, ascii_or_bytes, ascii_or_bytes_owned, contiguous_bytes_like,
@@ -232,10 +232,7 @@ fn prepare_batch_inputs<'py>(
     stage_snapshots(&mut prepared, |input, policy| {
         // ReleaseBeforeWrite also covers a temporary exact memoryview returned by
         // a string subclass. The items list does not retain that memoryview.
-        let needed = input.buffer_release_may_reenter()
-            || (policy == SnapshotPolicy::ReleaseBeforeWrite && input.has_borrowed_buffer());
-
-        input.snapshot_if(needed)
+        input.snapshot_before_output_write(policy == SnapshotPolicy::ReleaseBeforeWrite)
     })?;
 
     // Capture destination ranges after every reentrant release, then stage all
@@ -269,39 +266,34 @@ pub(super) fn b64encode_batch_parsed<'py>(
 ) -> PyResult<Bound<'py, PyList>> {
     #[cfg(not(Py_GIL_DISABLED))]
     let (items, exact_bytes_fast_path) = list_items_and_all(items, |item| {
-        PyBytes::is_exact_type_of(item)
-            && unsafe { item.cast_unchecked::<PyBytes>() }.as_bytes().len() <= EXACT_BYTES_BATCH_MAX
+        // Only the length is needed while retaining the list; avoid acquiring
+        // the bytes pointer through the C API for every tiny item.
+        unsafe {
+            ffi::PyBytes_CheckExact(item.as_ptr()) != 0
+                && ffi::Py_SIZE(item.as_ptr()) as usize <= EXACT_BYTES_BATCH_MAX
+        }
     })?;
     #[cfg(Py_GIL_DISABLED)]
     let items = list_items(items)?;
 
+    let encoder = PreparedEncoder::new(altchars, true, None);
+
     #[cfg(not(Py_GIL_DISABLED))]
     if exact_bytes_fast_path {
         // Validation retains every input before allocating the output list.
         // Creating a GC-tracked Python object can run finalizers which mutate
         // the original list.
-        let length = items.len();
-        let mut items = items.into_iter();
-        return list_from_fn(py, length, |_| {
-            let item = unsafe {
-                items
-                    .next()
-                    .expect("batch item count is exact")
-                    .cast_into_unchecked::<PyBytes>()
-            };
-            encode_exact(py, item.as_bytes(), altchars, true, None)
+        return list_from_fn(py, items.len(), |index| {
+            let input = BytesLike::Bytes(unsafe { items[index].cast_unchecked::<PyBytes>() });
+            unsafe { input.with_bytes(|input| encode_small_padded(py, input, &encoder)) }
         });
     }
     let length = items.len();
     let mut items = items.into_iter();
     list_from_fn(py, length, |_| {
-        encode_parsed(
-            py,
-            &items.next().expect("batch item count is exact"),
-            altchars,
-            true,
-            None,
-        )
+        let item = items.next().expect("batch item count is exact");
+        let input = crate::bindings::buffer::binascii_contiguous_bytes_like(&item)?;
+        encode_with_prepared(py, &input, &encoder)
     })
 }
 
@@ -342,14 +334,14 @@ pub(super) fn b64encode_batch_into_parsed<'py>(
         {
             Some(Ok(input)) => Ok(PyInt::new(
                 py,
-                encode_parsed_into(&input, output, altchars, true, None)?,
+                encode_into(&input, output, altchars, true, None)?,
             )),
             Some(Err(error)) => Err(error),
             None => {
                 let input = contiguous_bytes_like(&items[index], "s")?;
                 Ok(PyInt::new(
                     py,
-                    encode_parsed_into(&input, output, altchars, true, None)?,
+                    encode_into(&input, output, altchars, true, None)?,
                 ))
             }
         }
@@ -378,9 +370,11 @@ pub(super) fn b64decode_batch_parsed<'py>(
     altchars: Option<[u8; 2]>,
     validate: bool,
 ) -> PyResult<Bound<'py, PyList>> {
+    #[cfg(not(Py_GIL_DISABLED))]
+    let (items, exact_bytes_fast_path) = list_items_and_all(items, PyBytes::is_exact_type_of)?;
+    #[cfg(Py_GIL_DISABLED)]
     let items = list_items(items)?;
     let length = items.len();
-    let mut items = items.into_iter();
 
     let decoder = PreparedDecoder::new_for_batch(
         py,
@@ -388,6 +382,16 @@ pub(super) fn b64decode_batch_parsed<'py>(
         length,
     )?;
 
+    #[cfg(not(Py_GIL_DISABLED))]
+    if exact_bytes_fast_path {
+        // Retain the validated inputs through output-list allocation and borrow
+        // them throughout decoding, releasing the references after the batch.
+        return list_from_fn(py, length, |index| {
+            let input = BytesLike::Bytes(unsafe { items[index].cast_unchecked::<PyBytes>() });
+            decoder.decode_allocating(py, &input)
+        });
+    }
+    let mut items = items.into_iter();
     list_from_fn(py, length, |_| {
         let item = items.next().expect("batch item count is exact");
         let input = ascii_or_bytes(py, &item, "s")?;