hashcodecs 1.3.0__tar.gz → 1.4.0__tar.gz

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/.gitignore

@@ -10,6 +10,9 @@ site/
 .venv*/
 .coverage
 coverage.xml
+*.profraw
+*.lcov
+*.log
 
 # These are backup files generated by rustfmt
 **/*.rs.bk

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/BENCHMARK.md

@@ -9,6 +9,10 @@ Build the Python wheel with CPython 3.12 and the full C API. Keep competitor val
 Use `uv run --python 3.12 --no-project python benchmarks/render_charts.py` to render the charts. Read exact values in
 [docs/benchmarks/results.csv](docs/benchmarks/results.csv).
 
+The standard and URL-safe Python decode panels report CPython 3.12.10 measurements from 2026-09-09; the lenient
+decode chart retains its 2026-09-05 measurements. Each value is the median of 15 samples lasting at least 0.2
+seconds each, with one logical CPU pinned.
+
 ## Timing Controls
 
 Every Python benchmark accepts `--samples` (default: 15) and `--minimum-sample-seconds` (default: 0.2). Their
@@ -34,15 +38,38 @@ For Python, run the upstream `xxhash` extension beside hashcodecs. Pass 32 equal
 The Rust remainder cases pass two or three equal-size long inputs. Run Python remainder cases with
 `python benchmarks/python_xxhash.py --batch-counts 2 3`.
 
-The Rust mixed benchmarks use `[1024, 1024, 4096, 4096]`, `[240, 240, 241, 241]`, and the reverse boundary order.
-The 1024/4096 case measures adjacent two-item long runs. The 240/241 cases measure both orders across the
-short/long dispatch boundary.
+To measure a branch against its parent, build both extensions with the same Python and Rust toolchains, then run:
+
+```sh
+uv run --frozen --no-sync python benchmarks/compare_xxh3_batches.py path/to/parent/_hashcodecs.pyd path/to/branch/_hashcodecs.pyd
+```
+
+Use the corresponding `.so` paths on Linux or macOS. The comparison loads both builds into one interpreter, pins
+one CPU, and alternates their timing order across 15 samples. It checks matching digests and covers bytes,
+bytearrays, and writable memoryviews. Use `--batch-counts 2 9 32 33 --sizes 64` to inspect small batches and the
+32-result stack boundary. Positive `change_percent` values mean higher branch throughput.
+
+The [32-item parent comparison](docs/benchmarks/xxh3-batch-parent-comparison.csv) records CPython 3.12.10 results
+against parent commit `f17ab86`, measured on 2026-09-05. These paired measurements also cover the 256 KiB
+GIL-detachment threshold and 1 MiB items. The Python XXH3 chart uses the candidate's allocating-batch measurements;
+packed-output and upstream values retain their prior measurements.
+
+Across the 30 paired 32-item cases, branch throughput ranges from 2.59% lower to 2.70% higher than the parent.
+The [stack-boundary comparison](docs/benchmarks/xxh3-batch-boundary-comparison.csv) covers counts 2 and 33 with
+64-byte items: two-item bytearray batches lose 5.40–6.05%, and 33-item bytes batches lose 6.06–7.05%. These
+measurements show residual overhead for some small-input batches; they do not establish zero regression.
+
+The Rust mixed benchmarks use `[1024, 1024, 4096, 4096]`, `[257, 258, 259, 260]`, `[240, 240, 241, 241]`, and the
+reverse boundary order. The 1024/4096 case measures adjacent two-item long runs. The 257–260 case measures a
+four-item run with one shared stripe count and distinct final stripes. The 240/241 cases measure both orders across
+the short/long dispatch boundary.
 
 Use the focused one-shot run to cover the AVX2 four-chain boundaries:
 
 ```sh
 cargo bench --manifest-path benches/Cargo.toml --bench xxhash -- "xxh3_(64|128)/(240|241|512|768|1024|1536|2048|4096)/hashcodecs"
 cargo bench --manifest-path benches/Cargo.toml --bench xxhash -- "xxh3_batch/mixed/.*/hashcodecs_(64|128)"
+cargo bench --manifest-path benches/Cargo.toml --bench xxhash -- "xxh3_prepared"
 ```
 
 [![Rust XXH3 throughput](docs/benchmarks/xxh3-rust.svg)](docs/benchmarks/xxh3-rust.svg)
@@ -64,6 +91,11 @@ noisy cases insert `!` at the same boundaries. Both cases measure returned bytes
 
 [![Lenient Python Base64 throughput](docs/benchmarks/base64-python-lenient.svg)](docs/benchmarks/base64-python-lenient.svg)
 
+## Wrapped Python Base64
+
+Run `python benchmarks/python_base64.py --wrapped` with CPython 3.15 or newer. The benchmark inserts newlines after
+76 output characters and measures returned bytes and a reusable `bytearray`.
+
 ## Python Memoryview Inputs
 
 Use `--memoryview-input` for full immutable views and `--sliced-memoryview-input` for equal-length contiguous views

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/CHANGELOG.md

@@ -4,6 +4,35 @@ This file records notable user-facing changes to `hashcodecs`. Version 1.0.0 sta
 
 ## [Unreleased]
 
+## [1.4.0] - 2026-09-10
+
+### What's Changed
+* refactor: flatten internal module layout by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/91
+* refactor: prepare Base64 decoder policies once by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/92
+* refactor: consolidate binding and dispatch cleanup by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/93
+* refactor: simplify project module layout by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/94
+* refactor: prepare codec policies and unify bindings by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/95
+* refactor: accelerate Base64 codec hot paths by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/96
+* fix: protect XXH3 batches from GC reentrancy by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/97
+* refactor: unify Base64 decode execution by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/98
+* refactor: flatten Base64 binding ownership by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/99
+* refactor: improve XXH3 loads and internal naming by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/100
+* refactor: reduce Base64 decoder setup and retry costs by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/101
+* fix: preserve Base64 decode boundaries and restore ARM coverage by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/102
+* refactor: keep XXH3 AVX2 tail chains in registers by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/103
+* refactor: store AVX2 Base64 vectors in smaller groups by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/104
+* fix: reduce Python overhead and align canonical decoding by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/106
+* refactor: reduce cached AVX2 encoder setup by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/107
+* feat: optimize XXH3 seeded and batch hashing by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/108
+* Optimize AVX2 Base64 decoding by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/109
+* refactor: optimize Python batch orchestration by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/110
+* feat: optimize XXH3 batches and wrapped Base64 by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/111
+* Fuse Base64 SIMD lookup paths and Murmur3 hex output by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/112
+* fix: preserve custom alphabets and batch typing by @kozistr in https://github.com/kozistr/hashcodecs-rs/pull/113
+
+
+**Full Changelog**: https://github.com/kozistr/hashcodecs-rs/compare/v1.3.0...v1.4.0
+
 ## [1.3.0] - 2026-09-04
 
 ### What's Changed
@@ -164,7 +193,8 @@ This file records notable user-facing changes to `hashcodecs`. Version 1.0.0 sta
 - Initial Python and Rust APIs for Base64 and MurmurHash3.
 - Runtime SIMD dispatch and platform-specific CPython wheels.
 
-[Unreleased]: https://github.com/kozistr/hashcodecs-rs/compare/v1.3.0...HEAD
+[Unreleased]: https://github.com/kozistr/hashcodecs-rs/compare/v1.4.0...HEAD
+[1.4.0]: https://github.com/kozistr/hashcodecs-rs/compare/v1.3.0...v1.4.0
 [1.3.0]: https://github.com/kozistr/hashcodecs-rs/compare/v1.2.1...v1.3.0
 [1.2.1]: https://github.com/kozistr/hashcodecs-rs/compare/v1.2.0...v1.2.1
 [1.2.0]: https://github.com/kozistr/hashcodecs-rs/compare/v1.1.0...v1.2.0

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/CITATION.cff

@@ -5,8 +5,8 @@ authors:
     given-names: Hyeongchan
     orcid: https://orcid.org/0000-0002-1729-0580
 title: "hashcodecs: SIMD-accelerated Base64, MurmurHash3, and XXH3 for Python and Rust"
-version: 1.3.0
-date-released: 2026-09-04
+version: 1.4.0
+date-released: 2026-09-10
 license: "MIT OR Apache-2.0"
 repository-code: "https://github.com/kozistr/hashcodecs-rs"
 url: "https://github.com/kozistr/hashcodecs-rs"

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/Cargo.lock

@@ -194,7 +194,7 @@ dependencies = [
 
 [[package]]
 name = "hashcodecs"
-version = "1.3.0"
+version = "1.4.0"
 dependencies = [
  "base64",
  "memchr",

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "hashcodecs"
-version = "1.3.0"
+version = "1.4.0"
 edition = "2024"
 rust-version = "1.89"
 autobenches = false
@@ -11,6 +11,7 @@ keywords = ["base64", "simd", "murmur3", "xxhash", "python"]
 categories = ["encoding", "algorithms"]
 readme = "README.md"
 include = [
+    "/generated/**",
     "/src/**",
     "/benches/**",
     "/tests/sanitizers.rs",

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: hashcodecs
-Version: 1.3.0
+Version: 1.4.0
 Summary: SIMD-accelerated Base64, MurmurHash3, and xxHash codecs
 Project-URL: Documentation, https://hashcodecs-rs.readthedocs.io/
 Project-URL: Repository, https://github.com/kozistr/hashcodecs-rs
@@ -56,7 +56,7 @@ Move byte-heavy work into Rust without changing your Python inputs. `hashcodecs`
 
 - Base64 encode and decode with standard, URL-safe, padded, unpadded, wrapped, and canonical modes.
 - MurmurHash3 x86-32, x86-128, and x64-128 with one-shot and incremental APIs.
-- Bit-for-bit compatible XXH3-64 and XXH3-128 with allocating and allocation-free native batch APIs.
+- Bit-for-bit compatible XXH3-64 and XXH3-128 with prepared seeds and allocating and allocation-free native batch APIs.
 - Caller-managed `*_into` outputs for allocation-sensitive workloads.
 - Runtime dispatch across AVX-512, AVX2, SSE4.1, SSSE3, NEON, and scalar implementations where applicable.
 - Direct CPython buffer handling for `bytes`, `bytearray`, and `memoryview` inputs.
@@ -147,7 +147,18 @@ assert_eq!(
     0x2d06_8005_38d3_94c2
 );
 
+let seeded_input = vec![7; 1024];
+let prepared = hashcodecs::xxhash::PreparedXxh3::new(42);
+assert_eq!(
+    prepared.hash_64(&seeded_input),
+    hashcodecs::xxhash::xxh3_64(&seeded_input, 42)
+);
+
 let inputs: &[&[u8]] = &[b"hello", b"world"];
+assert_eq!(
+    prepared.hash_64_batch(inputs),
+    hashcodecs::xxhash::xxh3_64_batch(inputs, 42)
+);
 let mut hashes = [0_u64; 2];
 let mut index = 0;
 hashcodecs::xxhash::xxh3_64_batch_for_each(inputs, 0, |hash| {
@@ -242,9 +253,10 @@ cargo bench --manifest-path benches/Cargo.toml --bench xxhash
 
 ## Performance snapshot
 
-In the full 2026-09-02 hashcodecs-only run on the benchmark host, `hashcodecs.xxh3_64` processes a 1 MiB input at
-90.07 GiB/s. With 256 B items in batches of 64, the Base64 batch API reaches 11.84 GiB/s for encode and 6.23 GiB/s for
-decode. The run pins one logical CPU and uses 15 samples with a 0.2-second minimum per sample. Read the
+In the focused 2026-09-09 hashcodecs-only run on the benchmark host, `hashcodecs.xxh3_64` processes a 1 MiB input
+at 91.27 GiB/s. In the full 2026-09-02 run, the Base64 batch API reaches 11.84 GiB/s for encode and 6.23 GiB/s for
+decode with 256 B items in batches of 64. Each run pins one logical CPU and uses 15 samples with a 0.2-second
+minimum per sample. Read the
 [benchmark details](BENCHMARK.md) and [raw comparison results](docs/benchmarks/results.csv).
 
 ## Development
@@ -258,14 +270,12 @@ uv build
 Run the primary local checks:
 
 ```sh
-cargo fmt --check
-cargo clippy --all-targets --features python -- -D warnings
-cargo test --features python
-uv run --frozen --no-sync ruff check . --no-cache
-uv run --frozen --no-sync ruff format --check .
-uv run --frozen --no-sync pytest tests --cov=hashcodecs --cov-branch --cov-fail-under=100
+just check
 ```
 
+`just check` builds and reinstalls the current wheel before running Python tests. Run `just full-check` before
+committing to also check release builds, Rust core coverage, and the extracted source distribution.
+
 Optimized paths are also checked with differential fuzzing, Kani, strict-provenance Miri, AddressSanitizer, and
 MemorySanitizer in CI.
 

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/README.md

@@ -25,7 +25,7 @@ Move byte-heavy work into Rust without changing your Python inputs. `hashcodecs`
 
 - Base64 encode and decode with standard, URL-safe, padded, unpadded, wrapped, and canonical modes.
 - MurmurHash3 x86-32, x86-128, and x64-128 with one-shot and incremental APIs.
-- Bit-for-bit compatible XXH3-64 and XXH3-128 with allocating and allocation-free native batch APIs.
+- Bit-for-bit compatible XXH3-64 and XXH3-128 with prepared seeds and allocating and allocation-free native batch APIs.
 - Caller-managed `*_into` outputs for allocation-sensitive workloads.
 - Runtime dispatch across AVX-512, AVX2, SSE4.1, SSSE3, NEON, and scalar implementations where applicable.
 - Direct CPython buffer handling for `bytes`, `bytearray`, and `memoryview` inputs.
@@ -116,7 +116,18 @@ assert_eq!(
     0x2d06_8005_38d3_94c2
 );
 
+let seeded_input = vec![7; 1024];
+let prepared = hashcodecs::xxhash::PreparedXxh3::new(42);
+assert_eq!(
+    prepared.hash_64(&seeded_input),
+    hashcodecs::xxhash::xxh3_64(&seeded_input, 42)
+);
+
 let inputs: &[&[u8]] = &[b"hello", b"world"];
+assert_eq!(
+    prepared.hash_64_batch(inputs),
+    hashcodecs::xxhash::xxh3_64_batch(inputs, 42)
+);
 let mut hashes = [0_u64; 2];
 let mut index = 0;
 hashcodecs::xxhash::xxh3_64_batch_for_each(inputs, 0, |hash| {
@@ -211,9 +222,10 @@ cargo bench --manifest-path benches/Cargo.toml --bench xxhash
 
 ## Performance snapshot
 
-In the full 2026-09-02 hashcodecs-only run on the benchmark host, `hashcodecs.xxh3_64` processes a 1 MiB input at
-90.07 GiB/s. With 256 B items in batches of 64, the Base64 batch API reaches 11.84 GiB/s for encode and 6.23 GiB/s for
-decode. The run pins one logical CPU and uses 15 samples with a 0.2-second minimum per sample. Read the
+In the focused 2026-09-09 hashcodecs-only run on the benchmark host, `hashcodecs.xxh3_64` processes a 1 MiB input
+at 91.27 GiB/s. In the full 2026-09-02 run, the Base64 batch API reaches 11.84 GiB/s for encode and 6.23 GiB/s for
+decode with 256 B items in batches of 64. Each run pins one logical CPU and uses 15 samples with a 0.2-second
+minimum per sample. Read the
 [benchmark details](BENCHMARK.md) and [raw comparison results](docs/benchmarks/results.csv).
 
 ## Development
@@ -227,14 +239,12 @@ uv build
 Run the primary local checks:
 
 ```sh
-cargo fmt --check
-cargo clippy --all-targets --features python -- -D warnings
-cargo test --features python
-uv run --frozen --no-sync ruff check . --no-cache
-uv run --frozen --no-sync ruff format --check .
-uv run --frozen --no-sync pytest tests --cov=hashcodecs --cov-branch --cov-fail-under=100
+just check
 ```
 
+`just check` builds and reinstalls the current wheel before running Python tests. Run `just full-check` before
+committing to also check release builds, Rust core coverage, and the extracted source distribution.
+
 Optimized paths are also checked with differential fuzzing, Kani, strict-provenance Miri, AddressSanitizer, and
 MemorySanitizer in CI.
 

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/SAFETY.md

@@ -17,6 +17,15 @@ The checks are deliberately complementary:
   exact Base64 buffers, incremental MurmurHash3 states, and XXH3 batch kernels.
 - libFuzzer runs under sanitizers. Base64 is compared with `base64` 0.23.1,
   MurmurHash3 with `murmur3` 0.5, and XXH3 with `xxhash-rust` 0.8.18.
+  XXH3 cases use independent equal-length and heterogeneous buffers, with
+  distinct lane contents and batch counts through nine to cover SIMD groups and tails.
+
+The Python XXH3 batch bindings finish all input reads before allocating Python
+result containers: a GC finalizer can clear the input list or resize a bytearray
+even while the GIL is held. Up to 32 native results fit on the stack; larger
+batches use a fallible vector. Large immutable inputs keep their owners across
+GIL detachment. Subprocess tests on CPython 3.10/3.11 trigger finalizers during
+allocation and reuse freed storage, covering both sides of the stack boundary.
 
 Run the same checks locally on Linux:
 

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/benches/xxhash.rs

@@ -6,9 +6,10 @@ use criterion::{BenchmarkId, Criterion, Throughput, criterion_group, criterion_m
 
 mod support;
 
-const SIZES: [usize; 15] = [
+const SIZES: [usize; 16] = [
     16,
     17,
+    32,
     64,
     128,
     129,
@@ -23,8 +24,9 @@ const SIZES: [usize; 15] = [
     1024 * 1024,
     8 * 1024 * 1024,
 ];
-const MIXED_BATCHES: [(&str, [usize; 4]); 3] = [
+const MIXED_BATCHES: [(&str, [usize; 4]); 4] = [
     ("two_long_runs", [1024, 1024, 4 * 1024, 4 * 1024]),
+    ("nearby_long_lengths", [257, 258, 259, 260]),
     ("short_then_long_boundary", [240, 240, 241, 241]),
     ("long_then_short_boundary", [241, 241, 240, 240]),
 ];
@@ -140,7 +142,7 @@ fn benchmark_batch(c: &mut Criterion, group_name: &str, owned: &[Vec<u8>]) {
 
 fn batch(c: &mut Criterion) {
     for items in [2, 3, 32] {
-        for size in [64, 1024, 4 * 1024, 1024 * 1024] {
+        for size in [64, 241, 1024, 4 * 1024, 1024 * 1024] {
             let owned = (0..items)
                 .map(|index| data(size, index as u8))
                 .collect::<Vec<_>>();
@@ -158,10 +160,79 @@ fn batch(c: &mut Criterion) {
     }
 }
 
+fn prepared_seed(c: &mut Criterion) {
+    for size in [241, 1024] {
+        let input = data(size, 17);
+        let prepared = hashcodecs::xxhash::PreparedXxh3::new(42);
+        assert_eq!(
+            prepared.hash_64(&input),
+            hashcodecs::xxhash::xxh3_64(&input, 42)
+        );
+        assert_eq!(
+            prepared.hash_128(&input),
+            hashcodecs::xxhash::xxh3_128(&input, 42)
+        );
+
+        let mut group = c.benchmark_group(format!("xxh3_prepared/{size}"));
+        group.throughput(Throughput::Bytes(size as u64));
+        group.bench_function("one_shot_64", |bench| {
+            bench.iter(|| hashcodecs::xxhash::xxh3_64(black_box(&input), 42))
+        });
+        group.bench_function("prepared_64", |bench| {
+            bench.iter(|| prepared.hash_64(black_box(&input)))
+        });
+        group.bench_function("one_shot_128", |bench| {
+            bench.iter(|| hashcodecs::xxhash::xxh3_128(black_box(&input), 42))
+        });
+        group.bench_function("prepared_128", |bench| {
+            bench.iter(|| prepared.hash_128(black_box(&input)))
+        });
+        group.finish();
+    }
+}
+
+fn prepared_batch(c: &mut Criterion) {
+    for items in [2, 4] {
+        for size in [241, 1024] {
+            let owned = (0..items)
+                .map(|index| data(size, index as u8))
+                .collect::<Vec<_>>();
+            let inputs = owned.iter().map(Vec::as_slice).collect::<Vec<_>>();
+            let prepared = hashcodecs::xxhash::PreparedXxh3::new(42);
+            assert_eq!(
+                prepared.hash_64_batch(&inputs),
+                hashcodecs::xxhash::xxh3_64_batch(&inputs, 42)
+            );
+            assert_eq!(
+                prepared.hash_128_batch(&inputs),
+                hashcodecs::xxhash::xxh3_128_batch(&inputs, 42)
+            );
+
+            let mut group = c.benchmark_group(format!("xxh3_prepared_batch/{items}_items/{size}"));
+            group.throughput(Throughput::Bytes((items * size) as u64));
+            group.bench_function("batch_64", |bench| {
+                bench.iter(|| hashcodecs::xxhash::xxh3_64_batch(black_box(&inputs), 42))
+            });
+            group.bench_function("prepared_64", |bench| {
+                bench.iter(|| prepared.hash_64_batch(black_box(&inputs)))
+            });
+            group.bench_function("batch_128", |bench| {
+                bench.iter(|| hashcodecs::xxhash::xxh3_128_batch(black_box(&inputs), 42))
+            });
+            group.bench_function("prepared_128", |bench| {
+                bench.iter(|| prepared.hash_128_batch(black_box(&inputs)))
+            });
+            group.finish();
+        }
+    }
+}
+
 fn xxhash(c: &mut Criterion) {
     support::pin_to_one_cpu();
     one_shot(c);
     batch(c);
+    prepared_seed(c);
+    prepared_batch(c);
 }
 
 criterion_group! {

{hashcodecs-1.3.0 → hashcodecs-1.4.0}/docs/ARCHITECTURE.md

@@ -26,13 +26,17 @@ The scalar implementations set the portability and correctness baseline. Runtime
 | `src/base64.rs`, `src/base64/` | Base64 public API, operations, alphabets, output buffers, runtime dispatch, and kernels. |
 | `src/murmur3.rs`, `src/murmur3/` | MurmurHash3 public API, variants, incremental buffers, and dispatch. |
 | `src/xxhash.rs`, `src/xxhash/` | XXH3 public API, length-specific formulas, long-input accumulation, batching, and kernels. |
-| `src/bindings/mod.rs` | CPython extension composition root for public functions and classes. |
+| `src/bindings.rs` | CPython extension composition root for public functions and classes. |
 | `src/bindings/arguments.rs`, `objects.rs`, `runtime.rs` | Shared CPython parsing, object access, function registration, and GIL policy. |
 | `src/bindings/{base64,murmur3,xxhash}/` | Algorithm-specific CPython adapters. |
 | `hashcodecs/` | Typed Python facade and public module organization. |
 | `benches/`, `benchmarks/` | Rust and Python throughput measurements. |
 | `tests/`, `fuzz/` | Python compatibility tests, differential fuzzing, and safety validation. |
 
+Layout depth is counted from the relevant source root, such as `src/`, rather than from the repository root. Build,
+environment, cache, and generated-output directories—including `target/`, `.venv/`, `.uv-cache/`, `site/`, and
+`__pycache__/`—are excluded from layout conventions. Checked-in generated API metadata remains under `generated/`.
+
 ## Dependency Rules
 
 The crate groups code into layered modules within one crate:
@@ -42,7 +46,10 @@ The crate groups code into layered modules within one crate:
 - dispatch modules depend on the shared CPU capability snapshot and select interchangeable kernels;
 - architecture-specific kernels have no dependency on Python bindings;
 - algorithm-specific Python adapters depend on the Rust APIs and shared binding policies;
-- `bindings/mod.rs` is a composition root and contains no parsing, buffer, or execution policy.
+- `bindings.rs` is a composition root and contains no parsing, buffer, or execution policy.
+
+The core algorithms and feature-gated CPython bindings intentionally remain in this crate. Splitting them would
+either expose private pointer-oriented implementation APIs across a crate boundary or add copies to sensitive paths.
 
 Shared state machines own their invariants. For example, `murmur3/block_buffer.rs` keeps the pending block and its
 length together, so each incremental hasher cannot represent an inconsistent tail. At the CPython boundary,
@@ -95,34 +102,46 @@ Output allocation and failure behavior vary by API:
 - Base64 reusable-output batches stop at the first error and retain prior destination writes;
 - the XXH3 binding validates and stabilizes all packed-batch inputs before it mutates the destination.
 
+The Python Base64 binding stabilizes overlapping or free-threaded mutable input before decoding.
+A prepared policy selects the same attempt order for allocating and reusable outputs. Storage adapters own
+allocation and writes; native status values determine retries before the binding constructs Python errors.
+Direct probes write only complete validated blocks and preserve the remaining suffix for retries.
+Custom-alphabet probes validate the whole input first; strict attempts may write within a failing block.
+Strict and lenient scanners keep their distinct padding transitions and share byte-processing kernels.
+
 The Python Base64 binding sends strict input to the SIMD core without constructing a discarded Python exception.
 For lenient input, it keeps the MIME whitespace path on normalized SIMD input and decodes other ignored bytes into
 the final Python object or reusable buffer. The native state machine follows the padding behavior of each supported
 CPython patch series. It calls `binascii` for malformed input that needs CPython's exact exception.
+For older CPython padding rules, reusable-output sizing uses the decoder's SIMD alphabet-prefix scanner and
+handles padding between runs to preserve the stop at the first complete padding sequence.
 
 Large aligned x86 encoding may use non-temporal stores after the input exceeds the detected private-cache working
 set. Smaller work stays on ordinary cached stores.
+Wrapped inputs above the 4 MiB crossover send SIMD blocks to a line-aware output cursor. Blocks that fit remain
+vector stores; the cursor splits a block into quartets only when it crosses a newline boundary.
 
 ### MurmurHash3
 
 The `x86_32`, `x86_128`, and `x64_128` modules each own one-shot calls, incremental state, scalar block mixing,
-tail handling, and finalization. All variants use `incremental.rs` for pending blocks and `primitives.rs` for
+tail handling, and finalization. All variants use `block_buffer.rs` for pending blocks and `primitives.rs` for
 little-endian loads and finalizers. One-shot calls choose scalar, SSE4.1, or AVX2 using explicit size thresholds.
 
 ### XXH3
 
 `one_shot.rs` selects the input-length class. `short_inputs.rs` contains the formulas for 0 to 240 bytes.
-`long_inputs.rs` owns secret initialization, scheduling, accumulation, and merging. XXH3-64 and XXH3-128 share these modules.
-`long_inputs/aarch64.rs` and `long_inputs/x86/` contain the ISA kernels.
+`long_inputs.rs` owns secret initialization, scheduling, accumulation, and merging. `prepared.rs` retains one derived
+secret for repeated single and batch calls with the same seed. XXH3-64 and XXH3-128 share these modules.
+`long_inputs/aarch64.rs` and the kernels under `long_inputs/x86/` contain the ISA-specific implementations.
 The scalar long-input flow and backend selection use the same module. These kernels handle inputs longer than 240 bytes.
 
 The AVX2 one-shot kernel splits each full 1,024-byte block across four accumulator chains. It also splits tails
 that contain at least four stripes, including the final overlapping stripe. The kernel reduces the chains before
 each block scramble and before the final merge.
 
-Native batches reuse the initialized secret and process inputs in groups of up to four. Two to four equal-size
-inputs longer than 240 bytes use an AVX2 batch accumulator when available; single items and mixed sizes use the
-regular paths.
+Native batches reuse the initialized secret and inspect at most four inputs before hashing the group. Two to four
+equal-stripe-count inputs longer than 240 bytes use an AVX2 batch accumulator when available; single items and mixed
+sizes use the regular paths.
 Python exposes two result models:
 
 - `xxh3_*_batch` returns `list[int]` results;
@@ -165,7 +184,9 @@ writing and preserve bytes beyond the returned length.
 The typed `_hashcodecs.pyi` declaration is the canonical Python API description. It drives the public modules,
 package exports, module stubs, native text signatures and docstrings, and API-reference member lists through
 `tools/generate_api_metadata.py`. The generated `base64.py`, `murmur3.py`, and `xxhash.py` modules organize exports
-without adding per-call wrappers. `py.typed` makes the declarations visible to type checkers.
+without adding per-call wrappers. Generated Rust schemas live under `generated/rust` and are included by thin binding
+modules, so metadata generation never rewrites handwritten Rust source. `py.typed` makes the declarations visible to
+type checkers.
 
 Wheel tests execute the installed package. Coverage paths map its installed location back to the root source package.
 

hashcodecs-1.4.0/docs/requirements.txt

@@ -0,0 +1,2 @@
+mkdocstrings-python==2.0.8
+zensical==0.0.59