opencodecs 0.1.5__tar.gz → 0.1.7__tar.gz

{opencodecs-0.1.5 → opencodecs-0.1.7}/CHANGES.rst

@@ -10,6 +10,147 @@ Versions follow the same ``YYYY.M.D`` cadence as upstream when we
 publish; the entries below cluster work by date rather than by
 release because most of it has shipped continuously to ``main``.
 
+0.1.7 (2026-06-02)
+------------------
+
+**Patched libultrahdr bundle**
+
+Since v0.1.6 we build libuhdr ourselves; v0.1.7 starts carrying
+local patches via ``patches/libultrahdr/*.patch``, applied by
+``bench/build_codec_libs.sh::build_libultrahdr`` after the source
+fetch. Five upstream post-v1.4.0 cherry-picks land first:
+
+* ``5ed39d6`` — fix ``CLIP3`` parameter order in libuhdr's own
+  ``applyGainMap``. The bug clamped a constant ``0.0f`` instead of
+  the gainmap weight whenever ``display_boost ≠ hdr_capacity_max``,
+  silently producing wrong HDR output on libuhdr's wrapped
+  ``decode()`` path. ``opencodecs.uhdr.decode_native`` is unaffected
+  (it uses our own Cython gain-application kernel).
+* ``7088ca7`` — error-message typo in ``jpegr.cpp``.
+* ``13a058f`` — ``icc.h`` Endian_Swap macros now respect actual
+  host endianness instead of an unconditionally-true ``USE_BIG_ENDIAN_IN_ICC``.
+  No effect on x86_64/aarch64 (little-endian); fixes PowerPC/s390x.
+* ``5fa99b5`` — add missing ``<cstdint>`` include for GCC 15.
+* ``8cbc983`` — same ``CLIP3`` fix as ``5ed39d6`` in the GPU path
+  (we don't link the GPU path; carried for hygiene).
+
+**New API: ``opencodecs.uhdr.probe(data) -> dict``**
+
+Parse an Ultra-HDR container's MPF metadata without any pixel
+decode. Returns base + gainmap dimensions plus the gainmap
+metadata block (``max_content_boost`` / ``min_content_boost`` /
+``gamma`` / capacity). Wraps libuhdr's existing ``uhdr_dec_probe``
++ accessors. ~180× faster than ``decode()`` for any HDR-aware
+flow that only needs dimensions or capacity — image indexing,
+thumbnail generation, HTTP HEAD-style inspection, routing batches
+by content-boost.
+
+**Gain-map tunables on ``encode_native``: ``gain_quality`` + ``gain_scale``**
+
+Two new kwargs let callers shrink the gain-map layer
+independently of the SDR base:
+
+* ``gain_quality`` — JPEG quality for the gain-map layer only
+  (defaults to track ``quality``). The gain map is heavily
+  band-limited so ``gain_quality=70`` is visually equivalent to
+  ``q95`` and cuts ~30% off the gain-map bytes.
+* ``gain_scale`` — integer power-of-2 downsample factor for the
+  gain-map raster (``1`` keeps full resolution, ``2`` halves both
+  axes for quarter-area, etc.). Stride decimation; ~5 ms on a 2k²
+  uint8 gain map. Round-trips through ``probe`` / ``decode_native``
+  cleanly — the container records the actual gain-map dimensions
+  and the decoder upscales on apply.
+
+**True streaming encode: ``encode_native(..., out=fp)``**
+
+The old ``encode_to(fp, hdr, ...)`` was a forward-compatible
+``fp.write(encode_native(hdr, **kw))`` wrapper. v0.1.7 makes both
+genuinely streaming: ``encode_assembled`` (and by extension
+``encode_native`` / ``encode_to``) now accept an ``out=`` file-
+like, and when given they hand a zero-copy ``memoryview`` over
+libuhdr's internal output buffer to ``out.write()``. Skips the
+final ``PyBytes_FromStringAndSize`` allocation + memcpy. Saves
+~1× output-size peak memory and ~3 ms wall-clock on a 5 MB encode.
+``encode_to`` now returns ``None`` instead of the byte count.
+
+0.1.6 (2026-06-01)
+------------------
+
+**libultrahdr now bundled (fixes missing ``_uhdr`` on PyPI wheels)**
+
+* ``bench/build_codec_libs.sh::build_libultrahdr`` builds libuhdr
+  v1.4.0 (Apache-2.0) into ``$OPENCODECS_LIBS_PREFIX/{include,lib}``
+  alongside the other source-built codec libs. The Linux + Windows
+  CI ``--only=...`` lists pick it up; the macOS ``before-all`` brew
+  install now includes ``libultrahdr`` as well.
+* setup.py's ``_uhdr`` Extension is now built by a probe
+  (``_maybe_build_uhdr_ext``) that finds the cached prefix, sets
+  ``library_dirs``, and bakes the rpath into the resulting ``.so``
+  so delocate / auditwheel / delvewheel can bundle ``libuhdr``'s
+  dylib/.so/.dll into the wheel.
+* ``_uhdr`` is now in ``MUST_SHIP_ALL_PLATFORMS`` in
+  ``ci/check_wheel_contents.py``; a wheel without the Ultra-HDR
+  extension fails the wheel-coverage step.
+* The stale ``_ultrahdr.pyx`` source file (orphaned in commit
+  ``c9347f5`` when the direct libuhdr binding shipped as ``_uhdr``)
+  is removed.
+
+**New API: ``opencodecs.uhdr.decode_native``**
+
+Fused-Cython fast-path Ultra-HDR decoder. Uses libuhdr's parser to
+pull out the compressed SDR base + gain-map JPEGs + metadata (no
+pixel decode), then decodes both JPEGs in parallel via
+``imagecodecs.jpeg_decode`` (libjpeg-turbo SIMD, GIL released) and
+applies the gain map in a Cython kernel that uses the same sRGB
+EOTF LUT + IEEE-754 polynomial ``exp2`` the encoder uses. ~1.4×
+faster than libuhdr's reference decode on a 2k² float HDR (M-series
+Mac: ~52 ms vs ~72 ms). Output matches libuhdr's decode to within
+JPEG-q95 + 8-bit gain-quantisation noise. ``display_boost`` kwarg
+exposes the ISO 21496-1 headroom scaler — default is full HDR
+(``hdr_capacity_max``), pass ``1.0`` to match libuhdr's default
+SDR-equivalent decode.
+
+Backed by two new Cython helpers exposed for advanced callers:
+
+* ``opencodecs.codecs._uhdr.extract_layers(data)`` — parse the
+  container, return ``{base_jpeg, gainmap_jpeg, gainmap_metadata,
+  width, height, gainmap_{width,height}}`` without decoding pixels.
+* ``opencodecs.codecs._uhdr.apply_gainmap_fp32(sdr_u8, gain_u8,
+  metadata, display_boost=...)`` — the per-pixel gain-application
+  kernel.
+
+**New API: ``opencodecs.uhdr.encode_to``**
+
+Streaming variant of :func:`encode_native` — writes Ultra-HDR bytes
+directly to a file-like (anything with ``write(bytes)``: open file,
+``io.BytesIO``, HTTP upload streamer). Returns the byte count.
+Forward-compatible alias: the libuhdr api-4 path we currently use
+for container assembly doesn't expose a streaming writer, so the
+function is ``fp.write(encode_native(...))`` today; the API exists
+so callers can adopt it now and pick up any future libuhdr
+streaming write-out without changing their code.
+
+**CI / build robustness**
+
+* ``_uhdr`` extension's ``-ffast-math`` is now narrowed to
+  ``-ffast-math -fno-finite-math-only
+  -fno-unsafe-math-optimizations -fno-math-errno
+  -fno-trapping-math``. The ``-funsafe-math-optimizations``
+  sub-flag is what tells GCC to replace libm calls with libmvec's
+  vectorised ``_ZGV*`` variants, which link-fail on Ubuntu (Ubuntu's
+  default gcc doesn't auto-link libmvec like manylinux_2_28 does)
+  and aren't available at all on aarch64. Same flag set the edt
+  extension uses for the same reason. Keeps FMA + reordering
+  perf; eliminates the libmvec dependency entirely.
+* The libaec source URL moved from ``gitlab.dkrz.de`` (now
+  auth-gated; anonymous requests redirect to ``/users/sign_in``)
+  to DKRZ's GitHub mirror at
+  ``github.com/Deutsches-Klimarechenzentrum/libaec``.
+* ``fetch_tar`` now has a 4-attempt shell retry loop with empty-
+  extract detection — covers transient mirror outages that
+  curl's own ``--retry`` doesn't see (e.g. 200 OK with a
+  truncated body).
+
 0.1.5 (2026-05-31)
 ------------------
 

{opencodecs-0.1.5/src/opencodecs.egg-info → opencodecs-0.1.7}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: opencodecs
-Version: 0.1.5
+Version: 0.1.7
 Summary: Streaming, network-aware image codecs for scientific imaging (prototype: JPEG XL)
 Author: Kevin Cutler
 License: BSD-3-Clause

{opencodecs-0.1.5 → opencodecs-0.1.7}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "opencodecs"
-version = "0.1.5"
+version = "0.1.7"
 description = "Streaming, network-aware image codecs for scientific imaging (prototype: JPEG XL)"
 readme = "README.md"
 requires-python = ">=3.10"
@@ -142,7 +142,7 @@ before-all = [
     # bench/build_codec_libs.sh into the cached /cibw-jxl-prefix mount.
     # Skips encoders (aom, x265) — wheel ships decode-only AVIF/HEIF to
     # keep build < 12 min cold + avoid GPL x265.
-    "bash -c 'PATH=$HOME/.cargo/bin:$PATH bash bench/build_codec_libs.sh --only=libjpeg-turbo,c-blosc2,libde265,dav1d,libavif,libheif,libaec,lerc,zfp,SZ3,pcodec,SPERR,brunsli,mozjpeg'",
+    "bash -c 'PATH=$HOME/.cargo/bin:$PATH bash bench/build_codec_libs.sh --only=libjpeg-turbo,c-blosc2,libde265,dav1d,libavif,libheif,libaec,lerc,zfp,SZ3,pcodec,SPERR,brunsli,mozjpeg,libultrahdr'",
     "bash bench/build_libjxl.sh",
 ]
 # Don't set OPENCODECS_*_PREFIX here — build_wheels.yml's
@@ -162,7 +162,7 @@ before-all = [
     # mozjpeg removed from this brew install — the formula ships libjpeg
     # only, not libturbojpeg, so the _mozjpeg probe couldn't find what it
     # needed. Source-build via build_codec_libs.sh produces both libs.
-    "brew install jpeg-xl libavif libheif webp libpng jpeg-turbo openjpeg zstd brotli c-blosc2 lz4 libdeflate libaec liblerc zfp rust cmake ninja",
+    "brew install jpeg-xl libavif libheif webp libpng jpeg-turbo openjpeg zstd brotli c-blosc2 lz4 libdeflate libaec liblerc zfp rust cmake ninja libultrahdr",
     "bash bench/build_codec_libs.sh --only=SZ3,pcodec,SPERR,brunsli,mozjpeg",
 ]
 # macos-latest is now macOS 15 (Sequoia) on arm64. Homebrew formulas are

{opencodecs-0.1.5 → opencodecs-0.1.7}/setup.py

@@ -626,6 +626,123 @@ def _maybe_build_ext_simple(
     return []
 
 
+def _maybe_build_uhdr_ext() -> list[Extension]:
+    """Build the ``_uhdr`` (libultrahdr / Ultra-HDR JPEG) extension when
+    libuhdr is on the system. Probes the cibuildwheel cached prefix,
+    the per-user opencodecs cache (from ``build_codec_libs.sh``),
+    homebrew's keg, and the standard system prefixes. Bakes an rpath
+    into the resulting ``.so`` so the linked ``libuhdr`` dylib is
+    findable at import time without ``LD_LIBRARY_PATH``.
+
+    Apache-2.0; bundled into the wheel by the standard
+    delocate/auditwheel/delvewheel repair step.
+    """
+    candidates: list[Path] = []
+    # 1. Explicit codec-libs prefix wins (set in CI to /cibw-jxl-prefix
+    #    on Linux, $CONDA_PREFIX on Windows).
+    _libs_prefix_env = os.environ.get("OPENCODECS_CODEC_LIBS_PREFIX")
+    if _libs_prefix_env:
+        candidates.append(Path(_libs_prefix_env))
+        # Conda's Windows layout puts headers under <prefix>/Library/.
+        candidates.append(Path(_libs_prefix_env) / "Library")
+    # 2. Standalone CMake-built libuhdr from build_codec_libs.sh on a
+    #    dev machine (installs into the per-user `libs` cache root).
+    candidates.append(_OC_USER_CACHE / "libs")
+    # 3. Homebrew keg (macOS).
+    candidates.extend([
+        Path("/opt/homebrew/opt/libultrahdr"),
+        Path("/usr/local/opt/libultrahdr"),
+    ])
+    # 4. Generic system prefixes.
+    candidates.extend([Path("/usr/local"), Path("/usr")])
+
+    prefix = None
+    lib_subdir = None  # "lib" or "lib64" or "Library/lib"
+    lib_filename = None
+    lib_candidates = [
+        ("lib", "libuhdr.dylib"),
+        ("lib", "libuhdr.so"),
+        ("lib", "libuhdr.so.1"),
+        ("lib64", "libuhdr.so"),
+        ("lib64", "libuhdr.so.1"),
+        ("lib", "libuhdr.1.dylib"),
+        # Windows (conda-forge / build_codec_libs.sh layout): the
+        # CMake build emits uhdr.lib alongside uhdr.dll under bin/.
+        ("lib", "uhdr.lib"),
+        ("Library/lib", "uhdr.lib"),
+    ]
+    for c in candidates:
+        if not (c / "include" / "ultrahdr_api.h").is_file() \
+                and not (c / "Library" / "include" / "ultrahdr_api.h").is_file():
+            continue
+        # If Library/ subdir holds the headers, that's the real prefix.
+        if (c / "Library" / "include" / "ultrahdr_api.h").is_file():
+            c = c / "Library"
+        for subdir, name in lib_candidates:
+            if (c / subdir / name).is_file():
+                prefix = c
+                lib_subdir = subdir
+                lib_filename = name
+                break
+        if prefix is not None:
+            break
+    if prefix is None:
+        return []
+
+    extra_link_args: list[str] = []
+    if sys.platform == "darwin":
+        # Absolute-path link + rpath fallback. libuhdr's
+        # install_name is @rpath/libuhdr.X.dylib, so delocate needs
+        # the rpath to be present to resolve LC_LOAD_DYLIB.
+        extra_link_args = [
+            str(prefix / lib_subdir / lib_filename),
+            f"-Wl,-rpath,{prefix / lib_subdir}",
+        ]
+        libraries = []
+    else:
+        libraries = ["uhdr"]
+        if sys.platform == "linux":
+            extra_link_args = [f"-Wl,-rpath,{prefix / lib_subdir}"]
+
+    return [Extension(
+        name="opencodecs.codecs._uhdr",
+        sources=["src/opencodecs/codecs/_uhdr.pyx"],
+        include_dirs=[
+            str(PKG_CODECS),
+            numpy.get_include(),
+            str(prefix / "include"),
+        ],
+        library_dirs=[str(prefix / lib_subdir)],
+        libraries=libraries,
+        extra_link_args=extra_link_args,
+        define_macros=[("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION")],
+        # Aggressive optimisation for the gain-map / SDR-base kernels
+        # to auto-vectorise (NEON on Apple Silicon, AVX2 on x86). The
+        # polynomial log2 in _gain_map_kernel only beats numpy's
+        # vectorised log2 when the compiler turns it into SIMD.
+        #
+        # The fno-finite-math-only / fno-unsafe-math-optimizations pair
+        # subtracts back the two sub-flags of -ffast-math that bite us:
+        # ``-funsafe-math-optimizations`` is what tells GCC to replace
+        # libm calls with libmvec's vectorised ``_ZGV*`` variants, which
+        # link-fail on Ubuntu and aren't present at all on aarch64.
+        # ``-ffinite-math-only`` breaks our clip-to-[0,1] paths by
+        # assuming no NaN/Inf can appear. Same flag set that the edt
+        # extension uses for the same reason — keeps FMA + reordering
+        # speed without the libmvec dependency.
+        extra_compile_args=(
+            [
+                "-O3", "-ffast-math",
+                "-fno-finite-math-only",
+                "-fno-unsafe-math-optimizations",
+                "-fno-math-errno", "-fno-trapping-math",
+            ]
+            if sys.platform != "win32" else ["/O2"]
+        ),
+        language="c",
+    )]
+
+
 def _maybe_build_mozjpeg_ext() -> list[Extension]:
     """Build the optional ``_mozjpeg`` extension when MozJPEG is
     installed. MozJPEG ships its libturbojpeg under a keg-only prefix
@@ -1125,36 +1242,14 @@ extensions = [
         extra_link_args=extra_link_args,
         language="c",
     ),
-    # libultrahdr (Ultra-HDR / ISO 21496-1). Links against system libuhdr.
-    # macOS: `brew install libultrahdr` -> /opt/homebrew/{include,lib}.
-    # Linux: build from https://github.com/google/libultrahdr.
-    # If the header/dylib aren't present, the Extension still appears in
-    # the wheel build plan -- the public opencodecs.uhdr module catches
-    # ImportError and substitutes a clear "install libultrahdr" stub.
-    Extension(
-        name="opencodecs.codecs._uhdr",
-        sources=["src/opencodecs/codecs/_uhdr.pyx"],
-        include_dirs=[
-            str(PKG_CODECS),
-            numpy.get_include(),
-            *_resolve_include_dirs("ultrahdr_api.h"),
-        ],
-        library_dirs=[],
-        libraries=["uhdr"],
-        define_macros=[
-            ("NPY_NO_DEPRECATED_API", "NPY_1_7_API_VERSION"),
-        ],
-        # Aggressive optimisation is critical for the gain-map kernel
-        # to auto-vectorise (NEON on Apple Silicon, AVX2 on x86). The
-        # polynomial log2 in _gain_map_kernel only beats numpy's
-        # vectorised log2 when the compiler turns it into SIMD.
-        extra_compile_args=[
-            "-O3",
-            "-ffast-math",  # fine for gain-map output (uint8, ~1% tolerance)
-            "-fno-math-errno",
-        ],
-        language="c",
-    ),
+    # libultrahdr (Ultra-HDR / ISO 21496-1). Bundled via
+    # bench/build_codec_libs.sh::build_libultrahdr (Apache-2.0). The
+    # _maybe_build_uhdr_ext probe finds the cached install prefix
+    # (homebrew keg, $OPENCODECS_LIBS_PREFIX, per-user cache, or the
+    # cibuildwheel mount) and bakes in the matching rpath so delocate /
+    # auditwheel / delvewheel can bundle libuhdr's dylib/.so/.dll into
+    # the wheel.
+    *_maybe_build_uhdr_ext(),
     # QOI: vendored single-header (3rdparty/qoi/qoi.h). Compile the impl
     # via QOI_IMPLEMENTATION; no external library needed.
     Extension(
@@ -1410,23 +1505,10 @@ extensions = [
         )]
         if _has_header("isa-l/igzip_lib.h") else []
     ),
-    # Ultra HDR (gainmap JPEG, ISO 21496) via Google's libultrahdr.
-    # Optional; the prebuilt brew bottle is fine because the codec is
-    # bottlenecked by libjpeg-turbo's SIMD encoders (which we tune
-    # separately). depends on libjpeg-turbo, no C++ runtime visible at
-    # our ABI.
-    *_maybe_build_ext_simple(
-        name="opencodecs.codecs._ultrahdr",
-        source="src/opencodecs/codecs/_ultrahdr.pyx",
-        prefixes=[
-            str(_OC_USER_CACHE / "libs"),
-            "/opt/homebrew/opt/libultrahdr",
-            "/usr/local/opt/libultrahdr",
-            "/usr/local", "/usr",
-        ],
-        probe_header="ultrahdr_api.h",
-        libname="uhdr",
-    ),
+    # Ultra HDR is now wired via _maybe_build_uhdr_ext() above (see
+    # the `_uhdr` extension probe). The old stale `_ultrahdr` block
+    # was orphaned when commit c9347f5 replaced the Python wrapper
+    # with the direct libuhdr binding.
     # EER (Thermo Fisher Electron Event Representation) — cryo-EM
     # event-list decoder, vendored from imagecodecs imcd.c (BSD-3).
     # No external deps; always built.

{opencodecs-0.1.5 → opencodecs-0.1.7}/src/opencodecs/__init__.py

@@ -191,4 +191,4 @@ __all__ = [
     "CziWriter", "CziPyramidWriter",
 ]
 
-__version__ = "0.1.5"
+__version__ = "0.1.7"