usearch 2.21.3 → 2.25.0

package/numkong/include/numkong/scalar/README.md

@@ -0,0 +1,89 @@
+# Scalar Math Primitives in NumKong
+
+NumKong provides single-element math operations — square root, reciprocal square root, fused multiply-add, and saturating integer arithmetic — with per-ISA implementations.
+These primitives serve as building blocks for vectorized kernels: distance finalizers call `nk_f32_rsqrt` for angular normalization, packing routines call `nk_f32_sqrt` for norm computation.
+Ordering functions (`nk_f16_order`, `nk_bf16_order`, `nk_e4m3_order`) convert floating-point bit patterns to integers that sort in the same order as the original floats.
+
+Reciprocal square root:
+
+$$
+\text{rsqrt}(x) = \frac{1}{\sqrt{x}}
+$$
+
+Fused multiply-add:
+
+$$
+\text{fma}(a, b, c) = a \cdot b + c
+$$
+
+Saturating addition:
+
+$$
+\text{sat\_add}(a, b) = \text{clamp}(a + b, \text{T\_MIN}, \text{T\_MAX})
+$$
+
+Reformulating as Python pseudocode:
+
+```python
+import numpy as np
+
+def rsqrt(x: float) -> float:
+    return 1.0 / np.sqrt(x)
+
+def fma(a: float, b: float, c: float) -> float:
+    return a * b + c
+
+def saturating_add(a: int, b: int, bits: int, signed: bool) -> int:
+    lo, hi = (-(1 << (bits-1)), (1 << (bits-1)) - 1) if signed else (0, (1 << bits) - 1)
+    return max(lo, min(a + b, hi))
+```
+
+## Input & Output Types
+
+| Input Type | Output Type | Description                                     |
+| ---------- | ----------- | ----------------------------------------------- |
+| `f64`      | `f64`       | sqrt, rsqrt, fma for 64-bit doubles             |
+| `f32`      | `f32`       | sqrt, rsqrt, fma for 32-bit floats              |
+| `f16`      | `f16`       | sqrt, rsqrt, fma for 16-bit halfs               |
+| `i8`       | `i8`        | Saturating add and multiply                     |
+| `u8`       | `u8`        | Saturating add and multiply                     |
+| `i16`      | `i16`       | Saturating add and multiply                     |
+| `u16`      | `u16`       | Saturating add and multiply                     |
+| `i32`      | `i32`       | Saturating add and multiply                     |
+| `u32`      | `u32`       | Saturating add and multiply                     |
+| `i64`      | `i64`       | Saturating add and multiply                     |
+| `u64`      | `u64`       | Saturating add and multiply                     |
+| `i4x2`     | `i4x2`      | Saturating add for packed signed nibble pairs   |
+| `u4x2`     | `u4x2`      | Saturating add for packed unsigned nibble pairs |
+| `f16`      | `u16`       | Ordering: bit pattern to sortable integer       |
+| `bf16`     | `u16`       | Ordering: bit pattern to sortable integer       |
+| `e4m3`     | `u8`        | Ordering: bit pattern to sortable integer       |
+| `e5m2`     | `u8`        | Ordering: bit pattern to sortable integer       |
+| `e2m3`     | `u8`        | Ordering: bit pattern to sortable integer       |
+| `e3m2`     | `u8`        | Ordering: bit pattern to sortable integer       |
+
+## Optimizations
+
+### Quake 3 Fast Inverse Square Root
+
+`nk_f32_rsqrt_serial` uses the classic bit-manipulation trick: reinterpret Float32 bits as Int32, compute `0x5F375A86 - (bits >> 1)`, reinterpret back to Float32, then refine with 3 Newton-Raphson iterations reaching ~34.9 correct bits.
+Each Newton-Raphson iteration: `y = y * (1.5f - 0.5f * x * y * y)` — 2 multiplies and 1 subtract, ~4cy per iteration.
+`nk_f32_rsqrt_haswell` replaces this with hardware `VRSQRT14PS` ($2^{-14}$ relative error, ~4cy latency) plus one Newton-Raphson refinement (~22-24 correct bits).
+`nk_f64_rsqrt_serial` uses the Float64 magic constant `0x5FE6EB50C7B537A9` with 4 iterations for 52-bit mantissa coverage.
+
+### Dekker Error-Free Multiplication for FMA
+
+`nk_f32_fma_serial` emulates fused multiply-add on platforms without hardware FMA using Dekker's algorithm: splits each operand into high and low halves via `a_hi = (a * 134217729.0f) - ((a * 134217729.0f) - a)`, then computes the exact product error term.
+The magic constant $134217729 = 2^{27} + 1$ splits a 24-bit mantissa into two 12-bit halves that multiply without rounding.
+`nk_f32_fma_haswell` uses hardware `VFMADD231SS` — single instruction, single cycle, exact to the last bit.
+
+### Float-to-Integer Ordering
+
+`nk_f16_order_serial`, `nk_bf16_order_serial`, `nk_e4m3_order_serial` convert floating-point bit patterns to unsigned integers that preserve the total order.
+Positive floats are already ordered by their bit patterns; negative floats need bit inversion: `if (bits & sign_bit) bits = ~bits; else bits ^= sign_bit`.
+This enables integer comparison instructions (`VPCMPUD`) for floating-point sorting without branching — used by `nk_reduce_minmax_*` for Float8 and sub-32-bit types that lack native SIMD comparison.
+
+## Performance
+
+Scalar primitives operate on single elements and are not independently benchmarked.
+Their performance is captured within the vector kernels that call them.

package/numkong/include/numkong/scalar/haswell.h

@@ -0,0 +1,122 @@
+/**
+ *  @brief SIMD-accelerated Scalar Math Helpers for Haswell.
+ *  @file include/numkong/scalar/haswell.h
+ *  @author Ash Vardanian
+ *  @date March 1, 2026
+ *
+ *  @sa include/numkong/scalar.h
+ *
+ *  @section scalars_haswell_instructions Key AVX2/FMA Scalar Instructions
+ *
+ *      Intrinsic     Instruction               Haswell    Genoa
+ *      _mm_sqrt_ps   VSQRTPS (XMM, XMM)        11cy @ p0  15cy @ p01
+ *      _mm_sqrt_pd   VSQRTPD (XMM, XMM)        16cy @ p0  15cy @ p01
+ *      _mm_fmadd_ss  VFMADD (XMM, XMM, XMM)    5cy @ p01  4cy @ p01
+ *      _mm_fmadd_sd  VFMADD (XMM, XMM, XMM)    5cy @ p01  4cy @ p01
+ *      _mm_cvtps_ph  VCVTPS2PH (XMM, XMM, I8)  5cy @ p01  4cy @ p12+p23
+ *      _mm_cvtph_ps  VCVTPH2PS (XMM, XMM)      5cy @ p01  4cy @ p12+p23
+ */
+#ifndef NK_SCALAR_HASWELL_H
+#define NK_SCALAR_HASWELL_H
+
+#if NK_TARGET_X8664_
+#if NK_TARGET_HASWELL
+
+#include "numkong/types.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+#if defined(__clang__)
+#pragma clang attribute push(__attribute__((target("avx2,f16c,fma,bmi,bmi2"))), apply_to = function)
+#elif defined(__GNUC__)
+#pragma GCC push_options
+#pragma GCC target("avx2", "f16c", "fma", "bmi", "bmi2")
+#endif
+
+NK_PUBLIC nk_f32_t nk_f32_sqrt_haswell(nk_f32_t x) { return _mm_cvtss_f32(_mm_sqrt_ps(_mm_set_ss(x))); }
+NK_PUBLIC nk_f64_t nk_f64_sqrt_haswell(nk_f64_t x) { return _mm_cvtsd_f64(_mm_sqrt_pd(_mm_set_sd(x))); }
+NK_PUBLIC nk_f32_t nk_f32_rsqrt_haswell(nk_f32_t x) {
+    __m128 x_f32x4 = _mm_set_ss(x);
+    __m128 estimate_f32x4 = _mm_rsqrt_ss(x_f32x4);
+    __m128 refinement_f32x4 = _mm_mul_ss(_mm_mul_ss(x_f32x4, estimate_f32x4), estimate_f32x4);
+    refinement_f32x4 = _mm_sub_ss(_mm_set_ss(3.0f), refinement_f32x4);
+    return _mm_cvtss_f32(_mm_mul_ss(_mm_mul_ss(_mm_set_ss(0.5f), estimate_f32x4), refinement_f32x4));
+}
+NK_PUBLIC nk_f64_t nk_f64_rsqrt_haswell(nk_f64_t x) { return 1.0 / nk_f64_sqrt_haswell(x); }
+NK_PUBLIC nk_f32_t nk_f32_fma_haswell(nk_f32_t a, nk_f32_t b, nk_f32_t c) {
+    return _mm_cvtss_f32(_mm_fmadd_ss(_mm_set_ss(a), _mm_set_ss(b), _mm_set_ss(c)));
+}
+NK_PUBLIC nk_f64_t nk_f64_fma_haswell(nk_f64_t a, nk_f64_t b, nk_f64_t c) {
+    return _mm_cvtsd_f64(_mm_fmadd_sd(_mm_set_sd(a), _mm_set_sd(b), _mm_set_sd(c)));
+}
+NK_PUBLIC nk_f16_t nk_f16_sqrt_haswell(nk_f16_t x) {
+    nk_fui16_t x_fui, out_fui;
+    x_fui.f = x;
+    __m128 x_f32x4 = _mm_cvtph_ps(_mm_cvtsi32_si128(x_fui.u));
+    out_fui.u = (nk_u16_t)_mm_cvtsi128_si32(_mm_cvtps_ph(_mm_sqrt_ps(x_f32x4), _MM_FROUND_TO_NEAREST_INT));
+    return out_fui.f;
+}
+NK_PUBLIC nk_f16_t nk_f16_rsqrt_haswell(nk_f16_t x) {
+    nk_fui16_t x_fui, out_fui;
+    x_fui.f = x;
+    __m128 x_f32x4 = _mm_cvtph_ps(_mm_cvtsi32_si128(x_fui.u));
+    __m128 estimate_f32x4 = _mm_rsqrt_ss(x_f32x4);
+    __m128 refinement_f32x4 = _mm_mul_ss(_mm_mul_ss(x_f32x4, estimate_f32x4), estimate_f32x4);
+    refinement_f32x4 = _mm_sub_ss(_mm_set_ss(3.0f), refinement_f32x4);
+    estimate_f32x4 = _mm_mul_ss(_mm_mul_ss(_mm_set_ss(0.5f), estimate_f32x4), refinement_f32x4);
+    out_fui.u = (nk_u16_t)_mm_cvtsi128_si32(_mm_cvtps_ph(estimate_f32x4, _MM_FROUND_TO_NEAREST_INT));
+    return out_fui.f;
+}
+NK_PUBLIC nk_f16_t nk_f16_fma_haswell(nk_f16_t a, nk_f16_t b, nk_f16_t c) {
+    nk_fui16_t a_fui, b_fui, c_fui, out_fui;
+    a_fui.f = a, b_fui.f = b, c_fui.f = c;
+    __m128 a_f32x4 = _mm_cvtph_ps(_mm_cvtsi32_si128(a_fui.u));
+    __m128 b_f32x4 = _mm_cvtph_ps(_mm_cvtsi32_si128(b_fui.u));
+    __m128 c_f32x4 = _mm_cvtph_ps(_mm_cvtsi32_si128(c_fui.u));
+    out_fui.u = (nk_u16_t)_mm_cvtsi128_si32(
+        _mm_cvtps_ph(_mm_fmadd_ss(a_f32x4, b_f32x4, c_f32x4), _MM_FROUND_TO_NEAREST_INT));
+    return out_fui.f;
+}
+NK_PUBLIC nk_u8_t nk_u8_saturating_add_haswell(nk_u8_t a, nk_u8_t b) {
+    return (nk_u8_t)_mm_cvtsi128_si32(_mm_adds_epu8(_mm_cvtsi32_si128(a), _mm_cvtsi32_si128(b)));
+}
+NK_PUBLIC nk_i8_t nk_i8_saturating_add_haswell(nk_i8_t a, nk_i8_t b) {
+    return (nk_i8_t)_mm_cvtsi128_si32(_mm_adds_epi8(_mm_cvtsi32_si128(a), _mm_cvtsi32_si128(b)));
+}
+NK_PUBLIC nk_u16_t nk_u16_saturating_add_haswell(nk_u16_t a, nk_u16_t b) {
+    return (nk_u16_t)_mm_cvtsi128_si32(_mm_adds_epu16(_mm_cvtsi32_si128(a), _mm_cvtsi32_si128(b)));
+}
+NK_PUBLIC nk_i16_t nk_i16_saturating_add_haswell(nk_i16_t a, nk_i16_t b) {
+    return (nk_i16_t)_mm_cvtsi128_si32(_mm_adds_epi16(_mm_cvtsi32_si128(a), _mm_cvtsi32_si128(b)));
+}
+NK_PUBLIC nk_u64_t nk_u64_saturating_mul_haswell(nk_u64_t a, nk_u64_t b) {
+    unsigned long long high;
+    unsigned long long low = _mulx_u64(a, b, &high);
+    return high ? 18446744073709551615ull : low;
+}
+NK_PUBLIC nk_i64_t nk_i64_saturating_mul_haswell(nk_i64_t a, nk_i64_t b) {
+    int sign = (a < 0) ^ (b < 0);
+    nk_u64_t abs_a = a < 0 ? (0u - (nk_u64_t)a) : (nk_u64_t)a;
+    nk_u64_t abs_b = b < 0 ? (0u - (nk_u64_t)b) : (nk_u64_t)b;
+    unsigned long long high;
+    unsigned long long low = _mulx_u64(abs_a, abs_b, &high);
+    if (high || (sign && low > 9223372036854775808ull) || (!sign && low > 9223372036854775807ull))
+        return sign ? (-9223372036854775807ll - 1ll) : 9223372036854775807ll;
+    return sign ? -(nk_i64_t)low : (nk_i64_t)low;
+}
+
+#if defined(__clang__)
+#pragma clang attribute pop
+#elif defined(__GNUC__)
+#pragma GCC pop_options
+#endif
+
+#if defined(__cplusplus)
+} // extern "C"
+#endif
+
+#endif // NK_TARGET_HASWELL
+#endif // NK_TARGET_X8664_
+#endif // NK_SCALAR_HASWELL_H

package/numkong/include/numkong/scalar/loongsonasx.h

@@ -0,0 +1,74 @@
+/**
+ *  @brief SIMD-accelerated Scalar Math Helpers for LoongArch LASX.
+ *  @file include/numkong/scalar/loongsonasx.h
+ *  @author Ash Vardanian
+ *  @date March 23, 2026
+ *
+ *  @sa include/numkong/scalar.h
+ *
+ *  LASX provides `xvfrsqrt` (full-precision reciprocal sqrt) and `xvfsqrt`
+ *  (full-precision sqrt). No Newton-Raphson refinement needed.
+ *  Full-precision sqrt uses the hardware `xvfsqrt` instruction.
+ *  Broadcast via `xvreplgr2vr`, extract via `xvpickve2gr` — no memory round-trips.
+ */
+#ifndef NK_SCALAR_LOONGSONASX_H
+#define NK_SCALAR_LOONGSONASX_H
+
+#if NK_TARGET_LOONGARCH64_
+#if NK_TARGET_LOONGSONASX
+
+#include "numkong/types.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+/** @brief Broadcast f32 scalar into all 4 lanes of a 128-bit register (GCC/Clang portable). */
+NK_INTERNAL __m128 nk_xvreplgr2vr_s_128_(float x) {
+    nk_fui32_t c;
+    c.f = x;
+    return (__m128)__lsx_vreplgr2vr_w((int)c.u);
+}
+
+/** @brief Broadcast f32 scalar into all 8 lanes of a 256-bit register (GCC/Clang portable). */
+NK_INTERNAL __m256 nk_xvfreplgr2vr_s_(float x) {
+    nk_fui32_t c;
+    c.f = x;
+    return (__m256)__lasx_xvreplgr2vr_w((int)c.u);
+}
+
+/** @brief Broadcast f64 scalar into all 4 lanes of a 256-bit register (GCC/Clang portable). */
+NK_INTERNAL __m256d nk_xvfreplgr2vr_d_(double x) {
+    nk_fui64_t c;
+    c.f = x;
+    return (__m256d)__lasx_xvreplgr2vr_d((long long)c.u);
+}
+
+NK_PUBLIC nk_f32_t nk_f32_rsqrt_loongsonasx(nk_f32_t x) {
+    // xvfrsqrt.s is full precision — no Newton-Raphson needed
+    __m256 x_f32x8 = nk_xvfreplgr2vr_s_(x);
+    __m256 result_f32x8 = __lasx_xvfrsqrt_s(x_f32x8);
+    nk_fui32_t c;
+    c.u = (nk_u32_t)__lasx_xvpickve2gr_w((__m256i)result_f32x8, 0);
+    return c.f;
+}
+
+NK_PUBLIC nk_f32_t nk_f32_sqrt_loongsonasx(nk_f32_t x) { return x > 0 ? x * nk_f32_rsqrt_loongsonasx(x) : 0; }
+
+NK_PUBLIC nk_f64_t nk_f64_sqrt_loongsonasx(nk_f64_t x) {
+    __m256d x_f64x4 = nk_xvfreplgr2vr_d_(x);
+    __m256d result_f64x4 = __lasx_xvfsqrt_d(x_f64x4);
+    nk_fui64_t c;
+    c.u = (nk_u64_t)__lasx_xvpickve2gr_du((__m256i)result_f64x4, 0);
+    return c.f;
+}
+
+NK_PUBLIC nk_f64_t nk_f64_rsqrt_loongsonasx(nk_f64_t x) { return 1.0 / nk_f64_sqrt_loongsonasx(x); }
+
+#if defined(__cplusplus)
+} // extern "C"
+#endif
+
+#endif // NK_TARGET_LOONGSONASX
+#endif // NK_TARGET_LOONGARCH64_
+#endif // NK_SCALAR_LOONGSONASX_H

package/numkong/include/numkong/scalar/neon.h

@@ -0,0 +1,122 @@
+/**
+ *  @brief SIMD-accelerated Scalar Math Helpers for NEON.
+ *  @file include/numkong/scalar/neon.h
+ *  @author Ash Vardanian
+ *  @date March 1, 2026
+ *
+ *  @sa include/numkong/scalar.h
+ *
+ *  @section scalars_neon_instructions Key NEON Scalar Instructions
+ *
+ *      Intrinsic  Instruction         A76        M5
+ *      vsqrt_f32  FSQRT (S)           12cy @ 1p  9cy @ 1p
+ *      vsqrt_f64  FSQRT (D)           12cy @ 1p  9cy @ 1p
+ *      vfmas_f32  FMADD (S, S, S, S)  4cy @ 2p   3cy @ 4p
+ *      vfmad_f64  FMADD (D, D, D, D)  4cy @ 2p   3cy @ 4p
+ *      vqaddb_u8  UQADD (B)           2cy @ 2p   3cy @ 2p
+ *      vqaddb_s8  SQADD (B)           2cy @ 2p   3cy @ 2p
+ */
+#ifndef NK_SCALAR_NEON_H
+#define NK_SCALAR_NEON_H
+
+#if NK_TARGET_ARM64_
+#if NK_TARGET_NEON
+
+#include "numkong/types.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+#if defined(__clang__)
+#pragma clang attribute push(__attribute__((target("arch=armv8-a+simd"))), apply_to = function)
+#elif defined(__GNUC__)
+#pragma GCC push_options
+#pragma GCC target("arch=armv8-a+simd")
+#endif
+
+NK_PUBLIC nk_f32_t nk_f32_sqrt_neon(nk_f32_t x) { return vget_lane_f32(vsqrt_f32(vdup_n_f32(x)), 0); }
+NK_PUBLIC nk_f64_t nk_f64_sqrt_neon(nk_f64_t x) { return vget_lane_f64(vsqrt_f64(vdup_n_f64(x)), 0); }
+NK_PUBLIC nk_f32_t nk_f32_rsqrt_neon(nk_f32_t x) {
+    nk_f32_t r = vrsqrtes_f32(x);
+    r *= vrsqrtss_f32(x * r, r);
+    r *= vrsqrtss_f32(x * r, r);
+    return r;
+}
+NK_PUBLIC nk_f64_t nk_f64_rsqrt_neon(nk_f64_t x) {
+    nk_f64_t r = vrsqrted_f64(x);
+    r *= vrsqrtsd_f64(x * r, r);
+    r *= vrsqrtsd_f64(x * r, r);
+    r *= vrsqrtsd_f64(x * r, r);
+    return r;
+}
+NK_PUBLIC nk_f32_t nk_f32_fma_neon(nk_f32_t a, nk_f32_t b, nk_f32_t c) {
+    // MSVC lacks both GCC inline asm and scalar ACLE FMA intrinsics (vfmas_f32/vfmad_f64).
+    // GCC/Clang: use inline asm for scalar FMADD.
+    // MSVC: use vector FMA + lane extract (compiler may optimize to scalar FMADD).
+#if defined(_MSC_VER)
+    return vget_lane_f32(vfma_f32(vdup_n_f32(c), vdup_n_f32(a), vdup_n_f32(b)), 0);
+#else
+    nk_f32_t r;
+    __asm__("fmadd %s0, %s1, %s2, %s3" : "=w"(r) : "w"(a), "w"(b), "w"(c));
+    return r;
+#endif
+}
+NK_PUBLIC nk_f64_t nk_f64_fma_neon(nk_f64_t a, nk_f64_t b, nk_f64_t c) {
+    // MSVC lacks both GCC inline asm and scalar ACLE FMA intrinsics (vfmas_f32/vfmad_f64).
+    // GCC/Clang: use inline asm for scalar FMADD.
+    // MSVC: use vector FMA + lane extract (compiler may optimize to scalar FMADD).
+#if defined(_MSC_VER)
+    return vget_lane_f64(vfma_f64(vdup_n_f64(c), vdup_n_f64(a), vdup_n_f64(b)), 0);
+#else
+    nk_f64_t r;
+    __asm__("fmadd %d0, %d1, %d2, %d3" : "=w"(r) : "w"(a), "w"(b), "w"(c));
+    return r;
+#endif
+}
+
+NK_PUBLIC nk_u8_t nk_u8_saturating_add_neon(nk_u8_t a, nk_u8_t b) { return vqaddb_u8(a, b); }
+NK_PUBLIC nk_i8_t nk_i8_saturating_add_neon(nk_i8_t a, nk_i8_t b) { return vqaddb_s8(a, b); }
+NK_PUBLIC nk_u16_t nk_u16_saturating_add_neon(nk_u16_t a, nk_u16_t b) { return vqaddh_u16(a, b); }
+NK_PUBLIC nk_i16_t nk_i16_saturating_add_neon(nk_i16_t a, nk_i16_t b) { return vqaddh_s16(a, b); }
+NK_PUBLIC nk_u32_t nk_u32_saturating_add_neon(nk_u32_t a, nk_u32_t b) { return vqadds_u32(a, b); }
+NK_PUBLIC nk_i32_t nk_i32_saturating_add_neon(nk_i32_t a, nk_i32_t b) { return vqadds_s32(a, b); }
+NK_PUBLIC nk_u64_t nk_u64_saturating_add_neon(nk_u64_t a, nk_u64_t b) { return vqaddd_u64(a, b); }
+NK_PUBLIC nk_i64_t nk_i64_saturating_add_neon(nk_i64_t a, nk_i64_t b) { return vqaddd_s64(a, b); }
+
+NK_INTERNAL nk_u64_t nk_u64_mulhigh_neon_(nk_u64_t a, nk_u64_t b) {
+#if defined(_MSC_VER)
+    return __umulh(a, b);
+#else
+    nk_u64_t high;
+    __asm__("umulh %0, %1, %2" : "=r"(high) : "r"(a), "r"(b));
+    return high;
+#endif
+}
+NK_PUBLIC nk_u64_t nk_u64_saturating_mul_neon(nk_u64_t a, nk_u64_t b) {
+    return nk_u64_mulhigh_neon_(a, b) ? 18446744073709551615ull : (a * b);
+}
+NK_PUBLIC nk_i64_t nk_i64_saturating_mul_neon(nk_i64_t a, nk_i64_t b) {
+    int sign = (a < 0) ^ (b < 0);
+    nk_u64_t abs_a = a < 0 ? (0u - (nk_u64_t)a) : (nk_u64_t)a;
+    nk_u64_t abs_b = b < 0 ? (0u - (nk_u64_t)b) : (nk_u64_t)b;
+    nk_u64_t high = nk_u64_mulhigh_neon_(abs_a, abs_b);
+    nk_u64_t low = abs_a * abs_b;
+    if (high || (sign && low > 9223372036854775808ull) || (!sign && low > 9223372036854775807ull))
+        return sign ? (-9223372036854775807ll - 1ll) : 9223372036854775807ll;
+    return sign ? -(nk_i64_t)low : (nk_i64_t)low;
+}
+
+#if defined(__clang__)
+#pragma clang attribute pop
+#elif defined(__GNUC__)
+#pragma GCC pop_options
+#endif
+
+#if defined(__cplusplus)
+} // extern "C"
+#endif
+
+#endif // NK_TARGET_NEON
+#endif // NK_TARGET_ARM64_
+#endif // NK_SCALAR_NEON_H

package/numkong/include/numkong/scalar/neonhalf.h

@@ -0,0 +1,70 @@
+/**
+ *  @brief SIMD-accelerated Scalar Math Helpers for NEON FP16 (FEAT_FP16).
+ *  @file include/numkong/scalar/neonhalf.h
+ *  @author Ash Vardanian
+ *  @date March 1, 2026
+ *
+ *  @sa include/numkong/scalar.h
+ *
+ *  ARMv8.2-A FEAT_FP16 provides native scalar f16 sqrt, rsqrt estimate, and fma.
+ *  `vrsqrte_f16` gives ~4-bit estimate; 2 Newton-Raphson steps refine to ~16 bits,
+ *  exceeding f16's 10-bit mantissa precision.
+ */
+#ifndef NK_SCALAR_NEONHALF_H
+#define NK_SCALAR_NEONHALF_H
+
+#if NK_TARGET_ARM64_
+#if NK_TARGET_NEONHALF
+
+#include "numkong/types.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+#if defined(__clang__)
+#pragma clang attribute push(__attribute__((target("arch=armv8.2-a+simd+fp16"))), apply_to = function)
+#elif defined(__GNUC__)
+#pragma GCC push_options
+#pragma GCC target("arch=armv8.2-a+simd+fp16")
+#endif
+
+NK_PUBLIC nk_f16_t nk_f16_sqrt_neonhalf(nk_f16_t x) {
+    float16x4_t x_f16x4 = vld1_dup_f16((nk_f16_for_arm_simd_t const *)&x);
+    x_f16x4 = vsqrt_f16(x_f16x4);
+    nk_f16_t result;
+    vst1_lane_f16((nk_f16_for_arm_simd_t *)&result, x_f16x4, 0);
+    return result;
+}
+NK_PUBLIC nk_f16_t nk_f16_rsqrt_neonhalf(nk_f16_t x) {
+    float16x4_t x_f16x4 = vld1_dup_f16((nk_f16_for_arm_simd_t const *)&x);
+    float16x4_t estimate_f16x4 = vrsqrte_f16(x_f16x4);
+    estimate_f16x4 = vmul_f16(estimate_f16x4, vrsqrts_f16(vmul_f16(x_f16x4, estimate_f16x4), estimate_f16x4));
+    estimate_f16x4 = vmul_f16(estimate_f16x4, vrsqrts_f16(vmul_f16(x_f16x4, estimate_f16x4), estimate_f16x4));
+    nk_f16_t result;
+    vst1_lane_f16((nk_f16_for_arm_simd_t *)&result, estimate_f16x4, 0);
+    return result;
+}
+NK_PUBLIC nk_f16_t nk_f16_fma_neonhalf(nk_f16_t a, nk_f16_t b, nk_f16_t c) {
+    float16x4_t a_f16x4 = vld1_dup_f16((nk_f16_for_arm_simd_t const *)&a);
+    float16x4_t b_f16x4 = vld1_dup_f16((nk_f16_for_arm_simd_t const *)&b);
+    float16x4_t c_f16x4 = vld1_dup_f16((nk_f16_for_arm_simd_t const *)&c);
+    c_f16x4 = vfma_f16(c_f16x4, a_f16x4, b_f16x4);
+    nk_f16_t result;
+    vst1_lane_f16((nk_f16_for_arm_simd_t *)&result, c_f16x4, 0);
+    return result;
+}
+
+#if defined(__clang__)
+#pragma clang attribute pop
+#elif defined(__GNUC__)
+#pragma GCC pop_options
+#endif
+
+#if defined(__cplusplus)
+} // extern "C"
+#endif
+
+#endif // NK_TARGET_NEONHALF
+#endif // NK_TARGET_ARM64_
+#endif // NK_SCALAR_NEONHALF_H

package/numkong/include/numkong/scalar/powervsx.h

@@ -0,0 +1,96 @@
+/**
+ *  @brief SIMD-accelerated Scalar Math Helpers for Power VSX.
+ *  @file include/numkong/scalar/powervsx.h
+ *  @author Ash Vardanian
+ *  @date March 24, 2026
+ *
+ *  @sa include/numkong/scalar.h
+ *
+ *  @section scalars_powervsx_instructions Key Power VSX Scalar Instructions
+ *
+ *      Instruction     Description                  Latency
+ *      xssqrtsp        Scalar √ (f32)               26cy
+ *      xssqrtdp        Scalar √ (f64)               33cy
+ *      xsrsqrtesp      Scalar 1/√ estimate (f32)    6cy
+ *      xsrsqrtedp      Scalar 1/√ estimate (f64)    6cy
+ *      xsmaddadp       Scalar FMA (f64)             5cy
+ *      xsmaddasp       Scalar FMA (f32)             5cy
+ */
+#ifndef NK_SCALAR_POWERVSX_H
+#define NK_SCALAR_POWERVSX_H
+
+#if NK_TARGET_POWER64_
+#if NK_TARGET_POWERVSX
+
+#include "numkong/types.h"
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+#if defined(__clang__)
+#pragma clang attribute push(__attribute__((target("power9-vector"))), apply_to = function)
+#elif defined(__GNUC__)
+#pragma GCC push_options
+#pragma GCC target("power9-vector")
+#endif
+
+NK_PUBLIC nk_f32_t nk_f32_sqrt_powervsx(nk_f32_t x) {
+    nk_f32_t result;
+    __asm__("xssqrtsp %0, %1" : "=f"(result) : "f"(x));
+    return result;
+}
+NK_PUBLIC nk_f64_t nk_f64_sqrt_powervsx(nk_f64_t x) {
+    nk_f64_t result;
+    __asm__("xssqrtdp %0, %1" : "=d"(result) : "d"(x));
+    return result;
+}
+NK_PUBLIC nk_f32_t nk_f32_rsqrt_powervsx(nk_f32_t x) {
+    // xsrsqrtesp → ~12-bit estimate, then 2 Newton→Raphson iterations → ~24-bit precision
+    nk_f32_t r;
+    __asm__("xsrsqrtesp %0, %1" : "=f"(r) : "f"(x));
+    // Newton→Raphson: r = r * (3 - x * r * r) / 2
+    nk_f32_t half_x = x * 0.5f;
+    nk_f32_t three_half = 1.5f;
+    r = r * (three_half - half_x * r * r);
+    r = r * (three_half - half_x * r * r);
+    return r;
+}
+NK_PUBLIC nk_f64_t nk_f64_rsqrt_powervsx(nk_f64_t x) {
+    // xsrsqrtedp → ~14-bit estimate, then 3 Newton→Raphson iterations → ~48-bit precision
+    nk_f64_t r;
+    __asm__("xsrsqrtedp %0, %1" : "=d"(r) : "d"(x));
+    // Newton→Raphson: r = r * (3 - x * r * r) / 2
+    nk_f64_t half_x = x * 0.5;
+    nk_f64_t three_half = 1.5;
+    r = r * (three_half - half_x * r * r);
+    r = r * (three_half - half_x * r * r);
+    r = r * (three_half - half_x * r * r);
+    return r;
+}
+NK_PUBLIC nk_f32_t nk_f32_fma_powervsx(nk_f32_t a, nk_f32_t b, nk_f32_t c) {
+    // xsmaddasp: result = a * b + c (scalar f32 FMA)
+    nk_f32_t r = c;
+    __asm__("xsmaddasp %0, %1, %2" : "+f"(r) : "f"(a), "f"(b));
+    return r;
+}
+NK_PUBLIC nk_f64_t nk_f64_fma_powervsx(nk_f64_t a, nk_f64_t b, nk_f64_t c) {
+    // xsmaddadp: result = a * b + c (scalar f64 FMA)
+    nk_f64_t r = c;
+    __asm__("xsmaddadp %0, %1, %2" : "+d"(r) : "d"(a), "d"(b));
+    return r;
+}
+
+#if defined(__clang__)
+#pragma clang attribute pop
+#elif defined(__GNUC__)
+#pragma GCC pop_options
+#endif
+
+#if defined(__cplusplus)
+} // extern "C"
+#endif
+
+#endif // NK_TARGET_POWERVSX
+#endif // NK_TARGET_POWER64_
+#endif // NK_SCALAR_POWERVSX_H