usearch 2.21.4 → 2.25.0

package/numkong/bench/README.md

@@ -0,0 +1,217 @@
+# NumKong Built-in Benchmarks
+
+Internal profiling suite comparing NumKong's SIMD backends against each other and optionally against BLAS libraries.
+For broader comparisons — Rust, Python, etc. — see [NumWars](https://github.com/ashvardanian/NumWars).
+
+- On x86 it compares serial code to manually-vectorized Haswell, Skylake, Ice Lake, Genoa, Sapphire Rapids and newer-generation SIMD kernels.
+- On Arm it compares serial code to manually-vectorized NEON, SVE, SVE2, SME, SME2 with various extensions for BF16 and mixed-precision dot-products.
+- On RISC-V it compares serial code to manually-vectorized RVV 1.0 kernels with and without BB, BF16, and F16 extensions.
+- In WASM environments it compares serial code to manually-vectorized V128 kernels with Relaxed SIMD extensions.
+
+## C++
+
+### Building
+
+```sh
+cmake -B build_release -D CMAKE_BUILD_TYPE=Release -D NK_BUILD_BENCH=1
+cmake --build build_release --config Release --parallel
+```
+
+With BLAS or MKL cross-validation:
+
+```sh
+cmake -B build_release -D CMAKE_BUILD_TYPE=Release \
+      -D NK_BUILD_BENCH=1 \
+      -D NK_COMPARE_TO_BLAS=1 \
+      -D NK_COMPARE_TO_MKL=1
+```
+
+On macOS with Homebrew Clang and OpenBLAS — see [CONTRIBUTING.md](../CONTRIBUTING.md#macos) for the full recipe, adding `-DNK_BUILD_BENCH=1` to the cmake flags.
+
+Compiler requirements vary by ISA target — see [CONTRIBUTING.md](../CONTRIBUTING.md#compiler-requirements) for the full table.
+
+### Running
+
+```sh
+build_release/nk_bench                                    # run all benchmarks
+build_release/nk_bench --benchmark_filter=dot             # filter by name
+build_release/nk_bench --benchmark_min_time=10s           # longer runs for stable results
+build_release/nk_bench --filter=dot                       # shorthand for --benchmark_filter
+```
+
+### Environment Variables
+
+| Variable                  | Default | Description                                            |
+| ------------------------- | ------- | ------------------------------------------------------ |
+| `NK_FILTER`               | `.*`    | Regex to filter benchmarks by name                     |
+| `NK_SEED`                 | `42`    | RNG seed for reproducible inputs                       |
+| `NK_BUDGET_SECS`          | `10`    | Minimum time per benchmark in seconds                  |
+| `NK_BUDGET_MB`            | `1024`  | Memory budget for pre-allocated inputs                 |
+| `NK_DENSE_DIMENSIONS`     | `1536`  | Vector dimension for dot/spatial benchmarks            |
+| `NK_CURVED_DIMENSIONS`    | `64`    | Vector dimension for curved / bilinear form benchmarks |
+| `NK_MESH_POINTS`          | `1000`  | Point count for mesh / RMSD / Kabsch benchmarks        |
+| `NK_MATRIX_HEIGHT`        | `1024`  | GEMM M dimension, dataset size in kNN                  |
+| `NK_MATRIX_WIDTH`         | `128`   | GEMM N dimension, query count in kNN                   |
+| `NK_MATRIX_DEPTH`         | `1536`  | GEMM K dimension, vector dimension in kNN              |
+| `NK_SPARSE_FIRST_LENGTH`  | `1024`  | First set size for sparse benchmarks                   |
+| `NK_SPARSE_SECOND_LENGTH` | `8192`  | Second set size for sparse benchmarks                  |
+| `NK_SPARSE_INTERSECTION`  | `0.5`   | Intersection share [0.0, 1.0] for sparse benchmarks    |
+| `NK_MAX_COORD_ANGLE`      | `180`   | Maximum angle in degrees for geospatial benchmarks     |
+
+Disable multi-threading in BLAS libraries to avoid interference:
+
+```sh
+export OPENBLAS_NUM_THREADS=1    # for OpenBLAS
+export MKL_NUM_THREADS=1         # for Intel MKL
+export VECLIB_MAXIMUM_THREADS=1  # for Apple Accelerate
+export BLIS_NUM_THREADS=1        # for BLIS
+```
+
+### Reported Units
+
+| Benchmark Type                          | Counter        | Meaning                                                         |
+| --------------------------------------- | -------------- | --------------------------------------------------------------- |
+| Vector kernels — dot, spatial, set, ... | `bytes/s`      | Bytes of input consumed per second, both input vectors combined |
+| GEMM, symmetric, batch                  | `scalar-ops/s` | Scalar multiply-accumulate operations per second / FLOPS        |
+| Reductions, casts, trigonometry         | `bytes/s`      | Bytes of input consumed per second, single input vector         |
+
+__bytes__: total bytes across all input vectors read per call.
+For a pair of 1536-dimensional `f32` vectors: `2 * 1536 * 4 = 12288` bytes per call.
+
+__scalar-ops__: number of scalar arithmetic operations.
+For dense GEMM: `2 * M * N * K` per call.
+For symmetric GEMM: `N * (N + 1) * K` per call.
+
+## JavaScript
+
+### Running
+
+```sh
+npm run bench:native                            # Node.js native addon
+npm run bench:emscripten                        # Emscripten WASM with SIMD
+npm run bench:wasi                              # WASI portable execution
+npm run bench:browser                           # Chromium via Playwright
+npm run bench:all                               # all runtimes
+```
+
+```sh
+NK_DIMENSIONS=768 NK_FILTER="dot" npm run bench:native    # custom config
+```
+
+| Variable        | Default  | Description                             |
+| --------------- | -------- | --------------------------------------- |
+| `NK_DIMENSIONS` | `1536`   | Vector dimensionality                   |
+| `NK_ITERATIONS` | `1000`   | Number of benchmark iterations          |
+| `NK_FILTER`     | `.*`     | Regex to filter benchmarks              |
+| `NK_RUNTIME`    | `native` | Runtime: `native`, `emscripten`, `wasi` |
+| `NK_SEED`       | `42`     | Random seed for reproducible data       |
+
+### Output
+
+JSON results are written to `bench/results/`.
+Generate a Markdown comparison report:
+
+```sh
+npm run bench:report
+cat bench/results/report.md
+```
+
+## WASM
+
+__Emscripten — wasm32 and wasm64__
+
+```sh
+source ~/emsdk/emsdk_env.sh
+cmake -B build-wasm -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-wasm.cmake -DNK_BUILD_BENCH=1
+cmake --build build-wasm --parallel
+```
+
+For wasm64:
+
+```sh
+cmake -B build-wasm64 -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-wasm64.cmake -DNK_BUILD_BENCH=1
+cmake --build build-wasm64 --parallel
+```
+
+The toolchain files enable `-msimd128` and `-mrelaxed-simd` automatically.
+
+__WASI__
+
+```sh
+export WASI_SDK_PATH=~/wasi-sdk-24.0-x86_64-linux
+cmake -B build-wasi -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain-wasi.cmake -DNK_BUILD_BENCH=1
+cmake --build build-wasi --parallel
+```
+
+__Running__
+
+```sh
+wasmtime run -W simd=y,relaxed-simd=y,threads=y,shared-memory=y -S threads=y,inherit-env=y ./build-wasi/nk_bench.wasm
+wasmer run --enable-simd --enable-relaxed-simd ./build-wasi/nk_bench.wasm
+node ./build-wasm/nk_bench.js
+```
+
+Browser benchmarks via Playwright:
+
+```sh
+npm run bench:browser
+```
+
+__Interpreting WASM Results__
+
+WASM benchmarks run slower than native due to JIT compilation overhead and memory indirection.
+Expected performance relative to native:
+
+| Runtime              | Typical Throughput vs Native |
+| -------------------- | ---------------------------- |
+| Emscripten / Node.js | 60–80%                       |
+| WASI / Wasmtime      | 50–70%                       |
+| Browser / Chromium   | 40–60%                       |
+
+wasm64 / Memory64 adds ~5–10% overhead vs wasm32 due to 64-bit pointer arithmetic.
+Relaxed SIMD provides measurable gains for fused multiply-add patterns — compare with and without `--wasm-features relaxed-simd` to quantify.
+
+## Frequency Scaling on AMX and SME
+
+Intel AMX tiles on Sapphire Rapids and later cause P-state throttling: the CPU reduces its frequency when AMX instructions execute, similar to heavy AVX-512 workloads.
+Arm SME streaming mode on Graviton4 and Apple M4 has analogous frequency effects when entering and exiting streaming SVE mode.
+
+This means AMX/SME benchmarks that interleave with non-AMX/SME work will show misleading throughput numbers as the CPU oscillates between frequency states.
+
+Mitigations:
+
+- Use `--benchmark_min_time=10s` or higher to amortize warm-up over a longer measurement window.
+- Disable turbo boost with `echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo` on Linux.
+- Run AMX/SME benchmarks in isolation — do not mix them with non-AMX/SME benchmarks in the same invocation.
+- Filter with `--benchmark_filter=amx` or `--benchmark_filter=sme` for dedicated runs.
+
+## Pinning to Performance Cores
+
+### Linux
+
+```sh
+taskset -c 0-3 ./build_release/nk_bench
+numactl --physcpubind=0-3 ./build_release/nk_bench
+```
+
+For dedicated benchmarking machines, add `isolcpus=4-7` to the kernel command line and pin benchmarks to isolated cores.
+
+### macOS
+
+No direct core-pinning API exists on macOS.
+Use QoS to avoid efficiency cores:
+
+```sh
+taskpolicy -b ./build_release/nk_bench
+```
+
+On Apple Silicon there is no public API for P/E core pinning.
+Run with minimal background load for reproducible results.
+
+### Windows
+
+```sh
+start /affinity 0xF nk_bench.exe
+```
+
+The hex mask `0xF` pins to cores 0-3.

package/numkong/bench/bench-browser.html

@@ -0,0 +1,323 @@
+<!DOCTYPE html>
+<html lang="en">
+
+<head>
+    <meta charset="UTF-8">
+    <meta name="viewport" content="width=device-width, initial-scale=1.0">
+    <title>NumKong Browser Benchmarks</title>
+    <style>
+        body {
+            font-family: 'Courier New', monospace;
+            max-width: 1200px;
+            margin: 40px auto;
+            padding: 20px;
+            background: #1e1e1e;
+            color: #d4d4d4;
+        }
+
+        h1 {
+            color: #4ec9b0;
+            border-bottom: 2px solid #4ec9b0;
+            padding-bottom: 10px;
+        }
+
+        h2 {
+            color: #569cd6;
+            margin-top: 30px;
+        }
+
+        #progress {
+            background: #264f78;
+            padding: 15px;
+            border-radius: 5px;
+            margin: 20px 0;
+            font-weight: bold;
+        }
+
+        #results {
+            background: #252526;
+            padding: 20px;
+            border-radius: 5px;
+            margin-top: 20px;
+        }
+
+        .result-row {
+            padding: 8px 0;
+            border-bottom: 1px solid #3e3e42;
+        }
+
+        .result-row:last-child {
+            border-bottom: none;
+        }
+
+        .test-name {
+            color: #ce9178;
+            display: inline-block;
+            width: 250px;
+        }
+
+        .ops-sec {
+            color: #b5cea8;
+            display: inline-block;
+            width: 150px;
+            text-align: right;
+        }
+
+        .time {
+            color: #9cdcfe;
+            display: inline-block;
+            margin-left: 20px;
+        }
+
+        .error {
+            color: #f48771;
+            background: #5a1d1d;
+            padding: 10px;
+            border-radius: 5px;
+            margin: 10px 0;
+        }
+
+        .complete {
+            color: #4ec9b0;
+            font-weight: bold;
+        }
+    </style>
+</head>
+
+<body>
+    <h1>NumKong Browser Benchmarks</h1>
+
+    <div id="config">
+        <h2>Configuration</h2>
+        <div id="config-info">Loading...</div>
+    </div>
+
+    <div id="progress">Initializing NumKong WASM module...</div>
+
+    <div id="results"></div>
+
+    <script src="https://cdn.jsdelivr.net/npm/benchmark@2.1.4/benchmark.min.js"></script>
+    <script type="module">
+        // Configuration from URL parameters or defaults
+        const urlParams = new URLSearchParams(window.location.search);
+        const CONFIG = {
+            dimensions: parseInt(urlParams.get('dimensions') || '1536'),
+            iterations: parseInt(urlParams.get('iterations') || '1000'),
+            filter: new RegExp(urlParams.get('filter') || '.*'),
+            seed: parseInt(urlParams.get('seed') || '42')
+        };
+
+        // Benchmark matrix (same as bench-multi-runtime.mjs)
+        const BENCHMARK_MATRIX = {
+            functions: ['dot', 'inner', 'angular', 'sqeuclidean', 'euclidean', 'hamming', 'jaccard', 'kullbackleibler', 'jensenshannon'],
+            dtypes: ['f64', 'f32', 'f16', 'bf16', 'e4m3', 'e5m2', 'e2m3', 'e3m2', 'i8', 'u8', 'u1'],
+        };
+
+        const FUNCTION_DTYPE_SUPPORT = {
+            'dot': ['f64', 'f32', 'f16', 'bf16', 'e4m3', 'e5m2', 'e2m3', 'e3m2', 'i8', 'u8'],
+            'inner': ['f64', 'f32', 'f16', 'bf16', 'e4m3', 'e5m2', 'e2m3', 'e3m2', 'i8', 'u8'],
+            'angular': ['f64', 'f32', 'f16', 'bf16', 'e4m3', 'e5m2', 'e2m3', 'e3m2', 'i8'],
+            'sqeuclidean': ['f64', 'f32', 'f16', 'bf16', 'e4m3', 'e5m2', 'e2m3', 'e3m2', 'i8', 'u8'],
+            'euclidean': ['f64', 'f32', 'f16', 'bf16', 'e4m3', 'e5m2', 'e2m3', 'e3m2', 'i8', 'u8'],
+            'hamming': ['u1'],
+            'jaccard': ['u1'],
+            'kullbackleibler': ['f64', 'f32'],
+            'jensenshannon': ['f64', 'f32']
+        };
+
+        // Simple PRNG for reproducible benchmarks
+        class Random {
+            constructor(seed) {
+                this.seed = seed;
+            }
+
+            next() {
+                this.seed = (this.seed * 1103515245 + 12345) & 0x7fffffff;
+                return this.seed / 0x7fffffff;
+            }
+        }
+
+        // Generate test data
+        function generateTestData(dtype, length, seed) {
+            const rng = new Random(seed);
+
+            switch (dtype) {
+                case 'f64':
+                    return Float64Array.from({ length }, () => rng.next() * 2 - 1);
+                case 'f32':
+                    return Float32Array.from({ length }, () => rng.next() * 2 - 1);
+                case 'f16':
+                case 'bf16':
+                case 'e4m3':
+                case 'e5m2':
+                case 'e2m3':
+                case 'e3m2':
+                    return Float32Array.from({ length }, () => rng.next() * 2 - 1);
+                case 'i8':
+                    return Int8Array.from({ length }, () => Math.floor(rng.next() * 256) - 128);
+                case 'u8':
+                    return Uint8Array.from({ length }, () => Math.floor(rng.next() * 256));
+                case 'u1':
+                    const byteLength = Math.ceil(length / 8);
+                    return Uint8Array.from({ length: byteLength }, () => Math.floor(rng.next() * 256));
+                default:
+                    throw new Error(`Unknown dtype: ${dtype}`);
+            }
+        }
+
+        // Update UI
+        function updateProgress(message) {
+            document.getElementById('progress').textContent = message;
+        }
+
+
+        function benchmarkOperation(fn, iterations) {
+            const warmupIterations = Math.max(16, Math.min(256, iterations));
+            const samples = 5;
+            const sampleTimesMs = [];
+
+            for (let i = 0; i < warmupIterations; i++) {
+                fn();
+            }
+
+            for (let sample = 0; sample < samples; sample++) {
+                const start = performance.now();
+                for (let i = 0; i < iterations; i++) {
+                    fn();
+                }
+                const elapsedMs = performance.now() - start;
+                sampleTimesMs.push(elapsedMs / iterations);
+            }
+
+            const meanTime = sampleTimesMs.reduce((sum, value) => sum + value, 0) / sampleTimesMs.length;
+            const variance = sampleTimesMs.reduce((sum, value) => {
+                const delta = value - meanTime;
+                return sum + delta * delta;
+            }, 0) / sampleTimesMs.length;
+            const stdDev = Math.sqrt(variance);
+            const opsPerSec = meanTime > 0 ? 1000 / meanTime : 0;
+
+            return { opsPerSec, meanTime, stdDev };
+        }
+
+        function addResult(testName, opsPerSec, meanTime, stdDev) {
+            const resultsDiv = document.getElementById('results');
+            const row = document.createElement('div');
+            row.className = 'result-row';
+            row.innerHTML = `
+                <span class="test-name">${testName}</span>
+                <span class="ops-sec">${opsPerSec.toFixed(0).padStart(10)} ops/sec</span>
+                <span class="time">(${meanTime.toFixed(3)}ms ± ${stdDev.toFixed(3)}ms)</span>
+            `;
+            resultsDiv.appendChild(row);
+        }
+
+        function showError(message) {
+            const resultsDiv = document.getElementById('results');
+            const errorDiv = document.createElement('div');
+            errorDiv.className = 'error';
+            errorDiv.textContent = `❌ Error: ${message}`;
+            resultsDiv.appendChild(errorDiv);
+        }
+
+        // Main benchmark execution
+        async function runBenchmarks() {
+            try {
+                // Display configuration
+                document.getElementById('config-info').innerHTML = `
+                    <div>Dimensions: ${CONFIG.dimensions}</div>
+                    <div>Filter: ${CONFIG.filter.source}</div>
+                    <div>Seed: ${CONFIG.seed}</div>
+                    <div>Runtime: Browser (Chromium)</div>
+                `;
+
+                // Load NumKong WASM module
+                updateProgress('Loading NumKong WASM module...');
+
+                const wasmPath = '../build-wasm/numkong.js';
+                const wasmWrapperPath = '../javascript/dist/esm/numkong-wasm.js';
+                const numkongWrapper = await import(wasmWrapperPath);
+                const NumKongModule = await import(wasmPath);
+                const wasmInstance = await NumKongModule.default();
+                numkongWrapper.initWasm(wasmInstance);
+                const numkong = numkongWrapper;
+
+                updateProgress('NumKong WASM loaded successfully. Running benchmarks...');
+
+                const results = [];
+                let testCount = 0;
+                let totalTests = 0;
+
+                // Count total tests
+                for (const func of BENCHMARK_MATRIX.functions) {
+                    const supportedDtypes = FUNCTION_DTYPE_SUPPORT[func] || [];
+                    for (const dtype of supportedDtypes) {
+                        const testName = `${func}-${dtype}`;
+                        if (CONFIG.filter.test(testName)) {
+                            totalTests++;
+                        }
+                    }
+                }
+
+                // Run benchmarks
+                for (const func of BENCHMARK_MATRIX.functions) {
+                    const supportedDtypes = FUNCTION_DTYPE_SUPPORT[func] || [];
+
+                    for (const dtype of supportedDtypes) {
+                        const testName = `${func}-${dtype}`;
+
+                        // Apply filter
+                        if (!CONFIG.filter.test(testName)) {
+                            continue;
+                        }
+
+                        testCount++;
+                        updateProgress(`Running ${testCount}/${totalTests}: ${testName}...`);
+
+                        // Generate test data
+                        const a = generateTestData(dtype, CONFIG.dimensions, CONFIG.seed);
+                        const b = generateTestData(dtype, CONFIG.dimensions, CONFIG.seed + 1);
+
+                        const metrics = benchmarkOperation(() => numkong[func](a, b), CONFIG.iterations);
+
+                        addResult(testName, metrics.opsPerSec, metrics.meanTime, metrics.stdDev);
+
+                        results.push({
+                            function: func,
+                            dtype: dtype,
+                            runtime: 'browser',
+                            dimensions: CONFIG.dimensions,
+                            opsPerSec: metrics.opsPerSec,
+                            meanTime: metrics.meanTime,
+                            stdDev: metrics.stdDev,
+                            timestamp: new Date().toISOString()
+                        });
+                    }
+                }
+
+                // Make results available to Playwright
+                window.benchmarkResults = results;
+                window.benchmarkComplete = true;
+
+                updateProgress(`✅ Benchmark complete! Total tests run: ${results.length}`);
+
+                const completeDiv = document.createElement('div');
+                completeDiv.className = 'complete';
+                completeDiv.style.marginTop = '20px';
+                completeDiv.textContent = `All ${results.length} benchmarks completed successfully.`;
+                document.getElementById('results').appendChild(completeDiv);
+
+            } catch (error) {
+                showError(error.message);
+                updateProgress(`❌ Benchmark failed: ${error.message}`);
+                window.benchmarkError = error.message;
+                window.benchmarkComplete = true;
+            }
+        }
+
+        // Start benchmarks when page loads
+        runBenchmarks();
+    </script>
+</body>
+
+</html>