usearch 2.8.3 → 2.8.4

package/.vscode/settings.json

@@ -147,6 +147,7 @@
         "pytest",
         "Quickstart",
         "rtype",
+        "SIMD",
         "simsimd",
         "SLOC",
         "sorensen",

package/CITATION.cff

@@ -5,7 +5,7 @@ authors:
   given-names: "Ash"
   orcid: "https://orcid.org/0000-0002-4882-1815"
 title: "USearch by Unum Cloud"
-version: 2.8.3
+version: 2.8.4
 doi: 10.5281/zenodo.7949416
 date-released: 2023-10-22
 url: "https://github.com/unum-cloud/usearch"

package/CONTRIBUTING.md

@@ -1,12 +1,264 @@
 # Contribution Guide
 
-Thank you for even opening this page!
-It's always nice to have third-party contributors!
+Thank you for coming here!
+It's always nice to have third-party contributors 🤗
 
 ---
 
-To keep the quality of the code high, we have a set of guidelines common to all Unum projects.
+To keep the quality of the code high, we have a set of guidelines common to [all Unum projects](https://github.com/unum-cloud).
 
-- Whats the procedure? [answered](https://github.com/unum-cloud/awesome/blob/main/Workflow.md#organizing-software-development)
-- How to organize branches? [answered](https://github.com/unum-cloud/awesome/blob/main/Workflow.md#branches)
-- How to style commits? [answered](https://github.com/unum-cloud/awesome/blob/main/Workflow.md#commits)
+- [What's the procedure?](https://github.com/unum-cloud/awesome/blob/main/Workflow.md#organizing-software-development)
+- [How to organize branches?](https://github.com/unum-cloud/awesome/blob/main/Workflow.md#branches)
+- [How to style commits?](https://github.com/unum-cloud/awesome/blob/main/Workflow.md#commits)
+
+## Before you start
+
+Before building the first time, please pull `git` submodules.
+That's how we bring in SimSIMD and other optional dependencies to test all of the available functionality.
+
+```sh
+git submodule update --init --recursive
+```
+
+## C++ 11 and C 99
+
+Our primary C++ implementation uses CMake for builds.
+If this is your first experience with CMake, use the following commands to get started:
+
+```sh
+sudo apt-get update && sudo apt-get install cmake build-essential libjemalloc-dev g++-12 gcc-12 # Ubuntu
+brew install libomp llvm # MacOS
+```
+
+Using modern syntax, this is how you build and run the test suite:
+
+```sh
+cmake -DUSEARCH_BUILD_TEST_CPP=1 -B ./build_debug
+cmake --build ./build_debug --config Debug
+./build_debug/test_cpp
+```
+
+The CMakeLists.txt file has a number of options you can pass:
+
+- What to build:
+  - `USEARCH_BUILD_TEST_CPP` - build the C++ test suite
+  - `USEARCH_BUILD_BENCH_CPP` - build the C++ benchmark suite
+  - `USEARCH_BUILD_LIB_C` - build the C library
+  - `USEARCH_BUILD_TEST_C` - build the C test suite
+- Which dependencies to use:
+  - `USEARCH_USE_OPENMP` - use OpenMP for parallelism
+  - `USEARCH_USE_SIMSIMD` - use SimSIMD for vectorization
+  - `USEARCH_USE_JEMALLOC` - use Jemalloc for memory management
+  - `USEARCH_USE_NATIVE_F16` - use native half-precision floating point
+
+Putting all of this together on Ubuntu and compiling the "release" version using the GCC 12 compiler:
+
+```sh
+cmake \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DCMAKE_C_COMPILER=gcc-12 \
+    -DCMAKE_CXX_COMPILER=g++-12 \
+    -DUSEARCH_USE_NATIVE_F16=1 \
+    -DUSEARCH_USE_OPENMP=1 \
+    -DUSEARCH_USE_SIMSIMD=1 \
+    -DUSEARCH_USE_JEMALLOC=1 \
+    -DUSEARCH_BUILD_TEST_CPP=1 \
+    -DUSEARCH_BUILD_BENCH_CPP=1 \
+    -DUSEARCH_BUILD_LIB_C=1 \
+    -DUSEARCH_BUILD_TEST_C=1 \
+    -B ./build_release
+
+cmake --build ./build_release --config Release
+./build_release/test_cpp
+./build_release/test_c
+```
+
+Similarly, to use the most recent Clang compiler version from HomeBrew on MacOS:
+
+```sh
+cmake \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DCMAKE_C_COMPILER="/opt/homebrew/opt/llvm/bin/clang" \
+    -DCMAKE_CXX_COMPILER="/opt/homebrew/opt/llvm/bin/clang++" \
+    -DUSEARCH_USE_NATIVE_F16=1 \
+    -DUSEARCH_USE_OPENMP=1 \
+    -DUSEARCH_USE_SIMSIMD=1 \
+    -DUSEARCH_USE_JEMALLOC=1 \
+    -DUSEARCH_BUILD_TEST_CPP=1 \
+    -DUSEARCH_BUILD_BENCH_CPP=1 \
+    -DUSEARCH_BUILD_LIB_C=1 \
+    -DUSEARCH_BUILD_TEST_C=1 \
+    -B ./build_release
+
+cmake --build ./build_release --config Release
+./build_release/test_cpp
+./build_release/test_c
+```
+
+Linting:
+
+```sh
+cppcheck --enable=all --force --suppress=cstyleCast --suppress=unusedFunction \
+    include/usearch/index.hpp \
+    include/index_dense.hpp \
+    include/index_plugins.hpp
+```
+
+## Python 3
+
+Python bindings are built using PyBind11 and are available on [PyPi](https://pypi.org/project/usearch/).
+The compilation settings are controlled by the `setup.py` and are independent from CMake used for C/C++ builds.
+Use PyTest to validate the build.
+
+- The `-s` option will disable capturing the logs.
+- The `-x` option will exit after first failure to simplify debugging.
+
+```sh
+pip install -e . && pytest python/scripts/ -s -x
+```
+
+Linting:
+
+```sh
+pip install ruff
+ruff --format=github --select=E9,F63,F7,F82 --target-version=py37 python
+```
+
+Before merging your changes you may want to test your changes against the entire matrix of Python versions USearch supports.
+For that you need the `cibuildwheel`, which is tricky to use on MacOS and Windows, as it would target just the local environment.
+Still, if you have Docker running on any desktop OS, you can use it to build and test the Python bindings for all Python versions for Linux:
+
+```sh
+pip install cibuildwheel
+cibuildwheel --platform linux
+```
+
+## JavaScript
+
+USearch provides NAPI bindings for NodeJS available on [NPM](https://www.npmjs.com/package/usearch).
+The compilation settings are controlled by the `binding.gyp` and are independent from CMake used for C/C++ builds.
+
+```sh
+npm install
+node --test ./javascript/usearch.test.js
+npm publish
+```
+
+To compile to WebAssembly make sure you have `emscripten` installed and run the following script:
+
+```sh
+emcmake cmake -B ./build -DCMAKE_CXX_FLAGS="${CMAKE_CXX_FLAGS} -s TOTAL_MEMORY=64MB" && emmake make -C ./build
+node ./build/usearch.test.js
+```
+
+If you don't yet have `emcmake` installed:
+
+```sh
+git clone https://github.com/emscripten-core/emsdk.git && ./emsdk/emsdk install latest && ./emsdk/emsdk activate latest && source ./emsdk/emsdk_env.sh
+```
+
+## Rust
+
+USearch provides Rust bindings available on [Crates.io](https://crates.io/crates/usearch).
+The compilation settings are controlled by the `build.rs` and are independent from CMake used for C/C++ builds.
+
+```sh
+cargo test -p usearch
+cargo publish
+```
+
+## Objective-C and Swift
+
+USearch provides both Objective-C and Swift bindings through the [Swift Package Manager](https://swiftpackageindex.com/unum-cloud/usearch).
+The compilation settings are controlled by the `Package.swift` and are independent from CMake used for C/C++ builds.
+
+```sh
+swift build
+swift test -v
+```
+
+## GoLang
+
+USearch provides GoLang bindings, that depend on the C library that must be installed beforehand.
+So one should first compile the C library, link it with GoLang, and only then run tests.
+
+```sh
+cmake -B ./build_release -DUSEARCH_BUILD_LIB_C=1 -DUSEARCH_BUILD_TEST_C=1 -DUSEARCH_USE_OPENMP=1 -DUSEARCH_USE_SIMSIMD=1 
+cmake --build ./build_release --config Release -j
+
+mv ./c/libusearch_c.so ./golang/ # or .dylib to install the library on MacOS
+cp ./c/usearch.h ./golang/ # to make the header available to GoLang
+
+cd golang && go test -v ; cd ..
+```
+
+## Java
+
+USearch provides Java bindings available from the [GitHub Maven registry](https://github.com/unum-cloud/usearch/packages/1867475).
+The compilation settings are controlled by the `build.gradle` and are independent from CMake used for C/C++ builds.
+
+```sh
+gradle clean build
+java -cp . -Djava.library.path="$(pwd)/build/libs/usearch/shared" java/cloud/unum/usearch/Index.java
+```
+
+Or step by-step:
+
+```sh
+cs java/unum/cloud/usearch
+javac -h . Index.java
+
+# Ubuntu:
+g++ -c -fPIC -I${JAVA_HOME}/include -I${JAVA_HOME}/include/linux cloud_unum_usearch_Index.cpp -o cloud_unum_usearch_Index.o
+g++ -shared -fPIC -o libusearch_c.so cloud_unum_usearch_Index.o -lc
+
+# Windows
+g++ -c -I%JAVA_HOME%\include -I%JAVA_HOME%\include\win32 cloud_unum_usearch_Index.cpp -o cloud_unum_usearch_Index.o
+g++ -shared -o USearchJNI.dll cloud_unum_usearch_Index.o -Wl,--add-stdcall-alias
+
+# MacOS
+g++ -std=c++11 -c -fPIC \
+    -I../../../../include \
+    -I../../../../fp16/include \
+    -I../../../../simsimd/include \
+    -I${JAVA_HOME}/include -I${JAVA_HOME}/include/darwin cloud_unum_usearch_Index.cpp -o cloud_unum_usearch_Index.o
+g++ -dynamiclib -o libusearch.dylib cloud_unum_usearch_Index.o -lc
+# Run linking to that directory
+java -cp . -Djava.library.path="$(pwd)/java/cloud/unum/usearch/" Index.java
+java -cp . -Djava.library.path="$(pwd)/java" cloud.unum.usearch.Index
+```
+
+## Wolfram
+
+```sh
+brew install --cask wolfram-engine
+```
+
+## Docker
+
+```sh
+docker build -t unum/usearch . && docker run unum/usearch
+```
+
+For multi-architecture builds and publications:
+
+```sh
+version=$(cat VERSION)
+docker buildx create --use &&
+    docker login &&
+    docker buildx build \
+        --platform "linux/amd64,linux/arm64" \
+        --build-arg version=$version \
+        --file Dockerfile \
+        --tag unum/usearch:$version \
+        --tag unum/usearch:latest \
+        --push .
+```
+
+## Working on Sub-Modules
+
+Extending metrics in SimSIMD:
+
+```sh
+git push --set-upstream https://github.com/ashvardanian/simsimd.git HEAD:main
+```

package/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "usearch"
-version = "2.8.3"
+version = "2.8.4"
 authors = ["Ash Vardanian <1983160+ashvardanian@users.noreply.github.com>"]
 description = "Smaller & Faster Single-File Vector Search Engine from Unum"
 edition = "2021"

package/README.md

@@ -1,7 +1,7 @@
 <h1 align="center">USearch</h1>
 <h3 align="center">
-Faster & Smaller Single-File<br/>
-Search Engine for Vectors & Texts<br/>
+Smaller & <a href="https://www.unum.cloud/blog/2023-11-11-scaling-vector-search">Faster</a> Single-File<br/>
+Similarity Search Engine for <a href="https://github.com/ashvardanian/simsimd">Vectors</a> & 🔜 <a href="https://github.com/ashvardanian/stringzilla">Texts</a>
 </h3>
 <br/>
 
@@ -18,7 +18,7 @@ Search Engine for Vectors & Texts<br/>
 </p>
 
 <p align="center">
-Euclidean • Angular • Bitwise • Haversine • User-Defined Metrics
+Spatial • Binary • Probabilistic • User-Defined Metrics
 <br/>
 <a href="https://unum-cloud.github.io/usearch/cpp">C++ 11</a> •
 <a href="https://unum-cloud.github.io/usearch/python">Python 3</a> •
@@ -32,7 +32,7 @@ Euclidean • Angular • Bitwise • Haversine • User-Defined Metrics
 <a href="https://unum-cloud.github.io/usearch/golang">GoLang</a> •
 <a href="https://unum-cloud.github.io/usearch/wolfram">Wolfram</a>
 <br/>
-Linux • MacOS • Windows • iOS • Docker • WebAssembly
+Linux • MacOS • Windows • iOS • WebAssembly
 </p>
 
 <div align="center">
@@ -46,40 +46,57 @@ Linux • MacOS • Windows • iOS • Docker • WebAssembly
 
 ---
 
-- ✅ Benchmark-topping performance.
+- ✅ 10x faster than [FAISS][faiss] implementation of [HNSW algorithm][hnsw-algorithm].
 - ✅ Simple and extensible [single C++11 header][usearch-header] implementation.
-- ✅ SIMD-optimized and [user-defined metrics](#user-defined-functions) with JIT compilation.
-- ✅ Variable dimensionality vectors for unique applications, including search over compressed data.
-- ✅ Bitwise Tanimoto and Sorensen coefficients for [Genomics and Chemistry applications](#usearch--rdkit--molecular-search).
+- ✅ Compatible with a dozen programming languages out of the box.
+- ✅ [SIMD][simd]-optimized and [user-defined metrics](#user-defined-functions) with JIT compilation.
 - ✅ Hardware-agnostic `f16` & `i8` - [half-precision & quarter-precision support](#memory-efficiency-downcasting-and-quantization).
 - ✅ [View large indexes from disk](#serving-index-from-disk) without loading into RAM.
+- ✅ Heterogeneous lookups, renaming/relabeling, and on-the-fly deletions.
+- ✅ Variable dimensionality vectors for unique applications, including search over compressed data.
+- ✅ Binary Tanimoto and Sorensen coefficients for [Genomics and Chemistry applications](#usearch--rdkit--molecular-search).
 - ✅ Space-efficient point-clouds with `uint40_t`, accommodating 4B+ size.
 - ✅ Compatible with OpenMP and custom "executors", for fine-grained control over CPU utilization.
-- ✅ Heterogeneous lookups, renaming/relabeling, and on-the-fly deletions.
-- ✅ Near-real-time [clustering and sub-clusterings](#clustering) for Tens or Millions of clusters.
+- ✅ Near-real-time [clustering and sub-clustering](#clustering) for Tens or Millions of clusters.
 - ✅ [Semantic Search](#usearch--ai--multi-modal-semantic-search) and [Joins](#joins).
 
+[faiss]: https://github.com/facebookresearch/faiss
 [usearch-header]: https://github.com/unum-cloud/usearch/blob/main/include/usearch/index.hpp
 [obscure-use-cases]: https://ashvardanian.com/posts/abusing-vector-search
+[hnsw-algorithm]: https://arxiv.org/abs/1603.09320
+[simd]: https://en.wikipedia.org/wiki/Single_instruction,_multiple_data
 
 ## Comparison with FAISS
 
 FAISS is a widely recognized standard for high-performance vector search engines.
 USearch and FAISS both employ the same HNSW algorithm, but they differ significantly in their design principles.
-USearch is compact and broadly compatible without sacrificing performance, with a primary focus on user-defined metrics and fewer dependencies.
-
-|                     | FAISS                         | USearch                            |
-| :------------------ | :---------------------------- | :--------------------------------- |
-| Implementation      | 84 K [SLOC][sloc] in `faiss/` | 3 K [SLOC][sloc] in `usearch/`     |
-| Supported metrics   | 9 fixed metrics               | Any User-Defined metrics           |
-| Supported languages | C++, Python                   | 10 languages                       |
-| Supported ID types  | `uint32_t`, `uint64_t`        | `uint32_t`, `uint40_t`, `uint64_t` |
-| Dependencies        | BLAS, OpenMP                  | None                               |
-| Bindings            | SWIG                          | Native                             |
-| Acceleration        | Learned Quantization          | Downcasting                        |
+USearch is compact and broadly compatible without sacrificing performance, primarily focusing on user-defined metrics and fewer dependencies.
+
+|                                              |                         FAISS |                        USearch |              Improvement |
+| :------------------------------------------- | ----------------------------: | -----------------------------: | -----------------------: |
+| Indexing time                                |                               |                                |                          |
+| 100 Million 96d `f32`, `f16`, `i8` vectors   |           2.6 h, 2.6 h, 2.6 h |            0.3 h, 0.2 h, 0.2 h | __9.6x, 10.4x, 10.7x__ ⁰ |
+| 100 Million 1536d `f32`, `f16`, `i8` vectors |           5.0 h, 4.1 h, 3.8 h |            2.1 h, 1.1 h, 0.8 h |    __2.3x 3.6x, 4.4x__ ⁰ |
+|                                              |                               |                                |                          |
+| Codebase length                              | 84 K [SLOC][sloc] in `faiss/` | 3 K [SLOC][sloc] in `usearch/` |           maintainable ¹ |
+| Supported metrics                            |               9 fixed metrics |       any user-defined metrics |             extendible ² |
+| Supported languages                          |                   C++, Python |                   10 languages |               portable ³ |
+| Supported ID types                           |                32-bit, 64-bit |         32-bit, 40-bit, 64-bit |              efficient ⁴ |
+| Required dependencies                        |                  BLAS, OpenMP |                              - |           light-weight ⁵ |
+| Bindings                                     |                          SWIG |                         Native |            low-latency ⁶ |
 
 [sloc]: https://en.wikipedia.org/wiki/Source_lines_of_code
 
+> ⁰ [Tested][intel-benchmarks] on Intel Sapphire Rapids, with the simplest inner-product distance, equivalent recall, and memory consumption, while also providing far superior search speed.
+> ¹ A shorter codebase makes the project easier to maintain and audit.
+> ² User-defined metrics allow you to customize your search for various applications, from GIS to creating custom metrics for composite embeddings from multiple AI models or hybrid full-text and semantic search.
+> ³ With USearch, you can reuse the same preconstructed index in various programming languages.
+> ⁴ The 40-bit integer allows you to store 4B+ vectors without allocating 8 bytes for every neighbor reference in the proximity graph.
+> ⁵ Lack of obligatory dependencies makes USearch much more portable.
+> ⁶ Native bindings introduce lower call latencies than more straightforward approaches.
+
+[intel-benchmarks]: https://www.unum.cloud/blog/2023-11-11-scaling-vector-search
+
 Base functionality is identical to FAISS, and the interface must be familiar if you have ever investigated Approximate Nearest Neighbors search:
 
 ```py
@@ -88,44 +105,30 @@ $ pip install usearch
 import numpy as np
 from usearch.index import Index
 
-index = Index(
-    ndim=3, # Define the number of dimensions in input vectors
-    metric='cos', # Choose 'l2sq', 'haversine' or other metric, default = 'ip'
-    dtype='f32', # Quantize to 'f16' or 'i8' if needed, default = 'f32'
-    connectivity=16, # Optional: How frequent should the connections in the graph be
-    expansion_add=128, # Optional: Control the recall of indexing
-    expansion_search=64, # Optional: Control the quality of search
-)
+index = Index(ndim=3)
 
 vector = np.array([0.2, 0.6, 0.4])
 index.add(42, vector)
+
 matches = index.search(vector, 10)
 
-assert len(index) == 1
-assert len(matches) == 1
 assert matches[0].key == 42
 assert matches[0].distance <= 0.001
 assert np.allclose(index[42], vector)
 ```
 
-Comparing the performance of FAISS against USearch on 1 Million 96-dimensional vectors from the famous Deep1B dataset, once can expect the following numbers on modern AWS `c7g.metal` instances.
-
-|              | FAISS, `f32` | USearch, `f32` | USearch, `f16` |     USearch, `i8` |
-| :----------- | -----------: | -------------: | -------------: | ----------------: |
-| Batch Insert |       16 K/s |         73 K/s |        100 K/s | 104 K/s __+550%__ |
-| Batch Search |       82 K/s |        103 K/s |        113 K/s |  134 K/s __+63%__ |
-| Bulk Insert  |       76 K/s |        105 K/s |        115 K/s | 202 K/s __+165%__ |
-| Bulk Search  |      118 K/s |        174 K/s |        173 K/s | 304 K/s __+157%__ |
-| Recall @ 10  |          99% |          99.2% |          99.1% |             99.2% |
+More settings are always available, and the API is designed to be as flexible as possible.
 
-> HNSW was configured with identical hyper-parameters:
-> connectivity `M=16`,
-> expansion @ construction `efConstruction=128`,
-> and expansion @ search `ef=64`.
-> Batch size is 256.
-> Jump to the [Performance Tuning][benchmarking] section to read about the effects of those hyper-parameters.
-
-[benchmarking]: https://github.com/unum-cloud/usearch/blob/main/docs/benchmarks.md
+```py
+index = Index(
+    ndim=3, # Define the number of dimensions in input vectors
+    metric='cos', # Choose 'l2sq', 'haversine' or other metric, default = 'ip'
+    dtype='f32', # Quantize to 'f16' or 'i8' if needed, default = 'f32'
+    connectivity=16, # Optional: How frequent should the connections in the graph be
+    expansion_add=128, # Optional: Control the recall of indexing
+    expansion_search=64, # Optional: Control the quality of search
+)
+```
 
 ## User-Defined Functions
 
@@ -220,7 +223,7 @@ multi_index.search(...)
 ## Clustering
 
 Once the index is constructed, it can be used to cluster entries much faster.
-In essense, the `Index` itself can be seen as a clustering, and it allows iterative deepening.
+In essence, the `Index` itself can be seen as a clustering, and it allows iterative deepening.
 
 ```py
 clustering = index.cluster(
@@ -241,7 +244,7 @@ g = clustering.network
 # Get members of a specific cluster
 first_members = clustering.members_of(centroid_keys[0])
 
-# Deepen into that cluster spliting it into more parts, all same arguments supported
+# Deepen into that cluster splitting it into more parts, all same arguments supported
 sub_clustering = clustering.subcluster(min_count=..., max_count=...)
 ```
 
@@ -342,7 +345,7 @@ Comparing molecule graphs and searching for similar structures is expensive and
 It can be seen as a special case of the NP-Complete Subgraph Isomorphism problem.
 Luckily, domain-specific approximate methods exist.
 The one commonly used in Chemistry, is to generate structures from [SMILES][smiles], and later hash them into binary fingerprints.
-The latter are searchable with bitwise similarity metrics, like the Tanimoto coefficient.
+The latter are searchable with binary similarity metrics, like the Tanimoto coefficient.
 Below is an example using the RDKit package.
 
 ```python
@@ -380,9 +383,10 @@ The search is based on the Haversine distance metric, but can easily be extended
 ## Integrations
 
 - [x] GPTCache: [Python](https://github.com/zilliztech/GPTCache/releases/tag/0.1.29).
-- [x] LangChain: [Python](https://github.com/langchain-ai/langchain/releases/tag/v0.0.257) and [JavaScipt](https://github.com/hwchase17/langchainjs/releases/tag/0.0.125).
+- [x] LangChain: [Python](https://github.com/langchain-ai/langchain/releases/tag/v0.0.257) and [JavaScript](https://github.com/hwchase17/langchainjs/releases/tag/0.0.125).
 - [x] ClickHouse: [C++](https://github.com/ClickHouse/ClickHouse/pull/53447).
 - [x] Microsoft Semantic Kernel: [Python](https://github.com/microsoft/semantic-kernel/releases/tag/python-0.3.9.dev) and C#.
+- [x] LanternDB: [C++](https://github.com/lanterndata/lantern) and [Rust](https://github.com/lanterndata/lantern_extras).
 
 ## Citations
 
@@ -392,7 +396,7 @@ doi = {10.5281/zenodo.7949416},
 author = {Vardanian, Ash},
 title = {{USearch by Unum Cloud}},
 url = {https://github.com/unum-cloud/usearch},
-version = {2.8.3},
+version = {2.8.4},
 year = {2023},
 month = oct,
 }

package/conanfile.py

@@ -7,7 +7,7 @@ required_conan_version = ">=1.53.0"
 class USearchConan(ConanFile):
 
     name = "usearch"
-    version = '2.8.3'
+    version = '2.8.4'
     license = "Apache-2.0"
     description = "Smaller & Faster Single-File Vector Search Engine from Unum"
     homepage = "https://github.com/unum-cloud/usearch"

package/cpp/README.md

@@ -25,14 +25,20 @@ metric_punned_t metric(256, metric_kind_t::l2sq_k, scalar_kind_t::f32_k);
 index_dense_t index = index_dense_t::make(metric);
 float vec[3] = {0.1, 0.3, 0.2};
 
-index.reserve(10);
-index.add(/* key: */ 42, /* vector: */ {&vec[0], 3});
-auto results = index.search(/* query: */ {&vec[0], 3}, 5 /* neighbors */);
+index.reserve(10); // Pre-allocate memory for 10 vectors
+index.add(42, &vec[0]); // Pass a key and a vector
+auto results = index.search(&vec[0], 5); // Pass a query and limit number of results
 
 for (std::size_t i = 0; i != results.size(); ++i)
     results[i].element.key, results[i].element.vector, results[i].distance;
 ```
 
+Here we:
+
+- define a metric of kind [`metric_kind_t::l2sq_k`](https://unum-cloud.github.io/usearch/cpp/reference.html#_CPPv413metric_kind_t),
+- to be applied to [`scalar_kind_t::f32_k`](https://unum-cloud.github.io/usearch/cpp/reference.html#_CPPv413scalar_kind_t) floating-point vectors,
+- instantiate an [`index_dense_t`](https://unum-cloud.github.io/usearch/cpp/reference.html#_CPPv4I00EN4unum7usearch14index_dense_gtE) index.
+
 The `add` is thread-safe for concurrent index construction.
 It also has an overload for different vector types, casting them under the hood.
 The same applies to the `search`, `get`, `cluster`, and `distance_between` functions.
@@ -52,20 +58,78 @@ index.load("index.usearch"); // Copying from disk
 index.view("index.usearch"); // Memory-mapping from disk
 ```
 
-## User-Defined Metrics in C++
+## Multi-Threading
 
-For advanced users, more compile-time abstractions are available.
+Most AI, HPC, or Big Data packages use some form of a thread pool.
+Instead of spawning additional threads within USearch, we focus on the thread safety of `add()` function, simplifying resource management.
 
 ```cpp
-template <typename distance_at = default_distance_t,              // `float`
-          typename key_at = default_key_t,                        // `int64_t`, `uuid_t`
-          typename compressed_slot_at = default_slot_t,           // `uint32_t`, `uint40_t`
-          typename dynamic_allocator_at = std::allocator<byte_t>, //
-          typename tape_allocator_at = dynamic_allocator_at>      //
-class index_gt;
+#pragma omp parallel for
+    for (std::size_t i = 0; i < n; ++i)
+        native.add(key, span_t{vector, dims});
+```
+
+During initialization, we allocate enough temporary memory for all the cores on the machine.
+On the call, the user can supply the identifier of the current thread, making this library easy to integrate with OpenMP and similar tools.
+
+Moreover, you can take advantage of one of the provided "executors" to parallelize the search:
+
+- `executor_openmp_t`, that would use OpenMP under the hood.
+- `executor_stl_t`, that will spawn `std::thread` instances.
+- `dummy_executor_t`, that will run everything sequentially.
+
+## Clustering
+
+Aside from basic Create-Read-Update-Delete (CRUD) operations and search, USearch also supports clustering.
+Once the index is constructed, you can either:
+
+- Identify a cluster to which any external vector belongs, once mapped onto the index.
+- Split the entire index into a set of clusters, each with its own centroid.
+
+For the first, the interface accepts a vector and a "clustering level", which is essentially the index of the HNSW graph layer, in which to search.
+If you pass zero, the traversal will happen in every level except the bottom one.
+Otherwise, the search will be limited to the specified level.
+
+```cpp
+some_scalar_t vector[3] = {0.1, 0.3, 0.2};
+cluster_result_t result = index.cluster(&vector, index.max_level() / 2);
+match_t cluster = result.cluster;
+member_cref_t member = cluster.member;
+distance_t distance = cluster.distance;
 ```
 
-The following distances are pre-packaged:
+If you wish to split the whole structure into clusters, you must provide an iterator over a range of vectors, that will be processed in parallel using the previously described function.
+Unlike the previous function, you don't have to manually specify the level, as the algorithm will pick the best one for you, depending on the number of clusters you want to highlight.
+Aside from that auto-tuning, this function will regroup some of the clusters, if they are too small, and return the final number of clusters.
+
+```cpp
+std::size_t queries_count = queries_end - queries_begin;
+index_dense_clustering_config_t config;
+config.min_clusters = 1000;
+config.max_clusters = 2000;
+config.mode = index_dense_clustering_config_t::merge_smallest_k;
+
+// Outputs:
+vector_key_t cluster_centroids_keys[queries_count];
+distance_t distances_to_cluster_centroids[queries_count];
+executor_default_t thread_pool;
+dummy_progress_t progress_bar;
+
+clustering_result_t result = cluster(
+        queries_begin, queries_end,
+        config,
+        &cluster_centroids_keys, &distances_to_cluster_centroids,
+        thread_pool, progress_bar);
+```
+
+This approach requires basic understanding of templates meta-programming to implement the `queries_begin` and `queries_end` smart-iterators.
+On the bright side, it allows iteratively deepening into a specific cluster.
+
+As in many other bulk-processing APIs, the `executor` and `progress` are optional.
+
+## User-Defined Metrics
+
+In its high-level interface, USearch supports a variety of metrics, including the most popular ones:
 
 - `metric_cos_gt<scalar_t>` for "Cosine" or "Angular" distance.
 - `metric_ip_gt<scalar_t>` for "Inner Product" or "Dot Product" distance.
@@ -78,16 +142,20 @@ The following distances are pre-packaged:
 - `metric_haversine_gt<scalar_t>` for "Haversine" or "Great Circle" distance between coordinates used in GIS applications.
 - `metric_divergence_gt<scalar_t>` for the "Jensen Shannon" similarity between probability distributions.
 
-## Multi-Threading
+In reality, for most common types, one of the [SimSIMD](https://github.com/ashvardanian/SimSIMD) backends will be triggered, providing hardware-acceleration for most common CPUs.
 
-Most AI, HPC, or Big Data packages use some form of a thread pool.
-Instead of spawning additional threads within USearch, we focus on the thread safety of `add()` function, simplifying resource management.
+If you need a different metric, you can implement it yourself and wrap it into a `metric_punned_t`, which is our alternative to the `std::function`.
+Unlike the `std::function`, it is a trivial type, which is important for performance.
+
+## Advanced Interface
+
+If you are proficient in C++ and ready to get your hands dirty, you can use the low-level interface.
 
 ```cpp
-#pragma omp parallel for
-    for (std::size_t i = 0; i < n; ++i)
-        native.add(key, span_t{vector, dims});
+template <typename distance_at = default_distance_t,              // `float`
+          typename key_at = default_key_t,                        // `int64_t`, `uuid_t`
+          typename compressed_slot_at = default_slot_t,           // `uint32_t`, `uint40_t`
+          typename dynamic_allocator_at = std::allocator<byte_t>, //
+          typename tape_allocator_at = dynamic_allocator_at>      //
+class index_gt;
 ```
-
-During initialization, we allocate enough temporary memory for all the cores on the machine.
-On the call, the user can supply the identifier of the current thread, making this library easy to integrate with OpenMP and similar tools.