usearch 0.1.8 → 0.1.9

package/Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "usearch"
-version = "0.1.8"
+version = "0.1.9"
 authors = ["Ashot Vardanian <1983160+ashvardanian@users.noreply.github.com>"]
 description = "The Mighty Tiny Vector Search Engine with Automatic Quantization and Hardware Acceleration"
 edition = "2021"

package/README.md

@@ -17,11 +17,11 @@ Compact, yet Powerful<br/>
 <a href="https://github.com/unum-cloud/usearch"><img height="25" src="https://github.com/unum-cloud/.github/raw/main/assets/github.svg" alt="GitHub"></a>
 </p>
 
-- [x] Single C++11 header implementation, easily extendible.
-- [x] 4B+ sized space efficient point-clouds with `uint40_t`.
-- [x] Half-precision support with [`maratyszcza/fp16`](https://github.com/maratyszcza/fp16).
-- [x] View from disk, without loading into RAM.
-- [x] Any metric, includes: 
+- [x] [Single C++11 header][usearch-header] implementation, easily extendible.
+- [x] [4B+](#go-beyond-4b-entries) sized space efficient point-clouds with `uint40_t`.
+- [x] [Half-precision](#quantize-on-the-fly) support with [`maratyszcza/fp16`](https://github.com/maratyszcza/fp16).
+- [x] [View from disk](#view-larger-indexes-from-disk), without loading into RAM.
+- [x] [Any metric](#define-custom-metrics), includes: 
   - [x] Euclidean, Dot-product, Cosine,
   - [x] Jaccard, Hamming, Haversine.
   - [x] Hardware-accelerated [`ashvardanian/simsimd`](https://github.com/ashvardanian/simsimd). 
@@ -29,10 +29,10 @@ Compact, yet Powerful<br/>
 - [x] Don't copy vectors if not needed.
 - [x] Bring your threads.
 - [x] Multiple vectors per label.
-- [x] Python bindings: `pip install usearch`.
-- [x] JavaScript bindings: `npm install usearch`.
-- [x] Rust bindings: `cargo add usearch`.
-- [x] Java bindings: `cloud.unum:usearch` on GitHub.
+- [x] [Python](#python) bindings: `pip install usearch`.
+- [x] [JavaScript](#java) bindings: `npm install usearch`.
+- [x] [Rust](#rust) bindings: `cargo add usearch`.
+- [x] [Java](#java) bindings: `cloud.unum:usearch` on GitHub.
 - [ ] GoLang bindings.
 - [ ] Wolfram language bindings.
 - [x] For Linux: GCC, Clang.
@@ -41,6 +41,96 @@ Compact, yet Powerful<br/>
 - [ ] Multi-index lookups in Python.
 - [ ] Thread-safe `reserve`.
 - [ ] Distributed construction.
+- [x] AI + Vector Search = Semantic Search.
+
+[usearch-header]: https://github.com/unum-cloud/usearch/blob/main/include/usearch/usearch.hpp
+
+## Features
+
+### Define Custom Metrics
+
+Most vector-search packages focus on just 2 metrics - "Inner Product distance" and "Euclidean distance".
+The lack of dedicated "Cosine distance" can be justified with the simplicity of normalizing such vectors on the fly.
+But that hardly exhausts the list of possible metrics.
+
+![USearch: Vector Search Approaches](assets/usearch-approaches-white.png)
+
+Older approaches indexing high-dimensional spaces, like KD-Trees and Locality Sensitive Hashing are hardly extendible to vectors/objects of variable length.
+Modern NSW-like approaches, however, only require two objects to be comparable.
+This means, the index can be extended to support fuzzy search with Levenstein distance over strings.
+USearch implements HNSW, and provides some uncommon-to-vector-search "similarity measures" and "distances" out of the box: Haversine, Jaccard, Hamming, etc.
+
+### Bring your Threads
+
+Most AI, HPC, or Big Data packages use some form of a thread pool.
+Instead of spawning additional threads within USearch, we focus on thread-safety of the `add` function.
+
+```cpp
+#pragma omp parallel for
+    for (std::size_t i = 0; i < n; ++i)
+        native.add(label, span_t{vector, dims}, omp_get_thread_num());
+```
+
+During initialization we allocate enough temporary memory for all the cores on the machine.
+On call, the user can simply supply the identifier of the current thread, making this library easy to integrate with OpenMP and similar tools.
+
+### Go Beyond 4B Entries
+
+Constructing a vector index is a memory-intensive task.
+Depending on your configuration parameters, part of the memory will be allocated towards copies of vectors, and another part to the Small World Graph, common to NSW-like algorithms.
+Choosing the right data-structures and numeric types can have profound implications on both.
+
+---
+
+Internally, some point "label" is mapped into an integer identifier for every stored vector.
+Most implementations hard-code it to `uint32_t`, as people rarely store more than 4 Billion vectors.
+But when they do, software becomes unusable.
+
+Others hard-code it to `uint64_t`, which hardly space-efficient.
+We added a tiny class - `uint40_t`, that should be enough for collections up to 1 Trillion vectors.
+
+### View Larger Indexes from Disk
+
+To construct a large index one may use a beefy RAM-optimized machine, like the AWS `u-24tb1.metal` instances with 24 TB of RAM.
+Those are, however, pricy, and would cost you over $200/hour.
+
+You wouldn't want to have them running all the time.
+So, if you are working on large datasets, but don't need the in-RAM throughput, you can simply view an existing dataset from disk, without ever loading it fully into memory.
+
+Construct on large machines, deploy on the cheap ones.
+
+### Quantize on the Fly
+
+Most modern CPU have at least partial support for `half`-precision `f16_t` arithmetic.
+USearch supports automatic down-casting and up-casting between `f32_t`, `f16_t`, `f64_t`, and `i8q100_t` representations.
+
+Making vectors smaller will help pack more of them in-memory, while also increasing performance on CPUs implementing native support for target type.
+
+## Performance
+
+Below are the performance numbers for a benchmark running on the 64 cores of AWS `c7g.metal` "Graviton 3"-based instances.
+We fix the default configuration in the top line and show the affects of various parameters by changing one parameter at a time.
+
+|  Vectors   | Connectivity | EF @ A | EF @ S | **Add**, QPS | **Search**, QPS | **Recall @1** |
+| :--------: | :----------: | :----: | :----: | :----------: | :-------------: | ------------: |
+| `f32` x256 |      16      |  128   |   64   |    75'640    |     131'654     |         99.3% |
+|            |              |        |        |              |                 |               |
+| `f32` x256 |      12      |  128   |   64   |    81'747    |     149'728     |         99.0% |
+| `f32` x256 |      32      |  128   |   64   |    64'368    |     104'050     |         99.4% |
+|            |              |        |        |              |                 |               |
+| `f32` x256 |      16      |   64   |   32   |   128'644    |     228'422     |         97.2% |
+| `f32` x256 |      16      |  256   |  128   |    39'981    |     69'065      |         99.2% |
+|            |              |        |        |              |                 |               |
+| `f16` x256 |      16      |   64   |   32   |   128'644    |     228'422     |         97.2% |
+| `f32` x256 |      16      |  256   |  128   |    39'981    |     69'065      |         99.2% |
+
+The main columns are:
+
+- Add: Number of insertion Queries Per Second.
+- Search: Number search Queries Per Second.
+- Recall @1: How often does approximate search yield the exact best match?
+
+To read more and reproduce, jump to [benchmarking section](#benchmarking).
 
 ## Usage
 
@@ -91,13 +181,32 @@ index.view("index.usearch"); // Memory-mapping from disk
 ```
 
 The `add` is thread-safe for concurrent index construction.
+For advanced users, more compile-time abstraction are available.
+
+```cpp
+template <typename metric_at = ip_gt<float>,            //
+          typename label_at = std::size_t,              // `uint32_t`, `uuid_t`...
+          typename id_at = std::uint32_t,               // `uint40_t`, `uint64_t`...
+          typename scalar_at = float,                   // `double`, `half`, `char`...
+          typename allocator_at = std::allocator<char>> //
+class index_gt;
+```
+
+One may also define a custom metric, such as Damerau–Levenshtein distance, to compute the similarity between variable length strings.
+The only constraint is the function signature:
+
+```cpp
+struct custom_metric_t {
+    T operator()(T const* a, T const* b, std::size_t a_length, std::size_t b_length) const;
+};
+```
 
 ### Python
 
 Python bindings are implemented with [`pybind/pybind11`](https://github.com/pybind/pybind11).
 Assuming the presence of Global Interpreter Lock in Python, on large insertions we spawn threads in the C++ layer.
 
-```sh
+```python
 $ pip install usearch
 
 import numpy as np
@@ -125,13 +234,184 @@ assert len(index) == n
 matches, distances, counts = index.search(vectors, 10)
 ```
 
-## Features
+### JavaScript
 
-### Bring your Threads
+```js
+// npm install usearch
 
-## Performance
+var index = new usearch.Index({ metric: 'cos', connectivity: 16, dimensions: 2 })
+assert.equal(index.connectivity(), 16)
+assert.equal(index.dimensions(), 2)
+assert.equal(index.size(), 0)
+
+index.add(15, new Float32Array([10, 20]))
+assert.equal(index.size(), 2)
+
+var results = index.search(new Float32Array([13, 14]), 2)
+assert.deepEqual(results.labels, new Uint32Array([15, 16]))
+assert.deepEqual(results.distances, new Float32Array([45, 130]))
+```
+
+### Rust
+
+Being a systems-programming language, Rust has better control over memory management and concurrency, but lacks function overloading.
+Aside from the `add` and `search`, it also provides `add_in_thread` and `search_in_thread` which let users identify the calling thread to use underlying temporary memory more efficiently.
+
+```rust
+// cargo add usearch
+
+let quant: &str = "f16";
+let index = new_ip(5,  &quant, 0, 0, 0).unwrap();
+
+assert!(index.reserve(10).is_ok());
+assert!(index.capacity() >= 10);
+assert!(index.connectivity() != 0);
+assert_eq!(index.dimensions(), 5);
+assert_eq!(index.size(), 0);
+
+let first: [f32; 5] = [0.2, 0.1, 0.2, 0.1, 0.3];
+let second: [f32; 5] = [0.2, 0.1, 0.2, 0.1, 0.3];
+
+assert!(index.add(42, &first).is_ok());
+assert!(index.add(43, &second).is_ok());
+assert_eq!(index.size(), 2);
+
+// Read back the tags
+let results = index.search(&first, 10).unwrap();
+assert_eq!(results.count, 2);
+
+// Validate serialization
+assert!(index.save("index.usearch").is_ok());
+assert!(index.load("index.usearch").is_ok());
+assert!(index.view("index.usearch").is_ok());
+
+// There are more "metrics" available
+assert!(new_l2(5,  &quant, 0, 0, 0).is_ok());
+assert!(new_cos(5,  &quant, 0, 0, 0).is_ok());
+assert!(new_haversine(&quant, 0, 0, 0).is_ok());
+```
+
+### Java
+
+```java
+Index index = new Index.Config().metric("cos").dimensions(2).build();
+float vec[] = {10, 20};
+index.add(42, vec);
+int[] labels = index.search(vec, 5);
+```
+
+### GoLang
+
+### Wolfram
+
+## Benchmarking
+
+All major HNSW implementation share an identical list of hyper-parameters:
+
+- connectivity (often called `M`),
+- expansion on additions (often called `efConstruction`),
+- expansion on search (often called `ef`).
+
+The default values vary drastically.
+
+|  Library  | Connectivity | EF @ A | EF @ S |
+| :-------: | :----------: | :----: | :----: |
+| `hnswlib` |      16      |  200   |   10   |
+|  `FAISS`  |      32      |   40   |   16   |
+| `USearch` |      16      |  128   |   64   |
+
+
+To achieve best results, please compile locally and check out various configuration options.
+
+```sh
+cmake -B ./build_release \
+    -DCMAKE_BUILD_TYPE=Release \
+    -DUSEARCH_USE_OPENMP=1 \
+    -DUSEARCH_USE_JEMALLOC=1 && \
+    make -C ./build_release -j
+
+./build_release/bench --help
+```
+
+Which would print the following instructions:
+
+```txt
+SYNOPSIS
+        ./build_release/bench [--vectors <path>] [--queries <path>] [--neighbors <path>] [-b] [-j
+                              <integer>] [-c <integer>] [--expansion-add <integer>]
+                              [--expansion-search <integer>] [--native|--f16quant|--i8quant]
+                              [--ip|--l2|--cos|--haversine] [-h]
+
+OPTIONS
+        --vectors <path>
+                    .fbin file path to construct the index
+
+        --queries <path>
+                    .fbin file path to query the index
+
+        --neighbors <path>
+                    .ibin file path with ground truth
+
+        -b, --big   Will switch to uint40_t for neighbors lists with over 4B entries
+        -j, --threads <integer>
+                    Uses all available cores by default
+
+        -c, --connectivity <integer>
+                    Index granularity
+
+        --expansion-add <integer>
+                    Affects indexing depth
+
+        --expansion-search <integer>
+                    Affects search depth
+
+        --native    Use raw templates instead of type-punned classes
+        --f16quant  Enable `f16_t` quantization
+        --i8quant   Enable `int8_t` quantization
+        --ip        Choose Inner Product metric
+        --l2        Choose L2 Euclidean metric
+        --cos       Choose Angular metric
+        --haversine Choose Haversine metric
+        -h, --help  Print this help information on this tool and exit
+```
 
 ## TODO
 
 - JavaScript: Allow calling from "worker threads".
 - Rust: Allow passing a custom thread ID.
+
+## AI + Vector Search = Semantic Search
+
+AI has a growing number of applications, but one of the coolest classic ideas is to use it for Semantic Search.
+One can take an encoder model, like the multi-modal UForm, and a web-programming framework, like UCall, and build an image search platform in just 20 lines of Python.
+
+```python
+import ucall.rich_posix as ucall
+import uform
+import usearch
+
+import numpy as np
+from PIL import Image
+
+server = ucall.Server()
+model = uform.get_model('unum-cloud/uform-vl-multilingual')
+index = usearch.Index(dim=256)
+
+@server
+def add(label: int, photo: Image.Image):
+    image = model.preprocess_image(photo)
+    vector = model.encode_image(image).detach().numpy()
+    labels = np.array([label], dtype=np.longlong)
+    index.add(labels, vector, copy=True)
+
+@server
+def search(query: str) -> np.ndarray:
+    tokens = model.preprocess_text(query)
+    vector = model.encode_text(tokens).detach().numpy()
+    neighbors = index.search(vector, 3)
+    return neighbors[0][:neighbors[2][0]]
+
+server.run()
+```
+
+Check [that](https://github.com/ashvardanian/image-search) and [other](https://github.com/unum-cloud/examples) examples on our corporate GitHub 🤗

package/VERSION

@@ -1 +1 @@
-0.1.8
+0.1.9

package/java/test/IndexTest.java

@@ -19,7 +19,12 @@ public class IndexTest {
     public void test() {
         String path = "./tmp/";
         deleteDirectoryFiles(path);
-        // new Index("cos", "f32", 1, 10, 1, 1, 1);
+
+        // Index index = new Index.Config().metric("cos").dimensions(2).build();
+        // float vec[] = {10, 20};
+        // index.add(42, vec);
+        // int[] labels = index.search(vec, 5);
+
         System.out.println("Java Tests Passed!");
     }
 }

package/package.json

@@ -1,6 +1,6 @@
 {
     "name": "usearch",
-    "version": "0.1.8",
+    "version": "0.1.9",
     "description": "The Mighty Tiny Vector Search Engine with Automatic Quantization and Hardware Acceleration",
     "author": "Ashot Vardanian",
     "license": "Apache 2.0",

package/python/bench.py

@@ -60,7 +60,7 @@ def measure(f) -> float:
     f()
     b = datetime.datetime.now()
     c = b - a
-    print(f'Took: {c.seconds} seconds')
+    print(f'Took: {c.seconds:.2f} seconds')
     return c.seconds
 
 
@@ -71,12 +71,15 @@ def construct_both() -> tuple:
     index_meta.hnsw.efConstruction = efConstruction
     dt_meta = measure(lambda: index_meta.add(xb))
 
-    index_unum = usearch.make_index(
+    index_unum = usearch.Index(
+        dim=d,
         expansion_add=efConstruction,
         expansion_search=efSearch,
         connectivity=M,
     )
-    dt_unum = measure(lambda: index_unum.add(labels, xb, copy=False))
+    assert len(index_unum) == 0
+    dt_unum = measure(lambda: index_unum.add(labels, xb))
+    assert len(index_unum) == xb.shape[0]
 
     return dt_meta, dt_unum
 

package/python/test.py

@@ -6,15 +6,14 @@ count_vectors = 100
 count_dimensions = 96
 
 index = usearch.Index(dim=96)
-print(index)
-
 vectors = np.random.uniform(
     0, 0.3, (count_vectors, count_dimensions)).astype(np.float32)
 labels = np.array(range(count_vectors), dtype=np.longlong)
-print('will insert', labels.shape, vectors.shape)
 
 index.add(labels, vectors, copy=True)
+assert index.ndim == count_dimensions
+assert index.capacity >= count_vectors
+assert len(index) == count_vectors
 
-print('will search')
 results = index.search(vectors, 10)
 print('found', results[0].shape, results[1].shape, results[2].shape)

package/src/advanced.hpp

@@ -127,17 +127,17 @@ void multithreaded(std::size_t threads, std::size_t tasks, callback_at&& callbac
     }
 
     std::vector<std::thread> threads_pool;
-    std::size_t tasks_per_thread = threads / tasks + (threads % tasks) != 0;
-    for (std::size_t thread = 0; thread != threads; ++thread) {
+    std::size_t tasks_per_thread = tasks / threads + (tasks % threads) != 0;
+    for (std::size_t thread_idx = 0; thread_idx != threads; ++thread_idx) {
         threads_pool.emplace_back([=]() {
-            for (std::size_t task_idx = thread * tasks_per_thread;
-                 task_idx < std::min(tasks, thread * tasks_per_thread + tasks_per_thread); ++task_idx)
-                callback(thread, task_idx);
+            for (std::size_t task_idx = thread_idx * tasks_per_thread;
+                 task_idx < std::min(tasks, thread_idx * tasks_per_thread + tasks_per_thread); ++task_idx)
+                callback(thread_idx, task_idx);
         });
     }
 
-    for (std::size_t thread = 0; thread != threads; ++thread)
-        threads_pool[thread].join();
+    for (std::size_t thread_idx = 0; thread_idx != threads; ++thread_idx)
+        threads_pool[thread_idx].join();
 }
 
 /**
@@ -172,6 +172,16 @@ enum class accuracy_t {
     i8q100_k,
 };
 
+inline char const* accuracy_name(accuracy_t accuracy) noexcept {
+    switch (accuracy) {
+    case accuracy_t::f32_k: return "f32";
+    case accuracy_t::f16_k: return "f16";
+    case accuracy_t::f64_k: return "f64";
+    case accuracy_t::i8q100_k: return "i8q100";
+    default: return "";
+    }
+}
+
 inline bool str_equals(char const* begin, std::size_t len, char const* other_begin) noexcept {
     std::size_t other_len = strlen(other_begin);
     return len == other_len && strncmp(begin, other_begin, len) == 0;
@@ -217,13 +227,14 @@ enum class isa_t {
     avx512_k,
 };
 
-inline char const* isa(isa_t isa) noexcept {
+inline char const* isa_name(isa_t isa) noexcept {
     switch (isa) {
     case isa_t::auto_k: return "auto";
     case isa_t::neon_k: return "neon";
     case isa_t::sve_k: return "sve";
     case isa_t::avx2_k: return "avx2";
     case isa_t::avx512_k: return "avx512";
+    default: return "";
     }
 }
 
@@ -445,7 +456,6 @@ class auto_index_gt {
     }
 
     static metric_and_meta_t ip_metric_f32(std::size_t dimensions) {
-#if 0
 #if defined(__x86_64__)
         if (dimensions % 4 == 0)
             return {
@@ -458,7 +468,7 @@ class auto_index_gt {
         if (supports_arm_sve())
             return {
                 pun_metric<simsimd_f32_t>([](simsimd_f32_t const* a, simsimd_f32_t const* b, size_t d) noexcept {
-                return 1 - simsimd_dot_f32sve(a, b, d);
+                    return 1 - simsimd_dot_f32sve(a, b, d);
                 }),
                 isa_t::sve_k,
             };
@@ -469,13 +479,11 @@ class auto_index_gt {
                 }),
                 isa_t::neon_k,
             };
-#endif
 #endif
         return {pun_metric<f32_t>(ip_gt<f32_t>{}), isa_t::auto_k};
     }
 
     static metric_and_meta_t ip_metric_f16(std::size_t dimensions) {
-#if 0
 #if defined(__x86_64__)
 #elif defined(__aarch64__)
         if (supports_arm_sve())
@@ -492,7 +500,6 @@ class auto_index_gt {
                 }),
                 isa_t::neon_k,
             };
-#endif
 #endif
         return {pun_metric<f16_converted_t>(ip_gt<f16_converted_t>{}), isa_t::auto_k};
     }

package/src/bench.cpp

@@ -295,8 +295,10 @@ void run_type_punned(dataset_at& dataset, args_t const& args, config_t config) {
     if (args.quantize_i8)
         accuracy = accuracy_t::i8q100_k;
 
+    std::printf("-- Accuracy: %s\n", accuracy_name(accuracy));
+
     index_at index{type_punned_index_for_metric<index_at>(dataset, args, config, accuracy)};
-    std::printf("- Hardware acceleration: %s\n", isa(index.acceleration()));
+    std::printf("- Hardware acceleration: %s\n", isa_name(index.acceleration()));
     std::printf("-- Will benchmark in-memory\n");
 
     single_shot(dataset, index, true);
@@ -410,6 +412,11 @@ int main(int argc, char** argv) {
     config.max_threads_add = config.max_threads_search = args.threads;
     config.max_elements = dataset.vectors_count();
 
+    std::printf("- Index: \n");
+    std::printf("-- Connectivity: %zu\n", config.connectivity);
+    std::printf("-- Expansion @ Add: %zu\n", config.expansion_add);
+    std::printf("-- Expansion @ Search: %zu\n", config.expansion_search);
+
     if (args.big)
         run_big_or_small<uint40_t>(dataset, args, config);
     else