usearch 2.10.5 → 2.11.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -194,21 +194,40 @@ When compared to FAISS's `IndexFlatL2` in Google Colab, __[USearch may offer up
194
194
 
195
195
  ## User-Defined Functions
196
196
 
197
- While most vector search packages concentrate on just a few metrics - "Inner Product distance" and "Euclidean distance," USearch extends this list to include any user-defined metrics.
198
- This flexibility allows you to customize your search for various applications, from computing geospatial coordinates with the rare [Haversine][haversine] distance to creating custom metrics for composite embeddings from multiple AI models.
197
+ While most vector search packages concentrate on just two metrics, "Inner Product distance" and "Euclidean distance", USearch allows arbitrary user-defined metrics.
198
+ This flexibility allows you to customize your search for various applications, from computing geospatial coordinates with the rare [Haversine][haversine] distance to creating custom metrics for composite embeddings from multiple AI models, like joint image-text embeddings.
199
+ You can use [Numba][numba], [Cppyy][cppyy], or [PeachPy][peachpy] to define your [custom metric even in Python](https://unum-cloud.github.io/usearch/python#user-defined-metrics-and-jit-in-python):
199
200
 
200
- ![USearch: Vector Search Approaches](https://github.com/unum-cloud/usearch/blob/main/assets/usearch-approaches-white.png?raw=true)
201
+ ```py
202
+ from numba import cfunc, types, carray
203
+ from usearch.index import Index, MetricKind, MetricSignature, CompiledMetric
204
+
205
+ @cfunc(types.float32(types.CPointer(types.float32), types.CPointer(types.float32)))
206
+ def python_inner_product(a, b):
207
+ a_array = carray(a, ndim)
208
+ b_array = carray(b, ndim)
209
+ c = 0.0
210
+ for i in range(ndim):
211
+ c += a_array[i] * b_array[i]
212
+ return 1 - c
213
+
214
+ metric = CompiledMetric(pointer=python_inner_product.address, kind=MetricKind.IP, signature=MetricSignature.ArrayArray)
215
+ index = Index(ndim=ndim, metric=metric, dtype=np.float32)
216
+ ```
201
217
 
202
- Unlike older approaches indexing high-dimensional spaces, like KD-Trees and Locality Sensitive Hashing, HNSW doesn't require vectors to be identical in length.
218
+ Similar effect is even easier to achieve in C, C++, and Rust interfaces.
219
+ Moreover, unlike older approaches indexing high-dimensional spaces, like KD-Trees and Locality Sensitive Hashing, HNSW doesn't require vectors to be identical in length.
203
220
  They only have to be comparable.
204
- So you can apply it in [obscure][obscure] applications, like searching for similar sets or fuzzy text matching, using [GZip][gzip-similarity] as a distance function.
205
-
206
- > Read more about [JIT and UDF in USearch Python SDK](https://unum-cloud.github.io/usearch/python#user-defined-metrics-and-jit-in-python).
221
+ So you can apply it in [obscure][obscure] applications, like searching for similar sets or fuzzy text matching, using [GZip][gzip-similarity] compression-ratio as a distance function.
207
222
 
208
223
  [haversine]: https://ashvardanian.com/posts/abusing-vector-search#geo-spatial-indexing
209
224
  [obscure]: https://ashvardanian.com/posts/abusing-vector-search
210
225
  [gzip-similarity]: https://twitter.com/LukeGessler/status/1679211291292889100?s=20
211
226
 
227
+ [numba]: https://numba.readthedocs.io/en/stable/reference/jit-compilation.html#c-callbacks
228
+ [cppyy]: https://cppyy.readthedocs.io/en/latest/
229
+ [peachpy]: https://github.com/Maratyszcza/PeachPy
230
+
212
231
  ## Memory Efficiency, Downcasting, and Quantization
213
232
 
214
233
  Training a quantization model and dimension-reduction is a common approach to accelerate vector search.
@@ -315,6 +334,7 @@ In some cases, like Batch operations, feature parity is meaningless, as the host
315
334
  | Save, load, view | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
316
335
  | User-defined metrics | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
317
336
  | Batch operations | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ |
337
+ | Filter predicates | ✅ | ❌ | ✅ | ❌ | ❌ | ✅ | ❌ | ❌ |
318
338
  | Joins | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
319
339
  | Variable-length vectors | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
320
340
  | 4B+ capacities | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
@@ -432,7 +452,7 @@ doi = {10.5281/zenodo.7949416},
432
452
  author = {Vardanian, Ash},
433
453
  title = {{USearch by Unum Cloud}},
434
454
  url = {https://github.com/unum-cloud/usearch},
435
- version = {2.10.5},
455
+ version = {2.11.0},
436
456
  year = {2023},
437
457
  month = oct,
438
458
  }
package/c/README.md CHANGED
@@ -2,33 +2,243 @@
2
2
 
3
3
  ## Installation
4
4
 
5
- The simplest form to integrate is to copy the contents of `usearch/c/` into your project.
6
- To build the library `libusearch_static_c` and `libusearch_c`, pass enable the `USEARCH_BUILD_LIB_C` CMake option:
5
+ USearch vector-search engine can be integrated into your project using CMake.
6
+ Alternatively, you can download one of the precompiled binaries from the [releases page](https://github.com/unum-cloud/usearch/releases).
7
7
 
8
- ```bash
9
- cmake -DCMAKE_BUILD_TYPE=Release -DUSEARCH_BUILD_LIB_C=1 -DUSEARCH_BUILD_TEST_C=1 -DUSEARCH_BUILD_TEST_CPP=0 -DUSEARCH_BUILD_BENCH_CPP=0 -B ./build_release
10
- cmake --build ./build_release --config Release
11
- ./build_release/test_c
8
+ ## Quickstart
9
+
10
+ ```c
11
+ #include <stdio.h> // For printing.
12
+ #include <assert.h> // For assertions!
13
+ #include <usearch/usearch.h> // For the win 🚀
14
+
15
+ int main() {
16
+
17
+ // Construct:
18
+ size_t dimensions = 128;
19
+ usearch_error_t error = NULL;
20
+ usearch_init_options_t opts = {
21
+ .metric_kind = usearch_metric_cos_k,
22
+ .scalar_kind = usearch_scalar_f16_k,
23
+ .dimensions = dimensions,
24
+ .expansion_add = 0, // for defaults
25
+ .expansion_search = 0 // for defaults
26
+ };
27
+ usearch_index_t index = usearch_init(&opts, &error);
28
+
29
+ size_t vectors_count = 1000;
30
+ usearch_reserve(index, vectors_count, &error);
31
+ if (error) goto cleanup;
32
+
33
+ // Populate:
34
+ float vector[dimensions]; // don't forget to fill the vector with data
35
+ usearch_add(index, 42, &vector[0], usearch_scalar_f32_k, &error);
36
+ if (error) goto cleanup;
37
+
38
+ // Check up:
39
+ assert(usearch_size(index, &error) == vectors_count);
40
+ assert(usearch_capacity(index, &error) == vectors_count);
41
+ assert(usearch_contains(index, 42, &error));
42
+
43
+ // Search:
44
+ usearch_key_t found_keys[10];
45
+ float found_distances[10];
46
+ size_t found_count = usearch_search(
47
+ index, &vector[0], usearch_scalar_f32_k, 10,
48
+ &found_keys[0], &found_distances[0], &error);
49
+
50
+ usearch_free(index, &error);
51
+
52
+ cleanup:
53
+ if (error) fprintf(stderr, "Error: %s\n", error);
54
+ if (index) usearch_free(index, &error);
55
+ return error ? 1 : 0;
56
+ }
12
57
  ```
13
58
 
14
- ## Quickstart
59
+ ## Serialization
60
+
61
+ To save and load the index from disk, use the following methods:
62
+
63
+ ```c
64
+ usearch_save(index, "index.usearch", &error);
65
+ usearch_load(index, "index.usearch", &error);
66
+ usearch_view(index, "index.usearch", &error);
67
+ ```
68
+
69
+ Before saving them do disk, you can determine exactly how much disk space the index will take:
70
+
71
+ ```c
72
+ size_t bytes = usearch_serialized_length(index, &error);
73
+ ```
74
+
75
+ With a known size, you can serialize the index into an in-memory a buffer:
76
+
77
+ ```c
78
+ void* buffer = malloc(bytes);
79
+ usearch_save_buffer(index, buffer, bytes, &error);
80
+ usearch_load_buffer(index, buffer, bytes, &error);
81
+ usearch_view_buffer(index, buffer, bytes, &error);
82
+ ```
83
+
84
+ One can also retrieve index metadata from the buffer or from disk:
85
+
86
+ ```c
87
+ usearch_init_options_t opts;
88
+ usearch_metadata("index.usearch", &opts, &error);
89
+ usearch_metadata_buffer(buffer, bytes, &opts, &error);
90
+ ```
91
+
92
+ ## Metrics
93
+
94
+ USearch comes pre-packaged with SimSIMD, bringing over 100 SIMD-accelerated distance kernels for x86 and ARM architectures.
95
+ That includes:
96
+
97
+ - `usearch_metric_cos_k` - Cosine Similarity metric, defined as `Cos = 1 - sum(a[i] * b[i]) / (sqrt(sum(a[i]^2) * sqrt(sum(b[i]^2)))`.
98
+ - `usearch_metric_ip_k` - Inner Product metric, defined as `IP = 1 - sum(a[i] * b[i])`.
99
+ - `usearch_metric_l2sq_k` - Squared Euclidean Distance metric, defined as `L2 = sum((a[i] - b[i])^2)`.
100
+ - `usearch_metric_haversine_k` - Haversine (Great Circle) Distance metric.
101
+ - `usearch_metric_divergence_k` - Jensen Shannon Divergence metric.
102
+ - `usearch_metric_pearson_k` - Pearson Correlation metric.
103
+ - `usearch_metric_hamming_k` - Bit-level Hamming Distance metric, defined as the number of differing bits.
104
+ - `usearch_metric_tanimoto_k` - Bit-level Tanimoto (Jaccard) metric, defined as the number of intersecting bits divided by the number of union bits.
105
+ - `usearch_metric_sorensen_k` - Bit-level Sorensen metric.
106
+
107
+ ### User-Defined Metrics
108
+
109
+ You can also define your own metrics by implementing the `usearch_metric_t` interface:
110
+
111
+ ```c
112
+ simsimd_distance_t callback(void const* a, void const* b, void* state) {
113
+ // Your custom metric implementation here
114
+ }
115
+
116
+ usearch_change_metric(index, callback, NULL, usearch_metric_unknown_k, &error);
117
+ ```
118
+
119
+ You can always revert back to one of the native metrics by calling:
120
+
121
+ ```c
122
+ usearch_change_metric_kind(index, usearch_metric_cos_k, &error);
123
+ ```
124
+
125
+ ## Filtering with Predicates
126
+
127
+ Sometimes you may want to cross-reference search-results against some external database or filter them based on some criteria.
128
+ In most engines, you'd have to manually perform paging requests, successively filtering the results.
129
+ In USearch you can simply pass a predicate function to the search method, which will be applied directly during graph traversal.
130
+
131
+ ```c
132
+ int is_odd(usearch_key_t key, void* state) {
133
+ return key % 2;
134
+ }
135
+
136
+ usearch_filtered_search(
137
+ index, &vector[0], usearch_scalar_f32_k, 10,
138
+ &is_odd, NULL, // no state needed for this callback
139
+ &found_keys[0], &found_distances[0], &error);
140
+ ```
141
+
142
+ ## Extracting, Updating, and Removing Values
143
+
144
+ It is generally not recommended to use HNSW indexes in case of frequent removals or major distribution shifts.
145
+ For small updates, you can use the following methods:
146
+
147
+ ```c
148
+ float recovered_vector[dimensions];
149
+ size_t count_retrieved = usearch_get(index, 42, 1,
150
+ &recovered_vector[0], usearch_scalar_f32_k, &error);
151
+ assert(count_retrieved <= 1);
152
+
153
+ size_t count_renamed = usearch_rename(index, 42, 43, &error);
154
+ size_t count_removed = usearch_remove(index, 43, &error);
155
+ assert(count_renamed == count_removed);
156
+ ```
157
+
158
+ USearch can also be used for multi-indicies, where each key may map to multiple vectors.
159
+ That's a common case when implementing semantic search for long documents, where the entire documents can't fit into one input sequence in a transformer-model, and you have to split them into chunks.
160
+
161
+ ```c
162
+ float many_vectors[10][dimensions];
163
+ size_t count_retrieved = usearch_get(index, 42, 10,
164
+ &many_vectors[0][0], usearch_scalar_f32_k, &error);
165
+ assert(count_retrieved <= 10);
166
+ ```
167
+
168
+ ## Exact Search
169
+
170
+ USearch exposes its internal SIMD-accelerated distance functions for exact search.
171
+ To invoke them for a pair of vectors, use the following methods:
172
+
173
+ ```c
174
+ float vector_a[dimensions], vector_b[dimensions];
175
+ usearch_distance_t distance = usearch_distance(
176
+ &vector_a[0], &vector_b[0], usearch_scalar_f32_k, dimensions, usearch_metric_cos_k, &error);
177
+ ```
178
+
179
+ Alternatively, you can benefir from faster thread-pools and priority queues for parallel exact batch-search.
15
180
 
16
181
  ```c
17
- usearch_error_t error = NULL;
18
- usearch_init_options_t opts = create_options(dimensions);
19
- usearch_index_t index = usearch_init(&opts, &error);
20
- usearch_reserve(index, vectors_count, &error);
182
+ size_t threads = 0;
183
+ size_t top_k = 10;
184
+ size_t dataset_count = 1000, queries_count = 10;
185
+ simsimd_f16_t dataset[dataset_count][dimensions];
186
+ simsimd_f16_t queries[queries_count][dimensions];
21
187
 
22
- float vector[dimensions] = {...};
23
- usearch_add(index, 42, &vector[0], usearch_scalar_f32_k, &error);
188
+ usearch_key_t resulting_keys[queries_count][top_k];
189
+ usearch_distance_t resulting_distances[queries_count][top_k];
24
190
 
25
- ASSERT(!error, error);
26
- ASSERT(usearch_size(index, &error) == vectors_count, error);
27
- ASSERT(usearch_capacity(index, &error) == vectors_count, error);
191
+ usearch_exact_search(
192
+ &dataset[0][0], dataset_count, dimensions * sizeof(simsimd_f16_t),
193
+ &queries[0][0], queries_count, dimensions * sizeof(simsimd_f16_t),
194
+ usearch_scalar_f16_k, top_k, threads,
195
+ &resulting_keys[0][0], sizeof(usearch_key_t) * top_k,
196
+ &resulting_distances[0][0], sizeof(usearch_distance_t) * top_k,
197
+ &error);
198
+ ```
199
+
200
+ ## Concurrency and Parallelism
28
201
 
29
- usearch_key_t found_keys[10];
30
- float found_distances[10];
31
- size_t found_count = usearch_search(index, &vector[0], usearch_scalar_f32_k, 10, &found_keys[0], &found_distances[0], &error);
202
+ USearch manages a pool of "thread local" data-structures to avoid contention and improve performance.
203
+ By default, as many of these structures are created as there are logical cores on the machine.
204
+ Then, one can construct the index in parallel using many cores using third-party thread-pools or tools like OpenACC and OpenMP.
32
205
 
33
- usearch_free(index, &error);
206
+ ```c
207
+ #pragma omp parallel for
208
+ for (size_t i = 0; i < 1000; i++) {
209
+ usearch_add(index, i, &vector[0], usearch_scalar_f32_k, &error);
210
+ }
211
+
212
+ #pragma omp parallel for
213
+ for (size_t i = 0; i < 1000; i++) {
214
+ usearch_key_t found_keys[10];
215
+ usearch_distance_t found_distances[10];
216
+ size_t found_matches = usearch_search(index,
217
+ &vector[0], usearch_scalar_f32_k, 10,
218
+ &found_keys[0], &found_distances[0], &error);
219
+ }
34
220
  ```
221
+
222
+ ## Performance Tuning
223
+
224
+ To optimize the performance of the index, you can adjust the expansion values used during index creation and search operations.
225
+ Higher expansion values will lead to better search accuracy at the cost of slightly increased memory usage, but potentially much higher search times.
226
+ Following methods are available to adjust the expansion values:
227
+
228
+ ```c
229
+ printf("Connectivity: %zu\n", usearch_connectivity(index, &error)); // can't change
230
+ printf("Add expansion: %zu\n", usearch_expansion_add(index, &error)); // can change
231
+ printf("Search expansion: %zu\n", usearch_expansion_search(index, &error)); // can change
232
+ usearch_change_expansion_add(index, 32, &error);
233
+ usearch_change_expansion_search(index, 32, &error);
234
+ ```
235
+
236
+ Optimizing hardware utilization, you may want to check the SIMD hardware acceleration capabilities of the index and memory consumption.
237
+ The first will print the codename of the most advanced SIMD instruction set supported by the CPU and used by the index.
238
+ The second will print the memory usage of the index in bytes.
239
+
240
+ ```c
241
+ printf("Hardware acceleration: %s\n", usearch_hardware_acceleration(index, &error));
242
+ printf("Memory usage: %zu\n", usearch_memory_usage(index, &error));
243
+ ```
244
+
package/c/lib.cpp CHANGED
@@ -6,6 +6,13 @@ extern "C" {
6
6
  #include "usearch.h"
7
7
  }
8
8
 
9
+ // Check if NDEBUG is defined to determine if it's a release build
10
+ #ifdef NDEBUG
11
+ #define USEARCH_ASSERT(expression) (void)(expression)
12
+ #else
13
+ #define USEARCH_ASSERT(expression) assert(expression)
14
+ #endif
15
+
9
16
  using namespace unum::usearch;
10
17
  using namespace unum;
11
18
 
@@ -91,22 +98,38 @@ std::size_t get_(index_dense_t* index, usearch_key_t key, size_t count, void* ve
91
98
  }
92
99
  }
93
100
 
94
- search_result_t search_(index_dense_t* index, void const* vector, scalar_kind_t kind, size_t n) {
101
+ template <typename predicate_at = dummy_predicate_t>
102
+ search_result_t search_(index_dense_t* index, void const* vector, scalar_kind_t kind, size_t n,
103
+ predicate_at&& predicate = predicate_at{}) {
95
104
  switch (kind) {
96
- case scalar_kind_t::f32_k: return index->search((f32_t const*)vector, n);
97
- case scalar_kind_t::f64_k: return index->search((f64_t const*)vector, n);
98
- case scalar_kind_t::f16_k: return index->search((f16_t const*)vector, n);
99
- case scalar_kind_t::i8_k: return index->search((i8_t const*)vector, n);
100
- case scalar_kind_t::b1x8_k: return index->search((b1x8_t const*)vector, n);
105
+ case scalar_kind_t::f32_k:
106
+ return index->filtered_search((f32_t const*)vector, n, std::forward<predicate_at>(predicate));
107
+ case scalar_kind_t::f64_k:
108
+ return index->filtered_search((f64_t const*)vector, n, std::forward<predicate_at>(predicate));
109
+ case scalar_kind_t::f16_k:
110
+ return index->filtered_search((f16_t const*)vector, n, std::forward<predicate_at>(predicate));
111
+ case scalar_kind_t::i8_k:
112
+ return index->filtered_search((i8_t const*)vector, n, std::forward<predicate_at>(predicate));
113
+ case scalar_kind_t::b1x8_k:
114
+ return index->filtered_search((b1x8_t const*)vector, n, std::forward<predicate_at>(predicate));
101
115
  default: return search_result_t().failed("Unknown scalar kind!");
102
116
  }
103
117
  }
104
118
 
105
119
  extern "C" {
106
120
 
121
+ USEARCH_EXPORT char const* usearch_version(void) {
122
+ int major = USEARCH_VERSION_MAJOR;
123
+ int minor = USEARCH_VERSION_MINOR;
124
+ int patch = USEARCH_VERSION_PATCH;
125
+ static char version[32];
126
+ sprintf(version, "%d.%d.%d", major, minor, patch);
127
+ return version;
128
+ }
129
+
107
130
  USEARCH_EXPORT usearch_index_t usearch_init(usearch_init_options_t* options, usearch_error_t* error) {
108
131
 
109
- assert(options && error && "Missing arguments");
132
+ USEARCH_ASSERT(options && error && "Missing arguments");
110
133
 
111
134
  index_dense_config_t config(options->connectivity, options->expansion_add, options->expansion_search);
112
135
  config.multi = options->multi;
@@ -114,11 +137,11 @@ USEARCH_EXPORT usearch_index_t usearch_init(usearch_init_options_t* options, use
114
137
  scalar_kind_t scalar_kind = scalar_kind_to_cpp(options->quantization);
115
138
 
116
139
  metric_punned_t metric = //
117
- !options->metric ? metric_punned_t(options->dimensions, metric_kind, scalar_kind)
118
- : metric_punned_t(options->dimensions, //
119
- reinterpret_cast<std::uintptr_t>(options->metric), //
120
- metric_punned_signature_t::array_array_k, //
121
- metric_kind, scalar_kind);
140
+ !options->metric ? metric_punned_t::builtin(options->dimensions, metric_kind, scalar_kind)
141
+ : metric_punned_t::stateless(options->dimensions, //
142
+ reinterpret_cast<std::uintptr_t>(options->metric), //
143
+ metric_punned_signature_t::array_array_k, //
144
+ metric_kind, scalar_kind);
122
145
  if (metric.missing()) {
123
146
  *error = "Unknown metric kind!";
124
147
  return NULL;
@@ -136,13 +159,13 @@ USEARCH_EXPORT void usearch_free(usearch_index_t index, usearch_error_t*) {
136
159
  }
137
160
 
138
161
  USEARCH_EXPORT size_t usearch_serialized_length(usearch_index_t index, usearch_error_t*) {
139
- assert(index && "Missing arguments");
162
+ USEARCH_ASSERT(index && "Missing arguments");
140
163
  return reinterpret_cast<index_dense_t*>(index)->serialized_length();
141
164
  }
142
165
 
143
166
  USEARCH_EXPORT void usearch_save(usearch_index_t index, char const* path, usearch_error_t* error) {
144
167
 
145
- assert(index && path && error && "Missing arguments");
168
+ USEARCH_ASSERT(index && path && error && "Missing arguments");
146
169
  serialization_result_t result = reinterpret_cast<index_dense_t*>(index)->save(path);
147
170
  if (!result)
148
171
  *error = result.error.release();
@@ -150,7 +173,7 @@ USEARCH_EXPORT void usearch_save(usearch_index_t index, char const* path, usearc
150
173
 
151
174
  USEARCH_EXPORT void usearch_load(usearch_index_t index, char const* path, usearch_error_t* error) {
152
175
 
153
- assert(index && path && error && "Missing arguments");
176
+ USEARCH_ASSERT(index && path && error && "Missing arguments");
154
177
  serialization_result_t result = reinterpret_cast<index_dense_t*>(index)->load(path);
155
178
  if (!result)
156
179
  *error = result.error.release();
@@ -158,7 +181,7 @@ USEARCH_EXPORT void usearch_load(usearch_index_t index, char const* path, usearc
158
181
 
159
182
  USEARCH_EXPORT void usearch_view(usearch_index_t index, char const* path, usearch_error_t* error) {
160
183
 
161
- assert(index && path && error && "Missing arguments");
184
+ USEARCH_ASSERT(index && path && error && "Missing arguments");
162
185
  serialization_result_t result = reinterpret_cast<index_dense_t*>(index)->view(path);
163
186
  if (!result)
164
187
  *error = result.error.release();
@@ -166,7 +189,7 @@ USEARCH_EXPORT void usearch_view(usearch_index_t index, char const* path, usearc
166
189
 
167
190
  USEARCH_EXPORT void usearch_metadata(char const* path, usearch_init_options_t* options, usearch_error_t* error) {
168
191
 
169
- assert(path && options && error && "Missing arguments");
192
+ USEARCH_ASSERT(path && options && error && "Missing arguments");
170
193
  index_dense_metadata_result_t result = index_dense_metadata_from_path(path);
171
194
  if (!result)
172
195
  *error = result.error.release();
@@ -184,7 +207,7 @@ USEARCH_EXPORT void usearch_metadata(char const* path, usearch_init_options_t* o
184
207
 
185
208
  USEARCH_EXPORT void usearch_save_buffer(usearch_index_t index, void* buffer, size_t length, usearch_error_t* error) {
186
209
 
187
- assert(index && buffer && length && error && "Missing arguments");
210
+ USEARCH_ASSERT(index && buffer && length && error && "Missing arguments");
188
211
  memory_mapped_file_t memory_map((byte_t*)buffer, length);
189
212
  serialization_result_t result = reinterpret_cast<index_dense_t*>(index)->save(std::move(memory_map));
190
213
  if (!result)
@@ -194,7 +217,7 @@ USEARCH_EXPORT void usearch_save_buffer(usearch_index_t index, void* buffer, siz
194
217
  USEARCH_EXPORT void usearch_load_buffer(usearch_index_t index, void const* buffer, size_t length,
195
218
  usearch_error_t* error) {
196
219
 
197
- assert(index && buffer && length && error && "Missing arguments");
220
+ USEARCH_ASSERT(index && buffer && length && error && "Missing arguments");
198
221
  memory_mapped_file_t memory_map((byte_t*)buffer, length);
199
222
  serialization_result_t result = reinterpret_cast<index_dense_t*>(index)->load(std::move(memory_map));
200
223
  if (!result)
@@ -204,7 +227,7 @@ USEARCH_EXPORT void usearch_load_buffer(usearch_index_t index, void const* buffe
204
227
  USEARCH_EXPORT void usearch_view_buffer(usearch_index_t index, void const* buffer, size_t length,
205
228
  usearch_error_t* error) {
206
229
 
207
- assert(index && buffer && length && error && "Missing arguments");
230
+ USEARCH_ASSERT(index && buffer && length && error && "Missing arguments");
208
231
  memory_mapped_file_t memory_map((byte_t*)buffer, length);
209
232
  serialization_result_t result = reinterpret_cast<index_dense_t*>(index)->view(std::move(memory_map));
210
233
  if (!result)
@@ -214,7 +237,7 @@ USEARCH_EXPORT void usearch_view_buffer(usearch_index_t index, void const* buffe
214
237
  USEARCH_EXPORT void usearch_metadata_buffer(void const* buffer, size_t length, usearch_init_options_t* options,
215
238
  usearch_error_t* error) {
216
239
 
217
- assert(buffer && length && options && error && "Missing arguments");
240
+ USEARCH_ASSERT(buffer && length && options && error && "Missing arguments");
218
241
  index_dense_metadata_result_t result =
219
242
  index_dense_metadata_from_buffer(memory_mapped_file_t((byte_t*)(buffer), length));
220
243
  if (!result)
@@ -231,24 +254,80 @@ USEARCH_EXPORT void usearch_metadata_buffer(void const* buffer, size_t length, u
231
254
  options->metric = NULL;
232
255
  }
233
256
 
234
- USEARCH_EXPORT size_t usearch_size(usearch_index_t index, usearch_error_t*) { //
257
+ USEARCH_EXPORT size_t usearch_size(usearch_index_t index, usearch_error_t* error) {
258
+ USEARCH_ASSERT(index && error && "Missing arguments");
235
259
  return reinterpret_cast<index_dense_t*>(index)->size();
236
260
  }
237
261
 
238
- USEARCH_EXPORT size_t usearch_capacity(usearch_index_t index, usearch_error_t*) {
262
+ USEARCH_EXPORT size_t usearch_capacity(usearch_index_t index, usearch_error_t* error) {
263
+ USEARCH_ASSERT(index && error && "Missing arguments");
239
264
  return reinterpret_cast<index_dense_t*>(index)->capacity();
240
265
  }
241
266
 
242
- USEARCH_EXPORT size_t usearch_dimensions(usearch_index_t index, usearch_error_t*) {
267
+ USEARCH_EXPORT size_t usearch_dimensions(usearch_index_t index, usearch_error_t* error) {
268
+ USEARCH_ASSERT(index && error && "Missing arguments");
243
269
  return reinterpret_cast<index_dense_t*>(index)->dimensions();
244
270
  }
245
271
 
246
- USEARCH_EXPORT size_t usearch_connectivity(usearch_index_t index, usearch_error_t*) {
272
+ USEARCH_EXPORT size_t usearch_connectivity(usearch_index_t index, usearch_error_t* error) {
273
+ USEARCH_ASSERT(index && error && "Missing arguments");
247
274
  return reinterpret_cast<index_dense_t*>(index)->connectivity();
248
275
  }
249
276
 
277
+ USEARCH_EXPORT size_t usearch_expansion_add(usearch_index_t index, usearch_error_t* error) {
278
+ USEARCH_ASSERT(index && error && "Missing arguments");
279
+ return reinterpret_cast<index_dense_t*>(index)->expansion_add();
280
+ }
281
+
282
+ USEARCH_EXPORT size_t usearch_expansion_search(usearch_index_t index, usearch_error_t* error) {
283
+ USEARCH_ASSERT(index && error && "Missing arguments");
284
+ return reinterpret_cast<index_dense_t*>(index)->expansion_search();
285
+ }
286
+
287
+ USEARCH_EXPORT size_t usearch_memory_usage(usearch_index_t index, usearch_error_t* error) {
288
+ USEARCH_ASSERT(index && error && "Missing arguments");
289
+ return reinterpret_cast<index_dense_t*>(index)->memory_usage();
290
+ }
291
+
292
+ USEARCH_EXPORT char const* usearch_hardware_acceleration(usearch_index_t index, usearch_error_t* error) {
293
+ USEARCH_ASSERT(index && error && "Missing arguments");
294
+ return reinterpret_cast<index_dense_t*>(index)->metric().isa_name();
295
+ }
296
+
297
+ USEARCH_EXPORT void usearch_change_expansion_add(usearch_index_t index, size_t expansion, usearch_error_t* error) {
298
+ USEARCH_ASSERT(index && error && "Missing arguments");
299
+ reinterpret_cast<index_dense_t*>(index)->change_expansion_add(expansion);
300
+ }
301
+
302
+ USEARCH_EXPORT void usearch_change_expansion_search(usearch_index_t index, size_t expansion, usearch_error_t* error) {
303
+ USEARCH_ASSERT(index && error && "Missing arguments");
304
+ reinterpret_cast<index_dense_t*>(index)->change_expansion_search(expansion);
305
+ }
306
+
307
+ USEARCH_EXPORT void usearch_change_metric_kind(usearch_index_t index, usearch_metric_kind_t kind,
308
+ usearch_error_t* error) {
309
+ USEARCH_ASSERT(index && error && "Missing arguments");
310
+ auto& index_dense = *reinterpret_cast<index_dense_t*>(index);
311
+ index_dense.change_metric(
312
+ metric_punned_t::builtin(index_dense.dimensions(), metric_kind_to_cpp(kind), index_dense.scalar_kind()));
313
+ }
314
+
315
+ USEARCH_EXPORT void usearch_change_metric(usearch_index_t index, usearch_metric_t metric, void* state,
316
+ usearch_metric_kind_t kind, usearch_error_t* error) {
317
+ USEARCH_ASSERT(index && error && "Missing arguments");
318
+ auto& index_dense = *reinterpret_cast<index_dense_t*>(index);
319
+ auto metric_punned =
320
+ state ? metric_punned_t::statefull(reinterpret_cast<std::uintptr_t>(metric),
321
+ reinterpret_cast<std::uintptr_t>(state), metric_kind_to_cpp(kind),
322
+ index_dense.scalar_kind())
323
+ : metric_punned_t::stateless(index_dense.dimensions(), reinterpret_cast<std::uintptr_t>(metric),
324
+ metric_punned_signature_t::array_array_k, metric_kind_to_cpp(kind),
325
+ index_dense.scalar_kind());
326
+ index_dense.change_metric(std::move(metric_punned));
327
+ }
328
+
250
329
  USEARCH_EXPORT void usearch_reserve(usearch_index_t index, size_t capacity, usearch_error_t* error) {
251
- assert(index && error && "Missing arguments");
330
+ USEARCH_ASSERT(index && error && "Missing arguments");
252
331
  if (!reinterpret_cast<index_dense_t*>(index)->reserve(capacity))
253
332
  *error = "Out of memory!";
254
333
  }
@@ -257,29 +336,47 @@ USEARCH_EXPORT void usearch_add(
257
336
  usearch_index_t index, usearch_key_t key, void const* vector, usearch_scalar_kind_t kind, //
258
337
  usearch_error_t* error) {
259
338
 
260
- assert(index && vector && error && "Missing arguments");
339
+ USEARCH_ASSERT(index && vector && error && "Missing arguments");
261
340
  add_result_t result = add_(reinterpret_cast<index_dense_t*>(index), key, vector, scalar_kind_to_cpp(kind));
262
341
  if (!result)
263
342
  *error = result.error.release();
264
343
  }
265
344
 
266
345
  USEARCH_EXPORT bool usearch_contains(usearch_index_t index, usearch_key_t key, usearch_error_t*) {
267
- assert(index && "Missing arguments");
346
+ USEARCH_ASSERT(index && "Missing arguments");
268
347
  return reinterpret_cast<index_dense_t*>(index)->contains(key);
269
348
  }
270
349
 
271
350
  USEARCH_EXPORT size_t usearch_count(usearch_index_t index, usearch_key_t key, usearch_error_t*) {
272
- assert(index && "Missing arguments");
351
+ USEARCH_ASSERT(index && "Missing arguments");
273
352
  return reinterpret_cast<index_dense_t*>(index)->count(key);
274
353
  }
275
354
 
276
- USEARCH_EXPORT size_t usearch_search( //
277
- usearch_index_t index, void const* vector, usearch_scalar_kind_t kind, size_t results_limit, //
355
+ USEARCH_EXPORT size_t usearch_search( //
356
+ usearch_index_t index, void const* query, usearch_scalar_kind_t query_kind, size_t results_limit, //
357
+ usearch_key_t* found_keys, usearch_distance_t* found_distances, usearch_error_t* error) {
358
+
359
+ USEARCH_ASSERT(index && query && error && "Missing arguments");
360
+ search_result_t result =
361
+ search_(reinterpret_cast<index_dense_t*>(index), query, scalar_kind_to_cpp(query_kind), results_limit);
362
+ if (!result) {
363
+ *error = result.error.release();
364
+ return 0;
365
+ }
366
+
367
+ return result.dump_to(found_keys, found_distances);
368
+ }
369
+
370
+ USEARCH_EXPORT size_t usearch_filtered_search( //
371
+ usearch_index_t index, //
372
+ void const* query, usearch_scalar_kind_t query_kind, size_t results_limit, //
373
+ int (*filter)(usearch_key_t key, void* filter_state), void* filter_state, //
278
374
  usearch_key_t* found_keys, usearch_distance_t* found_distances, usearch_error_t* error) {
279
375
 
280
- assert(index && vector && error && "Missing arguments");
376
+ USEARCH_ASSERT(index && query && filter && error && "Missing arguments");
281
377
  search_result_t result =
282
- search_(reinterpret_cast<index_dense_t*>(index), vector, scalar_kind_to_cpp(kind), results_limit);
378
+ search_(reinterpret_cast<index_dense_t*>(index), query, scalar_kind_to_cpp(query_kind), results_limit,
379
+ [=](usearch_key_t key) noexcept { return filter(key, filter_state); });
283
380
  if (!result) {
284
381
  *error = result.error.release();
285
382
  return 0;
@@ -292,13 +389,13 @@ USEARCH_EXPORT size_t usearch_get( //
292
389
  usearch_index_t index, usearch_key_t key, size_t count, //
293
390
  void* vectors, usearch_scalar_kind_t kind, usearch_error_t*) {
294
391
 
295
- assert(index && vectors);
392
+ USEARCH_ASSERT(index && vectors);
296
393
  return get_(reinterpret_cast<index_dense_t*>(index), key, count, vectors, scalar_kind_to_cpp(kind));
297
394
  }
298
395
 
299
396
  USEARCH_EXPORT size_t usearch_remove(usearch_index_t index, usearch_key_t key, usearch_error_t* error) {
300
397
 
301
- assert(index && error && "Missing arguments");
398
+ USEARCH_ASSERT(index && error && "Missing arguments");
302
399
  labeling_result_t result = reinterpret_cast<index_dense_t*>(index)->remove(key);
303
400
  if (!result)
304
401
  *error = result.error.release();
@@ -308,7 +405,7 @@ USEARCH_EXPORT size_t usearch_remove(usearch_index_t index, usearch_key_t key, u
308
405
  USEARCH_EXPORT size_t usearch_rename( //
309
406
  usearch_index_t index, usearch_key_t from, usearch_key_t to, usearch_error_t* error) {
310
407
 
311
- assert(index && error && "Missing arguments");
408
+ USEARCH_ASSERT(index && error && "Missing arguments");
312
409
  labeling_result_t result = reinterpret_cast<index_dense_t*>(index)->rename(from, to);
313
410
  if (!result)
314
411
  *error = result.error.release();
@@ -334,7 +431,7 @@ USEARCH_EXPORT void usearch_exact_search( //
334
431
  usearch_distance_t* distances, size_t distances_stride, //
335
432
  usearch_error_t* error) {
336
433
 
337
- assert(dataset && queries && keys && distances && error && "Missing arguments");
434
+ USEARCH_ASSERT(dataset && queries && keys && distances && error && "Missing arguments");
338
435
 
339
436
  metric_punned_t metric(dimensions, metric_kind_to_cpp(metric_kind), scalar_kind_to_cpp(scalar_kind));
340
437
  executor_default_t executor(threads);
package/c/test.c CHANGED
@@ -72,6 +72,8 @@ void test_init(size_t const collection_size, size_t const dimensions) {
72
72
  ASSERT(usearch_capacity(index, &error) == collection_size, error);
73
73
  ASSERT(usearch_dimensions(index, &error) == dimensions, error);
74
74
  ASSERT(usearch_connectivity(index, &error) == opts.connectivity, error);
75
+ ASSERT(usearch_hardware_acceleration(index, &error), error);
76
+ ASSERT(usearch_memory_usage(index, &error), error);
75
77
 
76
78
  usearch_free(index, &error);
77
79
  ASSERT(!error, error);
@@ -361,6 +363,8 @@ void test_view(size_t const collection_size, size_t const dimensions) {
361
363
  }
362
364
 
363
365
  int main(int argc, char const* argv[]) {
366
+ printf("Running tests...\n");
367
+ printf("USearch version: %s\n", usearch_version());
364
368
 
365
369
  size_t collection_sizes[] = {11, 512};
366
370
  size_t dimensions[] = {83, 2}; // Not all distance functions make sense for 1 dimensional data
package/c/usearch.h CHANGED
@@ -97,6 +97,12 @@ USEARCH_EXPORT typedef struct usearch_init_options_t {
97
97
  bool multi;
98
98
  } usearch_init_options_t;
99
99
 
100
+ /**
101
+ * @brief Retrieves the version of the library.
102
+ * @return The version of the library.
103
+ */
104
+ USEARCH_EXPORT char const* usearch_version(void);
105
+
100
106
  /**
101
107
  * @brief Initializes a new instance of the index.
102
108
  * @param options Pointer to the `usearch_init_options_t` structure containing initialization options.
@@ -112,6 +118,22 @@ USEARCH_EXPORT usearch_index_t usearch_init(usearch_init_options_t* options, use
112
118
  */
113
119
  USEARCH_EXPORT void usearch_free(usearch_index_t index, usearch_error_t* error);
114
120
 
121
+ /**
122
+ * @brief Reports the memory usage of the index.
123
+ * @param[in] index The handle to the USearch index to be queried.
124
+ * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
125
+ * @return Number of bytes used by the index.
126
+ */
127
+ USEARCH_EXPORT size_t usearch_memory_usage(usearch_index_t index, usearch_error_t* error);
128
+
129
+ /**
130
+ * @brief Reports the SIMD capabilities used by the index on the current CPU.
131
+ * @param[in] index The handle to the USearch index to be queried.
132
+ * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
133
+ * @return The codename of the SIMD instruction set used by the index.
134
+ */
135
+ USEARCH_EXPORT char const* usearch_hardware_acceleration(usearch_index_t index, usearch_error_t* error);
136
+
115
137
  /**
116
138
  * @brief Reports expected file size after serialization.
117
139
  * @param[in] index The handle to the USearch index to be serialized.
@@ -227,6 +249,58 @@ USEARCH_EXPORT size_t usearch_connectivity(usearch_index_t index, usearch_error_
227
249
  */
228
250
  USEARCH_EXPORT void usearch_reserve(usearch_index_t index, size_t capacity, usearch_error_t* error);
229
251
 
252
+ /**
253
+ * @brief Retrieves the expansion value used during index creation.
254
+ * @param[in] index The handle to the USearch index to be queried.
255
+ * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
256
+ * @return The expansion value used during index creation.
257
+ */
258
+ USEARCH_EXPORT size_t usearch_expansion_add(usearch_index_t index, usearch_error_t* error);
259
+
260
+ /**
261
+ * @brief Retrieves the expansion value used during search.
262
+ * @param[in] index The handle to the USearch index to be queried.
263
+ * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
264
+ * @return The expansion value used during search.
265
+ */
266
+ USEARCH_EXPORT size_t usearch_expansion_search(usearch_index_t index, usearch_error_t* error);
267
+
268
+ /**
269
+ * @brief Updates the expansion value used during index creation. Rarely used.
270
+ * @param[in] index The handle to the USearch index to be queried.
271
+ * @param[in] expansion The new expansion value.
272
+ * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
273
+ */
274
+ USEARCH_EXPORT void usearch_change_expansion_add(usearch_index_t index, size_t expansion, usearch_error_t* error);
275
+
276
+ /**
277
+ * @brief Updates the expansion value used during search. Rarely used.
278
+ * @param[in] index The handle to the USearch index to be queried.
279
+ * @param[in] expansion The new expansion value.
280
+ * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
281
+ */
282
+ USEARCH_EXPORT void usearch_change_expansion_search(usearch_index_t index, size_t expansion, usearch_error_t* error);
283
+
284
+ /**
285
+ * @brief Updates the metric kind used for distance calculation between vectors.
286
+ * @param[in] index The handle to the USearch index to be queried.
287
+ * @param[in] kind The metric kind used for distance calculation between vectors.
288
+ * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
289
+ */
290
+ USEARCH_EXPORT void usearch_change_metric_kind(usearch_index_t index, usearch_metric_kind_t kind,
291
+ usearch_error_t* error);
292
+
293
+ /**
294
+ * @brief Updates the custom metric function used for distance calculation between vectors.
295
+ * @param[in] index The handle to the USearch index to be queried.
296
+ * @param[in] metric The custom metric function used for distance calculation between vectors.
297
+ * @param[in] state The @b optional state pointer to be passed to the custom metric function.
298
+ * @param[in] kind The metric kind used for distance calculation between vectors. Needed for serialization.
299
+ * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
300
+ */
301
+ USEARCH_EXPORT void usearch_change_metric(usearch_index_t index, usearch_metric_t metric, void* state,
302
+ usearch_metric_kind_t kind, usearch_error_t* error);
303
+
230
304
  /**
231
305
  * @brief Adds a vector with a key to the index.
232
306
  * @param[inout] index The handle to the USearch index to be populated.
@@ -268,10 +342,31 @@ USEARCH_EXPORT size_t usearch_count(usearch_index_t index, usearch_key_t, usearc
268
342
  * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
269
343
  * @return Number of found matches.
270
344
  */
271
- USEARCH_EXPORT size_t usearch_search( //
272
- usearch_index_t index, //
273
- void const* query_vector, usearch_scalar_kind_t query_kind, //
274
- size_t count, usearch_key_t* keys, usearch_distance_t* distances, usearch_error_t* error);
345
+ USEARCH_EXPORT size_t usearch_search( //
346
+ usearch_index_t index, //
347
+ void const* query_vector, usearch_scalar_kind_t query_kind, size_t count, //
348
+ usearch_key_t* keys, usearch_distance_t* distances, usearch_error_t* error);
349
+
350
+ /**
351
+ * @brief Performs k-Approximate Nearest Neighbors (kANN) Search for closest vectors to query,
352
+ * predicated on a custom function that returns `true` for vectors to be included.
353
+ *
354
+ * @param[in] index The handle to the USearch index to be queried.
355
+ * @param[in] query_vector Pointer to the query vector data.
356
+ * @param[in] query_kind The scalar type used in the query vector data.
357
+ * @param[in] count Upper bound on the number of neighbors to search, the "k" in "kANN".
358
+ * @param[in] filter The custom filter function that returns `true` for vectors to be included.
359
+ * @param[in] filter_state The @b optional state pointer to be passed to the custom filter function.
360
+ * @param[out] keys Output buffer for up to `count` nearest neighbors keys.
361
+ * @param[out] distances Output buffer for up to `count` distances to nearest neighbors.
362
+ * @param[out] error Pointer to a string where the error message will be stored, if an error occurs.
363
+ * @return Number of found matches.
364
+ */
365
+ USEARCH_EXPORT size_t usearch_filtered_search( //
366
+ usearch_index_t index, //
367
+ void const* query_vector, usearch_scalar_kind_t query_kind, size_t count, //
368
+ int (*filter)(usearch_key_t key, void* filter_state), void* filter_state, //
369
+ usearch_key_t* keys, usearch_distance_t* distances, usearch_error_t* error);
275
370
 
276
371
  /**
277
372
  * @brief Retrieves the vector associated with the given key from the index.
package/cpp/test.cpp CHANGED
@@ -65,6 +65,17 @@ void test_cosine(index_at& index, std::vector<std::vector<scalar_at>> const& vec
65
65
 
66
66
  // Perform exact search
67
67
  matched_count = index.search(vector_first, 5, args...).dump_to(matched_keys, matched_distances);
68
+ expect(matched_count != 0);
69
+
70
+ // Perform filtered exact search, keeping only odd values
71
+ if constexpr (punned_ak) {
72
+ auto is_odd = [](vector_key_t key) -> bool { return (key & 1) != 0; };
73
+ matched_count =
74
+ index.filtered_search(vector_first, 5, is_odd, args...).dump_to(matched_keys, matched_distances);
75
+ expect(matched_count != 0);
76
+ for (std::size_t i = 0; i < matched_count; i++)
77
+ expect(is_odd(matched_keys[i]));
78
+ }
68
79
 
69
80
  // Validate scans
70
81
  std::size_t count = 0;
@@ -10,8 +10,8 @@
10
10
  #define UNUM_USEARCH_HPP
11
11
 
12
12
  #define USEARCH_VERSION_MAJOR 2
13
- #define USEARCH_VERSION_MINOR 10
14
- #define USEARCH_VERSION_PATCH 5
13
+ #define USEARCH_VERSION_MINOR 11
14
+ #define USEARCH_VERSION_PATCH 0
15
15
 
16
16
  // Inferring C++ version
17
17
  // https://stackoverflow.com/a/61552074
@@ -3497,9 +3497,12 @@ class index_gt {
3497
3497
 
3498
3498
  distance_t radius = context.measure(query, citerator_at(start_slot), metric);
3499
3499
  next.insert_reserved({-radius, static_cast<compressed_slot_t>(start_slot)});
3500
- top.insert_reserved({radius, static_cast<compressed_slot_t>(start_slot)});
3501
3500
  visits.set(static_cast<compressed_slot_t>(start_slot));
3502
3501
 
3502
+ // Don't populate the top list if the predicate is not satisfied
3503
+ if (is_dummy<predicate_at>() || predicate(member_cref_t{node_at_(start_slot).ckey(), start_slot}))
3504
+ top.insert_reserved({radius, static_cast<compressed_slot_t>(start_slot)});
3505
+
3503
3506
  while (!next.empty()) {
3504
3507
 
3505
3508
  candidate_t candidate = next.top();
@@ -3529,12 +3532,9 @@ class index_gt {
3529
3532
  if (top.size() < top_limit || successor_dist < radius) {
3530
3533
  // This can substantially grow our priority queue:
3531
3534
  next.insert({-successor_dist, successor_slot});
3532
- if (!is_dummy<predicate_at>())
3533
- if (!predicate(member_cref_t{node_at_(successor_slot).ckey(), successor_slot}))
3534
- continue;
3535
-
3536
- // This will automatically evict poor matches:
3537
- top.insert({successor_dist, successor_slot}, top_limit);
3535
+ if (is_dummy<predicate_at>() ||
3536
+ predicate(member_cref_t{node_at_(successor_slot).ckey(), successor_slot}))
3537
+ top.insert({successor_dist, successor_slot}, top_limit);
3538
3538
  radius = top.top().distance;
3539
3539
  }
3540
3540
  }
@@ -604,11 +604,17 @@ class index_dense_gt {
604
604
  add_result_t add(vector_key_t key, f32_t const* vector, std::size_t thread = any_thread(), bool force_vector_copy = true) { return add_(key, vector, thread, force_vector_copy, casts_.from_f32); }
605
605
  add_result_t add(vector_key_t key, f64_t const* vector, std::size_t thread = any_thread(), bool force_vector_copy = true) { return add_(key, vector, thread, force_vector_copy, casts_.from_f64); }
606
606
 
607
- search_result_t search(b1x8_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_b1x8); }
608
- search_result_t search(i8_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_i8); }
609
- search_result_t search(f16_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_f16); }
610
- search_result_t search(f32_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_f32); }
611
- search_result_t search(f64_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, thread, exact, casts_.from_f64); }
607
+ search_result_t search(b1x8_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, dummy_predicate_t {}, thread, exact, casts_.from_b1x8); }
608
+ search_result_t search(i8_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, dummy_predicate_t {}, thread, exact, casts_.from_i8); }
609
+ search_result_t search(f16_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, dummy_predicate_t {}, thread, exact, casts_.from_f16); }
610
+ search_result_t search(f32_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, dummy_predicate_t {}, thread, exact, casts_.from_f32); }
611
+ search_result_t search(f64_t const* vector, std::size_t wanted, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, dummy_predicate_t {}, thread, exact, casts_.from_f64); }
612
+
613
+ template <typename predicate_at> search_result_t filtered_search(b1x8_t const* vector, std::size_t wanted, predicate_at&& predicate, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, std::forward<predicate_at>(predicate), thread, exact, casts_.from_b1x8); }
614
+ template <typename predicate_at> search_result_t filtered_search(i8_t const* vector, std::size_t wanted, predicate_at&& predicate, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, std::forward<predicate_at>(predicate), thread, exact, casts_.from_i8); }
615
+ template <typename predicate_at> search_result_t filtered_search(f16_t const* vector, std::size_t wanted, predicate_at&& predicate, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, std::forward<predicate_at>(predicate), thread, exact, casts_.from_f16); }
616
+ template <typename predicate_at> search_result_t filtered_search(f32_t const* vector, std::size_t wanted, predicate_at&& predicate, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, std::forward<predicate_at>(predicate), thread, exact, casts_.from_f32); }
617
+ template <typename predicate_at> search_result_t filtered_search(f64_t const* vector, std::size_t wanted, predicate_at&& predicate, std::size_t thread = any_thread(), bool exact = false) const { return search_(vector, wanted, std::forward<predicate_at>(predicate), thread, exact, casts_.from_f64); }
612
618
 
613
619
  std::size_t get(vector_key_t key, b1x8_t* vector, std::size_t vectors_count = 1) const { return get_(key, vector, vectors_count, casts_.to_b1x8); }
614
620
  std::size_t get(vector_key_t key, i8_t* vector, std::size_t vectors_count = 1) const { return get_(key, vector, vectors_count, casts_.to_i8); }
@@ -708,7 +714,9 @@ class index_dense_gt {
708
714
  cluster_config.thread = lock.thread_id;
709
715
  cluster_config.expansion = config_.expansion_search;
710
716
  metric_proxy_t metric{*this};
711
- auto allow = [=](member_cref_t const& member) noexcept { return member.key != free_key_; };
717
+ auto allow = [free_key_ = this->free_key_](member_cref_t const& member) noexcept {
718
+ return member.key != free_key_;
719
+ };
712
720
 
713
721
  // Find the closest cluster for any vector under that key.
714
722
  while (key_range.first != key_range.second) {
@@ -930,7 +938,7 @@ class index_dense_gt {
930
938
  return result.failed("Slot type doesn't match, consider rebuilding");
931
939
 
932
940
  config_.multi = head.multi;
933
- metric_ = metric_t(head.dimensions, head.kind_metric, head.kind_scalar);
941
+ metric_ = metric_t::builtin(head.dimensions, head.kind_metric, head.kind_scalar);
934
942
  cast_buffer_.resize(available_threads_.size() * metric_.bytes_per_vector());
935
943
  casts_ = make_casts_(head.kind_scalar);
936
944
  }
@@ -1016,7 +1024,7 @@ class index_dense_gt {
1016
1024
  return result.failed("Slot type doesn't match, consider rebuilding");
1017
1025
 
1018
1026
  config_.multi = head.multi;
1019
- metric_ = metric_t(head.dimensions, head.kind_metric, head.kind_scalar);
1027
+ metric_ = metric_t::builtin(head.dimensions, head.kind_metric, head.kind_scalar);
1020
1028
  cast_buffer_.resize(available_threads_.size() * metric_.bytes_per_vector());
1021
1029
  casts_ = make_casts_(head.kind_scalar);
1022
1030
  offset += sizeof(buffer);
@@ -1767,10 +1775,9 @@ class index_dense_gt {
1767
1775
  : typed_->add(key, vector_data, metric, update_config, on_success);
1768
1776
  }
1769
1777
 
1770
- template <typename scalar_at>
1771
- search_result_t search_( //
1772
- scalar_at const* vector, std::size_t wanted, //
1773
- std::size_t thread, bool exact, cast_t const& cast) const {
1778
+ template <typename scalar_at, typename predicate_at>
1779
+ search_result_t search_(scalar_at const* vector, std::size_t wanted, predicate_at&& predicate, std::size_t thread,
1780
+ bool exact, cast_t const& cast) const {
1774
1781
 
1775
1782
  // Cast the vector, if needed for compatibility with `metric_`
1776
1783
  thread_lock_t lock = thread_lock_(thread);
@@ -1787,8 +1794,17 @@ class index_dense_gt {
1787
1794
  search_config.expansion = config_.expansion_search;
1788
1795
  search_config.exact = exact;
1789
1796
 
1790
- auto allow = [=](member_cref_t const& member) noexcept { return member.key != free_key_; };
1791
- return typed_->search(vector_data, wanted, metric_proxy_t{*this}, search_config, allow);
1797
+ if (std::is_same<typename std::decay<predicate_at>::type, dummy_predicate_t>::value) {
1798
+ auto allow = [free_key_ = this->free_key_](member_cref_t const& member) noexcept {
1799
+ return member.key != free_key_;
1800
+ };
1801
+ return typed_->search(vector_data, wanted, metric_proxy_t{*this}, search_config, allow);
1802
+ } else {
1803
+ auto allow = [free_key_ = this->free_key_, &predicate](member_cref_t const& member) noexcept {
1804
+ return member.key != free_key_ && predicate(member.key);
1805
+ };
1806
+ return typed_->search(vector_data, wanted, metric_proxy_t{*this}, search_config, allow);
1807
+ }
1792
1808
  }
1793
1809
 
1794
1810
  template <typename scalar_at>
@@ -1810,7 +1826,9 @@ class index_dense_gt {
1810
1826
  cluster_config.thread = lock.thread_id;
1811
1827
  cluster_config.expansion = config_.expansion_search;
1812
1828
 
1813
- auto allow = [=](member_cref_t const& member) noexcept { return member.key != free_key_; };
1829
+ auto allow = [free_key_ = this->free_key_](member_cref_t const& member) noexcept {
1830
+ return member.key != free_key_;
1831
+ };
1814
1832
  return typed_->cluster(vector_data, level, metric_proxy_t{*this}, cluster_config, allow);
1815
1833
  }
1816
1834
 
@@ -1320,6 +1320,7 @@ using span_punned_t = span_gt<byte_t const>;
1320
1320
  enum class metric_punned_signature_t {
1321
1321
  array_array_k = 0,
1322
1322
  array_array_size_k,
1323
+ array_array_state_k,
1323
1324
  };
1324
1325
 
1325
1326
  /**
@@ -1339,12 +1340,14 @@ class metric_punned_t {
1339
1340
  using metric_array_array_t = result_t (*)(uptr_t, uptr_t);
1340
1341
  /// Distance function that takes two arrays and their length and returns a scalar.
1341
1342
  using metric_array_array_size_t = result_t (*)(uptr_t, uptr_t, uptr_t);
1343
+ /// Distance function that takes two arrays and some callback state and returns a scalar.
1344
+ using metric_array_array_state_t = result_t (*)(uptr_t, uptr_t, uptr_t);
1342
1345
  /// Distance function callback, like `metric_array_array_size_t`, but depends on member variables.
1343
1346
  using metric_rounted_t = result_t (metric_punned_t::*)(uptr_t, uptr_t) const;
1344
1347
 
1345
1348
  metric_rounted_t metric_routed_ = nullptr;
1346
1349
  uptr_t metric_ptr_ = 0;
1347
- uptr_t metric_size_arg_ = 0;
1350
+ uptr_t metric_third_arg_ = 0;
1348
1351
 
1349
1352
  std::size_t dimensions_ = 0;
1350
1353
  metric_kind_t metric_kind_ = metric_kind_t::unknown_k;
@@ -1367,37 +1370,95 @@ class metric_punned_t {
1367
1370
  inline metric_punned_t() noexcept = default;
1368
1371
  inline metric_punned_t(metric_punned_t const&) noexcept = default;
1369
1372
  inline metric_punned_t& operator=(metric_punned_t const&) noexcept = default;
1370
- inline metric_punned_t( //
1371
- std::size_t dimensions, //
1372
- metric_kind_t metric_kind = metric_kind_t::l2sq_k, //
1373
- scalar_kind_t scalar_kind = scalar_kind_t::f32_k) noexcept
1374
- : metric_routed_(&metric_punned_t::invoke_array_array_size), metric_size_arg_(dimensions),
1375
- dimensions_(dimensions), metric_kind_(metric_kind), scalar_kind_(scalar_kind) {
1373
+
1374
+ inline metric_punned_t(std::size_t dimensions, metric_kind_t metric_kind = metric_kind_t::l2sq_k,
1375
+ scalar_kind_t scalar_kind = scalar_kind_t::f32_k) noexcept
1376
+ : metric_punned_t(builtin(dimensions, metric_kind, scalar_kind)) {}
1377
+
1378
+ inline metric_punned_t(std::size_t dimensions, std::uintptr_t metric_uintptr, metric_punned_signature_t signature,
1379
+ metric_kind_t metric_kind, scalar_kind_t scalar_kind) noexcept
1380
+ : metric_punned_t(stateless(dimensions, metric_uintptr, signature, metric_kind, scalar_kind)) {}
1381
+
1382
+ /**
1383
+ * @brief Creates a metric of a natively supported kind, choosing the best
1384
+ * available backend internally or from SimSIMD.
1385
+ *
1386
+ * @param dimensions The number of elements in the input arrays.
1387
+ * @param metric_kind The kind of metric to use.
1388
+ * @param scalar_kind The kind of scalar to use.
1389
+ * @return A metric object that can be used to compute distances between vectors.
1390
+ */
1391
+ inline static metric_punned_t builtin(std::size_t dimensions, metric_kind_t metric_kind = metric_kind_t::l2sq_k,
1392
+ scalar_kind_t scalar_kind = scalar_kind_t::f32_k) noexcept {
1393
+ metric_punned_t metric;
1394
+ metric.metric_routed_ = &metric_punned_t::invoke_array_array_third;
1395
+ metric.metric_ptr_ = 0;
1396
+ metric.metric_third_arg_ =
1397
+ scalar_kind == scalar_kind_t::b1x8_k ? divide_round_up<CHAR_BIT>(dimensions) : dimensions;
1398
+ metric.dimensions_ = dimensions;
1399
+ metric.metric_kind_ = metric_kind;
1400
+ metric.scalar_kind_ = scalar_kind;
1376
1401
 
1377
1402
  #if USEARCH_USE_SIMSIMD
1378
- if (!configure_with_simsimd())
1379
- configure_with_autovec();
1403
+ if (!metric.configure_with_simsimd())
1404
+ metric.configure_with_autovec();
1380
1405
  #else
1381
- configure_with_autovec();
1406
+ metric.configure_with_autovec();
1382
1407
  #endif
1383
1408
 
1384
- if (scalar_kind == scalar_kind_t::b1x8_k)
1385
- metric_size_arg_ = divide_round_up<CHAR_BIT>(dimensions_);
1409
+ return metric;
1386
1410
  }
1387
1411
 
1388
- inline metric_punned_t( //
1389
- std::size_t dimensions, //
1390
- std::uintptr_t metric_uintptr, metric_punned_signature_t signature, //
1391
- metric_kind_t metric_kind, //
1392
- scalar_kind_t scalar_kind) noexcept
1393
- : metric_routed_(signature == metric_punned_signature_t::array_array_k
1394
- ? &metric_punned_t::invoke_array_array
1395
- : &metric_punned_t::invoke_array_array_size),
1396
- metric_ptr_(metric_uintptr), metric_size_arg_(dimensions), dimensions_(dimensions), metric_kind_(metric_kind),
1397
- scalar_kind_(scalar_kind) {
1412
+ /**
1413
+ * @brief Creates a metric using the provided function pointer for a stateless metric.
1414
+ * So the provided ::metric_uintptr is a pointer to a function that takes two arrays
1415
+ * and returns a scalar. If the ::signature is metric_punned_signature_t::array_array_size_k,
1416
+ * then the third argument is the number of scalar words in the input vectors.
1417
+ *
1418
+ * @param dimensions The number of elements in the input arrays.
1419
+ * @param metric_uintptr The function pointer to the metric function.
1420
+ * @param signature The signature of the metric function.
1421
+ * @param metric_kind The kind of metric to use.
1422
+ * @param scalar_kind The kind of scalar to use.
1423
+ * @return A metric object that can be used to compute distances between vectors.
1424
+ */
1425
+ inline static metric_punned_t stateless(std::size_t dimensions, std::uintptr_t metric_uintptr,
1426
+ metric_punned_signature_t signature, metric_kind_t metric_kind,
1427
+ scalar_kind_t scalar_kind) noexcept {
1428
+ metric_punned_t metric;
1429
+ metric.metric_routed_ = signature == metric_punned_signature_t::array_array_k
1430
+ ? &metric_punned_t::invoke_array_array
1431
+ : &metric_punned_t::invoke_array_array_third;
1432
+ metric.metric_ptr_ = metric_uintptr;
1433
+ metric.metric_third_arg_ =
1434
+ scalar_kind == scalar_kind_t::b1x8_k ? divide_round_up<CHAR_BIT>(dimensions) : dimensions;
1435
+ metric.dimensions_ = dimensions;
1436
+ metric.metric_kind_ = metric_kind;
1437
+ metric.scalar_kind_ = scalar_kind;
1438
+ return metric;
1439
+ }
1398
1440
 
1399
- if (scalar_kind == scalar_kind_t::b1x8_k)
1400
- metric_size_arg_ = divide_round_up<CHAR_BIT>(dimensions_);
1441
+ /**
1442
+ * @brief Creates a metric using the provided function pointer for a statefull metric.
1443
+ * The third argument is the state that will be passed to the metric function.
1444
+ *
1445
+ * @param metric_uintptr The function pointer to the metric function.
1446
+ * @param metric_state The state to pass to the metric function.
1447
+ * @param metric_kind The kind of metric to use.
1448
+ * @param scalar_kind The kind of scalar to use.
1449
+ * @return A metric object that can be used to compute distances between vectors.
1450
+ */
1451
+ inline static metric_punned_t statefull(std::uintptr_t metric_uintptr, std::uintptr_t metric_state,
1452
+ metric_kind_t metric_kind = metric_kind_t::unknown_k,
1453
+ scalar_kind_t scalar_kind = scalar_kind_t::unknown_k) noexcept {
1454
+ metric_punned_t metric;
1455
+ metric.metric_routed_ = &metric_punned_t::invoke_array_array_third;
1456
+ metric.metric_ptr_ = metric_uintptr;
1457
+ metric.metric_third_arg_ = metric_state;
1458
+ metric.dimensions_ = 0;
1459
+ metric.metric_kind_ = metric_kind;
1460
+ metric.scalar_kind_ = scalar_kind;
1461
+ return metric;
1401
1462
  }
1402
1463
 
1403
1464
  inline std::size_t dimensions() const noexcept { return dimensions_; }
@@ -1485,16 +1546,17 @@ class metric_punned_t {
1485
1546
  simsimd_distance_t result;
1486
1547
  // Here `reinterpret_cast` raises warning... we know what we are doing!
1487
1548
  auto function_pointer = (simsimd_metric_punned_t)(metric_ptr_);
1488
- function_pointer(reinterpret_cast<void const*>(a), reinterpret_cast<void const*>(b), metric_size_arg_, &result);
1549
+ function_pointer(reinterpret_cast<void const*>(a), reinterpret_cast<void const*>(b), metric_third_arg_,
1550
+ &result);
1489
1551
  return (result_t)result;
1490
1552
  }
1491
1553
  result_t invoke_simsimd_reverse(uptr_t a, uptr_t b) const noexcept { return 1 - invoke_simsimd(a, b); }
1492
1554
  #else
1493
1555
  bool configure_with_simsimd() noexcept { return false; }
1494
1556
  #endif
1495
- result_t invoke_array_array_size(uptr_t a, uptr_t b) const noexcept {
1557
+ result_t invoke_array_array_third(uptr_t a, uptr_t b) const noexcept {
1496
1558
  auto function_pointer = (metric_array_array_size_t)(metric_ptr_);
1497
- result_t result = function_pointer(a, b, metric_size_arg_);
1559
+ result_t result = function_pointer(a, b, metric_third_arg_);
1498
1560
  return result;
1499
1561
  }
1500
1562
  result_t invoke_array_array(uptr_t a, uptr_t b) const noexcept {
@@ -40,8 +40,8 @@ var _Index_compiledIndex;
40
40
  Object.defineProperty(exports, "__esModule", { value: true });
41
41
  exports.Index = exports.BatchMatches = exports.Matches = exports.ScalarKind = exports.MetricKind = void 0;
42
42
  const node_gyp_build_1 = __importDefault(require("node-gyp-build"));
43
- const path = __importStar(require("node:path"));
44
- const node_fs_1 = require("node:fs");
43
+ const path = __importStar(require("path"));
44
+ const fs_1 = require("fs");
45
45
  const bindings_1 = require("bindings");
46
46
  const compiled = (0, node_gyp_build_1.default)(getBuildDir(getDirName()));
47
47
  /**
@@ -507,14 +507,14 @@ module.exports = {
507
507
  };
508
508
  // utility functions to help find native builds
509
509
  function getBuildDir(dir) {
510
- if ((0, node_fs_1.existsSync)(path.join(dir, "build")))
510
+ if ((0, fs_1.existsSync)(path.join(dir, "build")))
511
511
  return dir;
512
- if ((0, node_fs_1.existsSync)(path.join(dir, "prebuilds")))
512
+ if ((0, fs_1.existsSync)(path.join(dir, "prebuilds")))
513
513
  return dir;
514
514
  if (path.basename(dir) === ".next") {
515
515
  // special case for next.js on custom node (not vercel)
516
516
  const sideways = path.join(dir, "..", "node_modules", "usearch");
517
- if ((0, node_fs_1.existsSync)(sideways))
517
+ if ((0, fs_1.existsSync)(sideways))
518
518
  return getBuildDir(sideways);
519
519
  }
520
520
  if (dir === "/")
@@ -1,6 +1,6 @@
1
1
  import build from "node-gyp-build";
2
- import * as path from "node:path";
3
- import { existsSync } from "node:fs";
2
+ import * as path from "path";
3
+ import { existsSync } from "fs";
4
4
  import { getFileName, getRoot } from "bindings";
5
5
  const compiled = build(getBuildDir(getDirName()));
6
6
  /**
@@ -1,6 +1,6 @@
1
1
  import build from "node-gyp-build";
2
- import * as path from "node:path";
3
- import { existsSync } from "node:fs";
2
+ import * as path from "path";
3
+ import { existsSync } from "fs";
4
4
  import { getFileName, getRoot } from "bindings";
5
5
 
6
6
  const compiled: Compiled = build(getBuildDir(getDirName()));
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "usearch",
3
- "version": "2.10.5",
3
+ "version": "2.11.0",
4
4
  "description": "Smaller & Faster Single-File Vector Search Engine from Unum",
5
5
  "author": "Ash Vardanian (https://ashvardanian.com/)",
6
6
  "license": "Apache 2.0",
@@ -29,7 +29,7 @@
29
29
  ],
30
30
  "dependencies": {
31
31
  "bindings": "^1.5.0",
32
- "node-addon-api": "^3.0.0",
32
+ "node-addon-api": "^8.0.0",
33
33
  "node-gyp-build": "^4.8.0"
34
34
  },
35
35
  "scripts": {
@@ -42,10 +42,10 @@
42
42
  },
43
43
  "devDependencies": {
44
44
  "@types/bindings": "^1.5.5",
45
- "@types/node": "^20.11.0",
46
- "node-gyp": "^10.0.1",
45
+ "@types/node": "^20.12.4",
46
+ "node-gyp": "^10.1.0",
47
47
  "prebuildify": "^6.0.0",
48
- "typescript": "^5.3.3"
48
+ "typescript": "^5.4.4"
49
49
  },
50
50
  "main": "javascript/dist/cjs/usearch.js",
51
51
  "module": "javascript/dist/esm/usearch.js",
Binary file
Binary file
Binary file
Binary file