usearch 2.8.12 → 2.8.13
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CITATION.cff +1 -1
- package/Cargo.toml +1 -1
- package/README.md +31 -22
- package/VERSION +1 -1
- package/conanfile.py +1 -1
- package/include/usearch/index.hpp +1 -1
- package/include/usearch/index_dense.hpp +3 -4
- package/package.json +1 -1
- package/wasmer.toml +1 -1
package/CITATION.cff
CHANGED
package/Cargo.toml
CHANGED
package/README.md
CHANGED
|
@@ -36,18 +36,18 @@ Linux • MacOS • Windows • iOS • WebAssembly
|
|
|
36
36
|
</p>
|
|
37
37
|
|
|
38
38
|
<div align="center">
|
|
39
|
-
<a href="https://
|
|
40
|
-
<a href="https://www.npmjs.com/package/usearch"> <img alt="NPM" src="https://img.shields.io/npm/dy/usearch?label=NPM%
|
|
41
|
-
<a href="https://crates.io/crates/usearch"> <img alt="Crate" src="https://img.shields.io/crates/d/usearch?label=Crate%
|
|
42
|
-
<a href="https://www.nuget.org/packages/Cloud.Unum.USearch"> <img alt="NuGet" src="https://img.shields.io/nuget/dt/Cloud.Unum.USearch?label=NuGet%
|
|
39
|
+
<a href="https://pepy.tech/project/usearch"> <img alt="PyPI" src="https://static.pepy.tech/personalized-badge/usearch?period=month&units=abbreviation&left_color=black&right_color=blue&left_text=Python%20monthly%20installs"> </a>
|
|
40
|
+
<a href="https://www.npmjs.com/package/usearch"> <img alt="NPM" src="https://img.shields.io/npm/dy/usearch?label=NPM%20installs"> </a>
|
|
41
|
+
<a href="https://crates.io/crates/usearch"> <img alt="Crate" src="https://img.shields.io/crates/d/usearch?label=Crate%20installs"> </a>
|
|
42
|
+
<a href="https://www.nuget.org/packages/Cloud.Unum.USearch"> <img alt="NuGet" src="https://img.shields.io/nuget/dt/Cloud.Unum.USearch?label=NuGet%20installs"> </a>
|
|
43
43
|
<a href="https://central.sonatype.com/artifact/cloud.unum/usearch/overview"> <img alt="Maven" src="https://img.shields.io/nexus/r/cloud.unum/usearch?server=https%3A%2F%2Fs01.oss.sonatype.org%2F&label=Maven%20version"> </a>
|
|
44
|
-
<a href="https://hub.docker.com/r/unum/usearch"> <img alt="Docker" src="https://img.shields.io/docker/pulls/unum/usearch?label=Docker%
|
|
44
|
+
<a href="https://hub.docker.com/r/unum/usearch"> <img alt="Docker" src="https://img.shields.io/docker/pulls/unum/usearch?label=Docker%20installs"> </a>
|
|
45
45
|
<img alt="GitHub code size in bytes" src="https://img.shields.io/github/languages/code-size/unum-cloud/usearch?label=Repo%20size">
|
|
46
46
|
</div>
|
|
47
47
|
|
|
48
48
|
---
|
|
49
49
|
|
|
50
|
-
- ✅ [
|
|
50
|
+
- ✅ [10x faster][faster-than-faiss] [HNSW][hnsw-algorithm] implementation than [FAISS][faiss].
|
|
51
51
|
- ✅ Simple and extensible [single C++11 header][usearch-header] implementation.
|
|
52
52
|
- ✅ Compatible with a dozen programming languages out of the box.
|
|
53
53
|
- ✅ [Trusted](#integrations) by some of the most loved Datalakes and Databases, like [ClickHouse][clickhouse-docs].
|
|
@@ -86,28 +86,32 @@ FAISS is a widely recognized standard for high-performance vector search engines
|
|
|
86
86
|
USearch and FAISS both employ the same HNSW algorithm, but they differ significantly in their design principles.
|
|
87
87
|
USearch is compact and broadly compatible without sacrificing performance, primarily focusing on user-defined metrics and fewer dependencies.
|
|
88
88
|
|
|
89
|
-
| |
|
|
90
|
-
| :------------------------------------------- |
|
|
91
|
-
| Indexing time ⁰ |
|
|
92
|
-
| 100 Million 96d `f32`, `f16`, `i8` vectors |
|
|
93
|
-
| 100 Million 1536d `f32`, `f16`, `i8` vectors |
|
|
94
|
-
| |
|
|
95
|
-
| Codebase length ¹ |
|
|
96
|
-
| Supported metrics ² |
|
|
97
|
-
| Supported languages ³ |
|
|
98
|
-
| Supported ID types ⁴ |
|
|
99
|
-
| Required dependencies ⁵ |
|
|
100
|
-
| Bindings ⁶ |
|
|
89
|
+
| | FAISS | USearch | Improvement |
|
|
90
|
+
| :------------------------------------------- | ----------------------: | -----------------------: | ----------------------: |
|
|
91
|
+
| Indexing time ⁰ | | | |
|
|
92
|
+
| 100 Million 96d `f32`, `f16`, `i8` vectors | 2.6 · 2.6 · 2.6 h | 0.3 · 0.2 · 0.2 h | __9.6 · 10.4 · 10.7 x__ |
|
|
93
|
+
| 100 Million 1536d `f32`, `f16`, `i8` vectors | 5.0 · 4.1 · 3.8 h | 2.1 · 1.1 · 0.8 h | __2.3 · 3.6 · 4.4 x__ |
|
|
94
|
+
| | | | |
|
|
95
|
+
| Codebase length ¹ | 84 K [SLOC][sloc] | 3 K [SLOC][sloc] | maintainable |
|
|
96
|
+
| Supported metrics ² | 9 fixed metrics | any metric | extendible |
|
|
97
|
+
| Supported languages ³ | C++, Python | 10 languages | portable |
|
|
98
|
+
| Supported ID types ⁴ | 32-bit, 64-bit | 32-bit, 40-bit, 64-bit | efficient |
|
|
99
|
+
| Required dependencies ⁵ | BLAS, OpenMP | - | light-weight |
|
|
100
|
+
| Bindings ⁶ | SWIG | Native | low-latency |
|
|
101
|
+
| Python binding size ⁷ | [~ 10 MB][faiss-weight] | [< 1 MB][usearch-weight] | deployable |
|
|
101
102
|
|
|
102
103
|
[sloc]: https://en.wikipedia.org/wiki/Source_lines_of_code
|
|
104
|
+
[faiss-weight]: https://pypi.org/project/faiss-cpu/#files
|
|
105
|
+
[usearch-weight]: https://pypi.org/project/usearch/#files
|
|
103
106
|
|
|
104
107
|
> ⁰ [Tested][intel-benchmarks] on Intel Sapphire Rapids, with the simplest inner-product distance, equivalent recall, and memory consumption while also providing far superior search speed.
|
|
105
|
-
> ¹ A shorter codebase makes the project easier to maintain and audit.
|
|
108
|
+
> ¹ A shorter codebase of `usearch/` over `faiss/` makes the project easier to maintain and audit.
|
|
106
109
|
> ² User-defined metrics allow you to customize your search for various applications, from GIS to creating custom metrics for composite embeddings from multiple AI models or hybrid full-text and semantic search.
|
|
107
110
|
> ³ With USearch, you can reuse the same preconstructed index in various programming languages.
|
|
108
111
|
> ⁴ The 40-bit integer allows you to store 4B+ vectors without allocating 8 bytes for every neighbor reference in the proximity graph.
|
|
109
112
|
> ⁵ Lack of obligatory dependencies makes USearch much more portable.
|
|
110
113
|
> ⁶ Native bindings introduce lower call latencies than more straightforward approaches.
|
|
114
|
+
> ⁷ Lighter bindings make downloads and deployments faster.
|
|
111
115
|
|
|
112
116
|
[intel-benchmarks]: https://www.unum.cloud/blog/2023-11-07-scaling-vector-search-with-intel
|
|
113
117
|
|
|
@@ -141,6 +145,7 @@ index = Index(
|
|
|
141
145
|
connectivity=16, # Optional: Limit number of neighbors per graph node
|
|
142
146
|
expansion_add=128, # Optional: Control the recall of indexing
|
|
143
147
|
expansion_search=64, # Optional: Control the quality of the search
|
|
148
|
+
multi=False, # Optional: Allow multiple vectors per key, default = False
|
|
144
149
|
)
|
|
145
150
|
```
|
|
146
151
|
|
|
@@ -161,7 +166,7 @@ index.save("index.usearch")
|
|
|
161
166
|
loaded_copy = index.load("index.usearch")
|
|
162
167
|
view = Index.restore("index.usearch", view=True)
|
|
163
168
|
|
|
164
|
-
other_view = Index(ndim=..., metric
|
|
169
|
+
other_view = Index(ndim=..., metric=...)
|
|
165
170
|
other_view.view("index.usearch")
|
|
166
171
|
```
|
|
167
172
|
|
|
@@ -208,7 +213,9 @@ multi_index.search(...)
|
|
|
208
213
|
|
|
209
214
|
## Clustering
|
|
210
215
|
|
|
211
|
-
Once the index is constructed,
|
|
216
|
+
Once the index is constructed, USearch can perform K-Nearest Neighbors Clustering much faster than standalone clustering libraries, like SciPy,
|
|
217
|
+
UMap, and tSNE.
|
|
218
|
+
Same for dimensionality reduction with PCA.
|
|
212
219
|
Essentially, the `Index` itself can be seen as a clustering, allowing iterative deepening.
|
|
213
220
|
|
|
214
221
|
```py
|
|
@@ -396,6 +403,8 @@ index.add(keys, fingerprints)
|
|
|
396
403
|
matches = index.search(fingerprints, 10)
|
|
397
404
|
```
|
|
398
405
|
|
|
406
|
+
That method was used to build the ["USearch Molecules"](https://github.com/ashvardanian/usearch-molecules), one of the largest Chem-Informatics datasets, containing 7 billion small molecules and 28 billion fingerprints.
|
|
407
|
+
|
|
399
408
|
[smiles]: https://en.wikipedia.org/wiki/Simplified_molecular-input_line-entry_system
|
|
400
409
|
[rdkit-fingerprints]: https://www.rdkit.org/docs/RDKit_Book.html#additional-information-about-the-fingerprints
|
|
401
410
|
|
|
@@ -424,7 +433,7 @@ doi = {10.5281/zenodo.7949416},
|
|
|
424
433
|
author = {Vardanian, Ash},
|
|
425
434
|
title = {{USearch by Unum Cloud}},
|
|
426
435
|
url = {https://github.com/unum-cloud/usearch},
|
|
427
|
-
version = {2.8.
|
|
436
|
+
version = {2.8.13},
|
|
428
437
|
year = {2023},
|
|
429
438
|
month = oct,
|
|
430
439
|
}
|
package/VERSION
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
2.8.
|
|
1
|
+
2.8.13
|
package/conanfile.py
CHANGED
|
@@ -7,7 +7,7 @@ required_conan_version = ">=1.53.0"
|
|
|
7
7
|
class USearchConan(ConanFile):
|
|
8
8
|
|
|
9
9
|
name = "usearch"
|
|
10
|
-
version = '2.8.
|
|
10
|
+
version = '2.8.13'
|
|
11
11
|
license = "Apache-2.0"
|
|
12
12
|
description = "Smaller & Faster Single-File Vector Search Engine from Unum"
|
|
13
13
|
homepage = "https://github.com/unum-cloud/usearch"
|
|
@@ -1309,11 +1309,10 @@ class index_dense_gt {
|
|
|
1309
1309
|
shared_lock_t lock(slot_lookup_mutex_);
|
|
1310
1310
|
offset = (std::min)(offset, slot_lookup_.size());
|
|
1311
1311
|
slot_lookup_.for_each([&](key_and_slot_t const& key_and_slot) {
|
|
1312
|
-
if (offset)
|
|
1312
|
+
if (offset)
|
|
1313
|
+
// Skip the first `offset` entries
|
|
1313
1314
|
--offset;
|
|
1314
|
-
|
|
1315
|
-
}
|
|
1316
|
-
if (limit) {
|
|
1315
|
+
else if (limit) {
|
|
1317
1316
|
*keys = key_and_slot.key;
|
|
1318
1317
|
++keys;
|
|
1319
1318
|
--limit;
|
package/package.json
CHANGED