usearch 1.1.0 → 1.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/Cargo.toml +1 -1
- package/README.md +47 -8
- package/conanfile.py +1 -1
- package/docs/wolfram/index.rst +6 -0
- package/package.json +1 -1
- package/python/lib.cpp +5 -2
- package/wasmer.toml +1 -1
package/Cargo.toml
CHANGED
package/README.md
CHANGED
|
@@ -20,12 +20,12 @@ Vector Search Engine<br/>
|
|
|
20
20
|
<p align="center">
|
|
21
21
|
Euclidean • Angular • Jaccard • Hamming • Haversine • User-Defined Metrics
|
|
22
22
|
<br/>
|
|
23
|
-
<a href="https://unum-cloud.github.io/usearch/cpp">C++11</a> •
|
|
24
|
-
<a href="https://unum-cloud.github.io/usearch/python">Python</a> •
|
|
23
|
+
<a href="https://unum-cloud.github.io/usearch/cpp">C++ 11</a> •
|
|
24
|
+
<a href="https://unum-cloud.github.io/usearch/python">Python 3</a> •
|
|
25
25
|
<a href="https://unum-cloud.github.io/usearch/javascript">JavaScript</a> •
|
|
26
26
|
<a href="https://unum-cloud.github.io/usearch/java">Java</a> •
|
|
27
27
|
<a href="https://unum-cloud.github.io/usearch/rust">Rust</a> •
|
|
28
|
-
<a href="https://unum-cloud.github.io/usearch/c">
|
|
28
|
+
<a href="https://unum-cloud.github.io/usearch/c">C 99</a> •
|
|
29
29
|
<a href="https://unum-cloud.github.io/usearch/objective-c">Objective-C</a> •
|
|
30
30
|
<a href="https://unum-cloud.github.io/usearch/swift">Swift</a> •
|
|
31
31
|
<a href="https://unum-cloud.github.io/usearch/golang">GoLang</a> •
|
|
@@ -159,7 +159,46 @@ other_view = Index(ndim=..., metric=CompiledMetric(...))
|
|
|
159
159
|
other_view.view("index.usearch")
|
|
160
160
|
```
|
|
161
161
|
|
|
162
|
-
##
|
|
162
|
+
## Exact, Approximate, and Multi-Index Lookups
|
|
163
|
+
|
|
164
|
+
Approximate search methods, such as HNSW, are predominantly used when an exact brute-force search becomes too resource-intensive.
|
|
165
|
+
This typically occurs when you have millions of entries in a collection.
|
|
166
|
+
For smaller collections, we offer a more direct approach with the `search` method.
|
|
167
|
+
|
|
168
|
+
```py
|
|
169
|
+
from usearch.index import search, MetricKind, Matches, BatchMatches
|
|
170
|
+
import numpy as np
|
|
171
|
+
|
|
172
|
+
# Generate 10'000 random vectors with 1024 dimensions
|
|
173
|
+
vectors = np.random.rand(10_000, 1024).astype(np.float32)
|
|
174
|
+
vector = np.random.rand(1024).astype(np.float32)
|
|
175
|
+
|
|
176
|
+
one_in_many: Matches = search(vectors, vector, 50, MetricKind.L2sq, exact=True)
|
|
177
|
+
many_in_many: BatchMatches = search(vectors, vectors, 50, MetricKind.L2sq, exact=True)
|
|
178
|
+
```
|
|
179
|
+
|
|
180
|
+
By passing the `exact=True` argument, the system bypasses indexing altogether and performs a brute-force search through the entire dataset using SIMD-optimized similarity metrics from [SimSIMD](https://github.com/ashvardanian/simsimd).
|
|
181
|
+
When compared to FAISS's `IndexFlatL2` in Google Colab, **[USearch may offer up to a 20x performance improvement](https://github.com/unum-cloud/usearch/issues/176#issuecomment-1666650778)**:
|
|
182
|
+
|
|
183
|
+
- `faiss.IndexFlatL2`: **55.3 ms**.
|
|
184
|
+
- `usearch.index.search`: **2.54 ms**.
|
|
185
|
+
|
|
186
|
+
For larger workloads targeting billions or even trillions of vectors, parallel multi-index lookups become invaluable.
|
|
187
|
+
These lookups prevent the need to construct a single, massive index, allowing users to query multiple smaller ones instead.
|
|
188
|
+
|
|
189
|
+
```py
|
|
190
|
+
from usearch.index import Indexes
|
|
191
|
+
|
|
192
|
+
multi_index = Indexes(
|
|
193
|
+
indexes: Iterable[usearch.index.Index] = [...],
|
|
194
|
+
paths: Iterable[os.PathLike] = [...],
|
|
195
|
+
view: bool = False,
|
|
196
|
+
threads: int = 0,
|
|
197
|
+
)
|
|
198
|
+
multi_index.search(...)
|
|
199
|
+
```
|
|
200
|
+
|
|
201
|
+
## Joins, One-to-One, One-to-Many, and Many-to-Many Mappings
|
|
163
202
|
|
|
164
203
|
One of the big questions these days is how will AI change the world of databases and data management.
|
|
165
204
|
Most databases are still struggling to implement high-quality fuzzy search, and the only kind of joins they know are deterministic.
|
|
@@ -189,9 +228,9 @@ Broader functionality is ported per request.
|
|
|
189
228
|
| :---------------------- | :----: | :------: | :---: | :---: | :--------: | :---: | :----: | :---: |
|
|
190
229
|
| Add, search | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
|
191
230
|
| Save, load, view | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
|
|
192
|
-
| Join | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
|
193
231
|
| User-defined metrics | ✅ | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
|
194
|
-
|
|
|
232
|
+
| Joins | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
|
233
|
+
| Variable-length vectors | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
|
195
234
|
| 4B+ capacities | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
|
|
196
235
|
|
|
197
236
|
## Application Examples
|
|
@@ -281,9 +320,9 @@ matches = index.search(fingerprints, 10)
|
|
|
281
320
|
## Integrations
|
|
282
321
|
|
|
283
322
|
- [x] GPT-Cache.
|
|
284
|
-
- [
|
|
285
|
-
- [ ] Microsoft Semantic Kernel.
|
|
323
|
+
- [x] LangChain.
|
|
286
324
|
- [ ] ClickHouse.
|
|
325
|
+
- [ ] Microsoft Semantic Kernel.
|
|
287
326
|
|
|
288
327
|
## Citations
|
|
289
328
|
|
package/conanfile.py
CHANGED
|
@@ -4,7 +4,7 @@ from conans import ConanFile
|
|
|
4
4
|
class USearchConan(ConanFile):
|
|
5
5
|
|
|
6
6
|
name = 'USearch'
|
|
7
|
-
version = '1.1.
|
|
7
|
+
version = '1.1.1'
|
|
8
8
|
license = 'Apache 2.0'
|
|
9
9
|
url = 'https://github.com/unum-cloud/usearch'
|
|
10
10
|
description = 'Smaller & Faster Single-File Vector Search Engine from Unum'
|
package/package.json
CHANGED
package/python/lib.cpp
CHANGED
|
@@ -298,7 +298,10 @@ static void search_typed( //
|
|
|
298
298
|
if (!threads)
|
|
299
299
|
threads = std::thread::hardware_concurrency();
|
|
300
300
|
|
|
301
|
-
|
|
301
|
+
visits_bitset_t query_mutexes(static_cast<std::size_t>(vectors_count));
|
|
302
|
+
if (!query_mutexes)
|
|
303
|
+
throw std::bad_alloc();
|
|
304
|
+
|
|
302
305
|
executor_default_t{threads}.execute_bulk(indexes.shards_.size(), [&](std::size_t, std::size_t task_idx) {
|
|
303
306
|
dense_index_py_t& index = *indexes.shards_[task_idx].get();
|
|
304
307
|
|
|
@@ -318,7 +321,7 @@ static void search_typed( //
|
|
|
318
321
|
dense_search_result_t result = index.search(vector, wanted, config);
|
|
319
322
|
result.error.raise();
|
|
320
323
|
{
|
|
321
|
-
|
|
324
|
+
auto lock = query_mutexes.lock(vector_idx);
|
|
322
325
|
counts_py1d(vector_idx) = static_cast<Py_ssize_t>(result.merge_into( //
|
|
323
326
|
&keys_py2d(vector_idx, 0), //
|
|
324
327
|
&distances_py2d(vector_idx, 0), //
|