epzero 0.3.0__tar.gz → 0.5.0__tar.gz

epzero-0.5.0/CHANGELOG.md

@@ -0,0 +1,129 @@
+# Changelog
+
+## 0.5.0
+
+An index that stays accurate as its data changes, fixed in place; metadata and keyword search in the
+engine; LlamaIndex and Haystack beside LangChain; retrieval evaluation; and Windows. Files written by
+0.4 load unchanged.
+
+**Staying accurate after the build.**
+
+- `index.recalibrate(queries)` (C++: `epzero::recalibrate`): re-measures the beam width for the
+  recall target on real queries and saves it in the profile, without rebuilding.
+- `index.health()` (C++: `check_health`): live and deleted vectors, growth since calibration, and how
+  much of the data the principal directions still capture, with advice in plain words: compact,
+  recalibrate, or rebuild.
+- `RecallGuard(index)` takes its target and starting beam width from the profile.
+- `index.check_queries(queries)` (C++: `check_queries`) says whether queries look like the data: text
+  queries against image vectors sit 2.2-2.3× farther from their nearest neighbours than rows of the data,
+  against 0.99-1.09× for queries like the data, and recall calibrated on the data then falls to 0.74-0.80
+  for a 0.95 target. Its advice: `recalibrate(queries)`, which restored 0.945-0.958 on VIBE's
+  text-to-image sets.
+- `index.requantize()` (C++: `HnswIndex::requantize`, then `refine_links`): for vectors added since the
+  build that lie outside the ranges the 8-bit codes were fitted on. It refits the ranges on the live
+  vectors, re-encodes them and reconnects the graph, in about half a rebuild's time; after a 4 sd drift
+  it brought the beam width for recall 0.95 from 200 back to a rebuild's 12. `health()` reports
+  `code_error` and advises it above 0.01.
+- `index.get_vector(label)` returns a stored vector.
+
+**Information retrieval.** `epzero.ir`: a `Collection` of documents with string ids, retrieved by vector,
+by keyword (BM25) or both (reciprocal rank fusion), growing and shrinking after the build; TREC run files
+in and out; relevance judgements in TREC or BEIR form; nDCG@k, MAP, recall, precision and MRR, matching
+trec_eval (tested against it); and a BEIR dataset loader.
+
+**Stemming.** `TextIndex` reduces English words to their Porter stems, so "connections" matches
+"connected" (`stemmer="none"` keeps words as written). It is the reference implementation's algorithm,
+the one Lucene and Anserini use, and agrees with NLTK's on all 39,011 words of SciFact's and NFCorpus's
+vocabularies. nDCG@10 for BM25 went from 0.660 to 0.688 on SciFact and from 0.305 to 0.321 on NFCorpus
+(BEIR's Anserini baselines: 0.665 and 0.325), and for hybrid search with all-MiniLM-L6-v2 from 0.682 to
+0.713 and from 0.344 to 0.354. Stemmed text indexes are saved as text format 2; files saved before load
+unstemmed, as they were written.
+
+**Fixes.** `hybrid_search` searched with a fixed beam width of 64 and the default re-rank, ignoring the
+index's calibrated profile; it now uses the profile unless `ef` is given. Calibration could settle on a
+beam width below k, which only k-sized searches had measured.
+
+**File format 5.** Indexes with a profile or 16-bit vectors are written as version 5; 0.4's version 4
+files are still read (a file written by 0.4.0 is part of the test suite). See `docs/file-format.md`.
+
+**Indexes grow.** `add()` no longer fails when an index is full: it grows the capacity (at least
+doubling), copying the nodes and keeping the graph; `index.reserve(n)` and `index.capacity` do it ahead.
+The LangChain, LlamaIndex and Haystack stores, which rebuilt their whole graph each time they filled, now
+grow in place: loading 40,000 documents in batches of 100 went from 6.0 s to 3.4 s. Searches on other
+threads wait for a growth, which moves the index's memory; with a RecallGuard attached, growing must be
+done beforehand with `reserve()`.
+
+**Metadata in the engine.** `epzero.Metadata` (C++: `MetadataIndex`) keeps str, int, float and bool fields
+per label, with an inverted index per field and ordered numeric values, and `select(where)` returns the
+labels that pass ({field: value}, {field: [values]}, or `$eq`, `$ne`, `$in`, `$nin`, `$gt`, `$gte`, `$lt`,
+`$lte`, `$exists`) for a filtered search. The LangChain store now runs dict filters there instead of
+checking every document in Python. Saved in its own file format, fuzzed in CI.
+
+**Memory.** `half_vectors=True` keeps the full vectors as 16-bit floats: about half an index's memory,
+at some speed (up to 17% on GIST1M) and recall (0.003-0.004 on OpenAI embeddings), so never chosen
+automatically. Full-precision search is 4-9% faster, as vector addresses are resolved once per query.
+
+**LlamaIndex.** `epzero.llamaindex.EpzeroVectorStore` (`pip install "epzero[llamaindex]"`): nodes with
+their text and metadata, filters in the engine, `delete_ref_doc`, and persistence.
+
+**Haystack.** `epzero.haystack.EpzeroDocumentStore` and `EpzeroEmbeddingRetriever`
+(`pip install "epzero[haystack]"`): Haystack filters in the engine where the answer is provably the same,
+and Haystack's document-store test suite passes.
+
+**Windows.** The engine builds with MSVC and its tests run on Windows in CI; wheels for Windows (x64).
+
+**Easier to read.** `Index`, `Metadata`, `TextIndex` and `Database` print what they hold (size, metric,
+compression, calibrated beam width), and shape and name errors say what was passed.
+
+## 0.4.0
+
+**Automatic configuration.** `epzero.build(vectors, metric=...)` (C++: `epzero::build_auto` in
+`epzero/auto.h`) configures an index from its data and the machine it is built on:
+
+- From the data: 8-bit codes, whether to walk the graph on principal directions (at 256 dimensions or
+  more, when few directions hold most of the variance), the build passes, the re-rank, and the beam
+  width (`ef`) that reaches a recall target, calibrated on rows held out of the graph or on queries
+  you pass (`queries=`).
+- From the machine: the prefetch setting, timed on this CPU, and every core.
+- Measured on an Intel Xeon Gold 6226 and an AMD Threadripper PRO 9985WX: as fast as hand-tuned
+  settings (within 1-5%), recall 0.949-0.957 on real queries for a 0.95 target, and 1.3-4.1× Glass's
+  queries per second on GIST1M, OpenAI DBpedia and DEEP10M.
+- Every setting can be fixed instead (`epzero.build(vectors, nav_dim=0, M=32)`, or `epzero.plan()`,
+  edit, build). `index.plan` shows each setting and why it was chosen.
+- The choices are saved inside the index (`IndexProfile`, file format v4), and `search()` uses them
+  when called without settings. A loaded index re-applies its prefetch setting only on the same kind
+  of CPU (`epzero.machine()` names it).
+
+**Faster search and builds.**
+
+- `nav_dim`: walk the graph on 8-bit codes of principal directions, re-ranking with the full vectors.
+  With the other changes here, 2.8-4.2× Glass's queries per second on GIST1M and 1.6-2.7× on OpenAI
+  DBpedia embeddings at equal recall, on Intel and AMD servers.
+- `refine_links()`: a second build pass in which every node searches the finished graph and
+  reconnects; `fill_free_links()` gives free link slots to two-hop neighbours.
+- `fast_build`: build on the 8-bit codes.
+- `rerank_z`: stop the re-rank early once the remaining candidates can't reach the top k.
+- Queries are projected onto principal directions on the integer kernel; the AVX2 and AVX-VNNI integer
+  kernels finish with vector loads instead of a scalar tail.
+- The prefetch tuner can fetch a record's every cache line; on a Xeon Gold 6226 that was worth up to
+  1.7× on some datasets.
+- PCA runs as blocked matrix products (choosing directions 5× faster on GIST1M), training runs on
+  every core, and index memory is zeroed on every core, which spreads it across NUMA nodes.
+
+**Fixes.**
+
+- `Database`: a write that could not be applied (the database full, or not yet trained) was written to
+  the log first, after which every open failed replaying it. Writes are now checked before they are
+  logged, and replacing a vector in a full database frees its slot before compacting.
+- `RecallGuard` searched, and measured its reference, with the default re-rank depth; it now uses the
+  index's own settings and re-ranks the reference's whole beam. `explain()` and `trace()` likewise.
+- The LangChain store copied every stored vector on each add (quadratic in the store's size).
+- `pip install epzero` from source failed on machines with gRPC installed: it tried to build the
+  cluster node, whose protocol files aren't in the package.
+
+**Compatibility.** Indexes without a profile are still written in the format older versions read
+(v2, or v3 with principal directions); automatically configured ones are v4, which 0.3 cannot load.
+`search()`'s `k`, `ef`, `rerank` and `rerank_z` now default to the index's profile, which for an index
+built by hand is the old defaults (10, 64, 2 × k, off).
+
+Earlier releases are described by their git tags (`v0.2.0`, `v0.3.0`).

{epzero-0.3.0 → epzero-0.5.0}/CMakeLists.txt

@@ -48,9 +48,11 @@ endif()
 find_package(Threads REQUIRED)
 
 add_library(epzero_engine
+  src/auto.cpp
   src/distance.cpp
   src/distance_scalar.cpp
   src/hnsw.cpp
+  src/metadata.cpp
   src/recall.cpp
   src/database.cpp
   src/text.cpp
@@ -59,20 +61,38 @@ add_library(epzero_engine
 )
 target_include_directories(epzero_engine PUBLIC include PRIVATE src)
 target_link_libraries(epzero_engine PUBLIC Threads::Threads)
-target_compile_options(epzero_engine PRIVATE -Wall -Wextra -Wpedantic)
+if(MSVC)
+  target_compile_options(epzero_engine PRIVATE /W3 /utf-8 /permissive- /Zc:__cplusplus)
+  target_compile_definitions(epzero_engine PRIVATE _CRT_SECURE_NO_WARNINGS NOMINMAX)
+else()
+  target_compile_options(epzero_engine PRIVATE -Wall -Wextra -Wpedantic)
+endif()
 set_target_properties(epzero_engine PROPERTIES POSITION_INDEPENDENT_CODE ON)
 
 # SIMD kernels get ISA flags per file; the rest of the library stays baseline so the binary
 # runs on any CPU of the architecture and picks the best kernel at runtime.
 if(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64|AMD64|amd64")
-  target_sources(epzero_engine PRIVATE src/distance_avx2.cpp src/distance_avx512.cpp)
-  set_source_files_properties(src/distance_avx2.cpp PROPERTIES COMPILE_OPTIONS "-mavx2;-mfma;-mpopcnt")
-  set_source_files_properties(src/distance_avx512.cpp PROPERTIES COMPILE_OPTIONS "-mavx512f")
+  target_sources(epzero_engine PRIVATE src/distance_avx2.cpp src/distance_avx512.cpp src/distance_f16c.cpp)
+  if(MSVC)  # MSVC allows any intrinsic; /arch only lets the compiler use the ISA on its own
+    set_source_files_properties(src/distance_f16c.cpp src/distance_avx2.cpp PROPERTIES COMPILE_OPTIONS "/arch:AVX2")
+    set_source_files_properties(src/distance_avx512.cpp PROPERTIES COMPILE_OPTIONS "/arch:AVX512")
+  else()
+    set_source_files_properties(src/distance_f16c.cpp PROPERTIES COMPILE_OPTIONS "-mavx2;-mfma;-mf16c")
+    set_source_files_properties(src/distance_avx2.cpp PROPERTIES COMPILE_OPTIONS "-mavx2;-mfma;-mpopcnt")
+    set_source_files_properties(src/distance_avx512.cpp PROPERTIES COMPILE_OPTIONS "-mavx512f")
+  endif()
   # VNNI integer kernels, where the toolchain can build them: the compiler must know the
   # instructions (GCC 11+, Clang 12+ for AVX-VNNI) and so must the assembler it hands them to
   # (binutils 2.36+ for AVX-VNNI; RHEL 9's 2.35 doesn't), so each check compiles and links a use.
   include(CheckCXXSourceCompiles)
-  set(CMAKE_REQUIRED_FLAGS "-mavx2 -mavxvnni")
+  if(MSVC)
+    set(EPZERO_AVXVNNI_FLAGS "/arch:AVX2")
+    set(EPZERO_AVX512VNNI_FLAGS "/arch:AVX512")
+  else()
+    set(EPZERO_AVXVNNI_FLAGS "-mavx2 -mavxvnni")
+    set(EPZERO_AVX512VNNI_FLAGS "-mavx512f -mavx512bw -mavx512vnni")
+  endif()
+  set(CMAKE_REQUIRED_FLAGS "${EPZERO_AVXVNNI_FLAGS}")
   check_cxx_source_compiles([[
     #include <immintrin.h>
     int main(int argc, char**) {
@@ -80,7 +100,7 @@ if(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64|AMD64|amd64")
       a = _mm256_dpbusd_avx_epi32(a, a, a);
       return _mm256_extract_epi32(a, 0);
     }]] EPZERO_TOOLCHAIN_AVXVNNI)
-  set(CMAKE_REQUIRED_FLAGS "-mavx512f -mavx512bw -mavx512vnni")
+  set(CMAKE_REQUIRED_FLAGS "${EPZERO_AVX512VNNI_FLAGS}")
   check_cxx_source_compiles([[
     #include <immintrin.h>
     int main(int argc, char**) {
@@ -91,12 +111,14 @@ if(CMAKE_SYSTEM_PROCESSOR MATCHES "x86_64|AMD64|amd64")
   unset(CMAKE_REQUIRED_FLAGS)
   if(EPZERO_TOOLCHAIN_AVXVNNI)
     target_sources(epzero_engine PRIVATE src/distance_avxvnni.cpp)
-    set_source_files_properties(src/distance_avxvnni.cpp PROPERTIES COMPILE_OPTIONS "-mavx2;-mavxvnni")
+    string(REPLACE " " ";" flags "${EPZERO_AVXVNNI_FLAGS}")
+    set_source_files_properties(src/distance_avxvnni.cpp PROPERTIES COMPILE_OPTIONS "${flags}")
     set_property(SOURCE src/distance.cpp APPEND PROPERTY COMPILE_DEFINITIONS EPZERO_HAVE_AVXVNNI)
   endif()
   if(EPZERO_TOOLCHAIN_AVX512VNNI)
     target_sources(epzero_engine PRIVATE src/distance_avx512vnni.cpp)
-    set_source_files_properties(src/distance_avx512vnni.cpp PROPERTIES COMPILE_OPTIONS "-mavx512f;-mavx512bw;-mavx512vnni")
+    string(REPLACE " " ";" flags "${EPZERO_AVX512VNNI_FLAGS}")
+    set_source_files_properties(src/distance_avx512vnni.cpp PROPERTIES COMPILE_OPTIONS "${flags}")
     set_property(SOURCE src/distance.cpp APPEND PROPERTY COMPILE_DEFINITIONS EPZERO_HAVE_AVX512VNNI)
   endif()
 elseif(CMAKE_SYSTEM_PROCESSOR MATCHES "aarch64|arm64")
@@ -118,8 +140,9 @@ if(EPZERO_BUILD_TESTS)
   FetchContent_MakeAvailable(googletest)
 
   enable_testing()
-  add_executable(epzero_tests tests/distance_test.cpp tests/hnsw_test.cpp tests/recall_test.cpp tests/dataset_test.cpp tests/database_test.cpp tests/text_test.cpp)
+  add_executable(epzero_tests tests/auto_test.cpp tests/distance_test.cpp tests/half_test.cpp tests/hnsw_test.cpp tests/metadata_test.cpp tests/recall_test.cpp tests/dataset_test.cpp tests/database_test.cpp tests/text_test.cpp)
   target_link_libraries(epzero_tests PRIVATE epzero_engine GTest::gtest_main)
+  target_compile_definitions(epzero_tests PRIVATE EPZERO_TEST_DATA="${CMAKE_CURRENT_SOURCE_DIR}/tests/data")
   if(EMSCRIPTEN)
     # Runs under Node (CMAKE_CROSSCOMPILING_EMULATOR); workers are pre-spawned for the
     # concurrency tests and the real filesystem backs the save/load tests.
@@ -145,7 +168,7 @@ if(EPZERO_BUILD_BENCH)
 endif()
 
 if(EPZERO_BUILD_FUZZ)
-  foreach(target snapshot graph wal text)
+  foreach(target snapshot graph wal text metadata)
     add_executable(fuzz_${target} fuzz/fuzz_${target}.cpp)
     target_link_libraries(fuzz_${target} PRIVATE epzero_engine)
     target_link_options(fuzz_${target} PRIVATE -fsanitize=fuzzer)
@@ -192,20 +215,26 @@ if(EPZERO_BUILD_NODE AND NOT EMSCRIPTEN)
 endif()
 
 if(EMSCRIPTEN)
-  # ES module for the browser demo: `import createEpzero from "./epzero.js"`.
-  add_executable(epzero_web wasm/epzero_wasm.cpp)
-  target_link_libraries(epzero_web PRIVATE epzero_engine)
+  # ES modules for the browser demo: `import createEpzero from "./epzero.js"`. epzero-lab.js adds the
+  # automatic configuration (build_auto) for the benchmark lab, which is 36 KB more compressed: kept out
+  # of the module every other page loads first.
+  foreach(web_target epzero_web epzero_lab)
+    add_executable(${web_target} wasm/epzero_wasm.cpp)
+    target_link_libraries(${web_target} PRIVATE epzero_engine)
+    target_link_options(${web_target} PRIVATE
+      -lembind -sMODULARIZE=1 -sEXPORT_ES6=1 -sEXPORT_NAME=createEpzero -sENVIRONMENT=web,worker -sSTACK_SIZE=1MB
+      # No eval / new Function in the generated bindings, so the site's Content Security Policy can
+      # forbid it (only WebAssembly compilation is allowed).
+      -sDYNAMIC_EXECUTION=0)
+    if(EPZERO_WASM_THREADS)
+      # Pre-spawn the workers: the module's own thread blocks while a build runs, so it cannot
+      # create new ones on demand.
+      target_link_options(${web_target} PRIVATE "-sPTHREAD_POOL_SIZE=Math.min(navigator.hardwareConcurrency,16)")
+    endif()
+  endforeach()
   set_target_properties(epzero_web PROPERTIES OUTPUT_NAME epzero SUFFIX .js)
-  target_link_options(epzero_web PRIVATE
-    -lembind -sMODULARIZE=1 -sEXPORT_ES6=1 -sEXPORT_NAME=createEpzero -sENVIRONMENT=web,worker -sSTACK_SIZE=1MB
-    # No eval / new Function in the generated bindings, so the site's Content Security Policy can
-    # forbid it (only WebAssembly compilation is allowed).
-    -sDYNAMIC_EXECUTION=0)
-  if(EPZERO_WASM_THREADS)
-    # Pre-spawn the workers: the module's own thread blocks while a build runs, so it cannot
-    # create new ones on demand.
-    target_link_options(epzero_web PRIVATE "-sPTHREAD_POOL_SIZE=Math.min(navigator.hardwareConcurrency,16)")
-  endif()
+  set_target_properties(epzero_lab PROPERTIES OUTPUT_NAME epzero-lab SUFFIX .js)
+  target_compile_definitions(epzero_lab PRIVATE EPZERO_WASM_LAB_AUTO=1)
 endif()
 
 if(EPZERO_BUILD_PYTHON)

epzero-0.5.0/PKG-INFO

@@ -0,0 +1,242 @@
+Metadata-Version: 2.2
+Name: epzero
+Version: 0.5.0
+Summary: A fast HNSW vector database in C++20 that configures itself: SIMD search, filtered search, recall monitoring, crash-safe storage
+Keywords: vector database,hnsw,ann,nearest neighbor,embeddings,rag,langchain
+Author: Lajat
+License: MIT
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: C++
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Operating System :: MacOS
+Classifier: Operating System :: Microsoft :: Windows
+Classifier: Topic :: Database
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Project-URL: Homepage, https://epzero.dev/
+Project-URL: Source, https://github.com/Lazeeez/Epzero
+Requires-Python: >=3.10
+Requires-Dist: numpy
+Provides-Extra: langchain
+Requires-Dist: langchain-core>=0.3; extra == "langchain"
+Provides-Extra: llamaindex
+Requires-Dist: llama-index-core>=0.12; extra == "llamaindex"
+Provides-Extra: haystack
+Requires-Dist: haystack-ai>=2.10; extra == "haystack"
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: pytest-asyncio; extra == "test"
+Requires-Dist: langchain-core>=0.3; extra == "test"
+Requires-Dist: langchain-tests>=0.3; extra == "test"
+Requires-Dist: llama-index-core>=0.12; extra == "test"
+Requires-Dist: haystack-ai>=2.10; extra == "test"
+Description-Content-Type: text/markdown
+
+# epzero
+
+An HNSW vector database written from scratch in C++20, with Python bindings.
+
+- **Configures itself:** `epzero.build(vectors)` picks compression, the graph build and the search
+  effort for your data and recall target, times memory prefetching on your CPU, saves the choices
+  inside the index and tells you why it made each one. Any setting can be overridden.
+- **Fast search:** hand-written SIMD kernels (AVX2, AVX-512, VNNI, NEON) chosen at runtime, 8-bit
+  codes, an optional walk on principal directions, and lock-free reads. On dedicated Intel Xeon Gold
+  6226 and AMD EPYC 7513 servers, one thread, at equal recall: 1.4-4.2× the queries per second of
+  Glass, the fastest library we compared, on GIST1M, OpenAI DBpedia and DEEP10M, and 1.0-1.3× on
+  GloVe-100 (with each dataset's best settings; see the project's scoreboard for every loss).
+- **Filtered search that picks its own strategy:** an exact scan, a filtered graph walk, or a
+  two-hop walk (the idea behind ACORN), chosen per query by a cost model.
+- **Recall you can see:** `RecallGuard` measures recall on a sample of live queries with a
+  confidence interval and moves the search effort to hold a target; `explain()` says why any given
+  item was not returned. With `per_query=True` each query also stops as soon as its answer settles,
+  so easy queries cost less and hard ones search further.
+- **Hybrid search:** `TextIndex` (BM25 keyword search, with English stemming) and `hybrid_search()`,
+  which merges it with vector search by reciprocal rank fusion.
+- **Retrieval evaluation:** `epzero.ir` keeps documents under string ids, runs dense, keyword and
+  hybrid retrieval, writes TREC run files and scores them against relevance judgements exactly as
+  trec_eval does. On BEIR with all-MiniLM-L6-v2, hybrid reaches nDCG@10 0.713 on SciFact and 0.354 on
+  NFCorpus, above either half alone.
+- **Durable:** `Database` adds a write-ahead log with crash recovery and checkpoints.
+- **8-bit quantization** with exact re-ranking, stopped early per query once the rest can't reach
+  the top k.
+- **Stays accurate as data changes:** `health()` says when an index needs recalibrating,
+  requantizing, compacting or rebuilding, and each fix runs in place.
+- **LangChain, LlamaIndex and Haystack** stores, each passing its framework's own test suite.
+
+```bash
+pip install epzero                 # the engine
+pip install "epzero[langchain]"    # plus the LangChain vector store
+pip install "epzero[llamaindex]"   # plus the LlamaIndex vector store
+pip install "epzero[haystack]"     # plus the Haystack document store and retriever
+```
+
+## Quick start: let it configure itself
+
+```python
+import numpy as np
+import epzero
+
+vectors = np.random.rand(100_000, 768).astype(np.float32)
+index = epzero.build(vectors, metric="cosine")       # chooses every setting (below)
+labels, distances = index.search(vectors[:5], k=10)  # at the effort calibrated for recall 0.95
+print(index.plan)                                    # what it chose, and why
+```
+
+`build` decides from the data: 8-bit compression, whether to walk on principal directions (high
+dimensions with a concentrated spectrum), the build passes, the re-rank, and the beam width that
+reaches your recall target, measured on rows held out of the graph. From the machine: the SIMD
+kernel, the prefetch setting (timed on this CPU) and every core. Change anything:
+
+```python
+index = epzero.build(vectors, metric="cosine", recall=0.99)           # a different target
+index = epzero.build(vectors, metric="cosine", queries=my_queries)    # calibrate on real queries
+index = epzero.build(vectors, metric="cosine", nav_dim=0, refine=False, M=32)  # override choices
+plan = epzero.plan(vectors, metric="cosine")         # look before building, edit, then pass it in
+index = epzero.build(vectors, plan=plan.replace(ef_construction=400))
+labels, distances = index.search(queries, k=50, ef=200)               # per-call overrides
+index.save("my.epz")                                 # the settings and reasons are saved inside it
+index = epzero.Index.load("my.epz")                  # search() uses them again
+```
+
+`build` returns an ordinary `epzero.Index`: filters, deletes, `explain()` and `RecallGuard` all use
+its settings. A saved index re-applies its prefetch setting only on the same kind of CPU (call
+`tune_prefetch()` on another). The decisions are made in the C++ engine (`epzero/auto.h`), so C++
+users get the same `build_auto()`.
+
+If your queries differ from your data (text queries against image vectors, say), pass a few
+hundred of them as `queries=`: recall calibrated on data rows is optimistic for those.
+
+**Metadata.** Fields per label, filtered in the engine:
+
+```python
+meta = epzero.Metadata()
+meta.set_many(labels, [{"source": "faq", "year": 2024}, ...])
+allowed = meta.select({"source": ["faq", "docs"], "year": {"$gte": 2022}})
+labels, distances = index.search(queries, k=10, filter=index.filter(allowed))
+```
+
+**After the build.** An index keeps changing; its calibration doesn't, by itself:
+
+```python
+print(index.health())          # growth since calibration, deleted share, code error, fit of the directions, advice
+print(index.check_queries(queries))   # do these queries look like the data? (text vs images: no)
+index.recalibrate(queries)     # re-measure the beam width on real queries; saved in the profile
+index.requantize()             # vectors drifted outside the 8-bit codes' ranges: refit and reconnect
+guard = epzero.RecallGuard(index, sample_rate=0.01)   # or measure live recall and adjust as you go
+```
+
+When later vectors drift outside the ranges the 8-bit codes were fitted on, they are clipped and
+the graph links chosen for them suffer: in a synthetic test with a 4 sd drift, the beam width needed
+for recall 0.95 went from 12 to 200. `requantize()` refits the codes and reconnects the graph in about half a rebuild's
+time, back to 12.
+
+## Configured by hand
+
+```python
+vectors = np.random.rand(100_000, 128).astype(np.float32)
+index = epzero.Index(dim=128, metric="l2", max_elements=100_000, M=16, ef_construction=200)
+index.add(vectors)                                   # labels 0..n-1 (or pass labels=); grows as needed
+
+labels, distances = index.search(vectors[:5], k=10, ef=64)
+
+# Filtered search: compile the allowed labels once, reuse across queries.
+allowed = index.filter(np.arange(0, 100_000, 50))
+labels, distances = index.search(vectors[:5], k=10, filter=allowed)
+
+# Hold recall at 0.95 as the workload drifts, checking 1% of queries; per_query=True lets each
+# query stop once its answer settles (cheaper for the same recall).
+guard = epzero.RecallGuard(index, target=0.95, sample_rate=0.01, reference_ef=1024, per_query=True)
+labels, distances = guard.search(vectors[:5], k=10)
+print(guard.report())
+
+# Optional: measure the best memory-prefetch setting for this machine and data (not saved).
+index.tune_prefetch(vectors[:500])
+
+index.save("vectors.epzero")
+index = epzero.Index.load("vectors.epzero")
+
+# Hybrid: keyword search on the same labels, merged with the vector search.
+text = epzero.TextIndex()
+text.add(None, documents)                            # labels 0..n-1, matching the index
+labels, scores = epzero.hybrid_search(index, text, query_vector, "error E-1042 at checkout", k=10)
+```
+
+## LangChain
+
+```python
+from langchain_openai import OpenAIEmbeddings
+from epzero.langchain import EpzeroVectorStore
+
+store = EpzeroVectorStore.from_texts(texts, OpenAIEmbeddings(), metadatas=metadatas)
+store.similarity_search("how do I reset my password?", k=4, filter={"source": "faq"})
+store.max_marginal_relevance_search("pricing", k=4, fetch_k=20)
+store.hybrid_search("error E-1042 at checkout", k=4)   # keywords + vectors
+retriever = store.as_retriever(search_kwargs={"k": 8})
+
+store.save_local("./store")
+store = EpzeroVectorStore.load_local("./store", OpenAIEmbeddings())
+```
+
+Metadata filters are a dict of required values (a list means "any of"), or a callable on the
+metadata. They are compiled into an Epzero filter, so the engine chooses how to search the allowed
+documents instead of over-fetching and discarding. The store passes LangChain's standard
+vector-store test suite.
+
+## LlamaIndex
+
+```python
+from llama_index.core import StorageContext, VectorStoreIndex
+from epzero.llamaindex import EpzeroVectorStore
+
+store = EpzeroVectorStore(metric="cosine")
+index = VectorStoreIndex.from_documents(documents, storage_context=StorageContext.from_defaults(vector_store=store))
+index.as_query_engine().query("How do I reset my password?")
+store.persist("./epzero-store")        # EpzeroVectorStore.from_persist_path("./epzero-store")
+```
+
+Metadata filters (`==`, `!=`, `>`, `>=`, `<`, `<=`, `in`, `nin`, combined with AND or OR) run in the
+engine, as do `doc_ids` and `node_ids`; `delete_ref_doc` removes a document's nodes.
+
+## Haystack
+
+```python
+from haystack import Pipeline
+from epzero.haystack import EpzeroDocumentStore, EpzeroEmbeddingRetriever
+
+store = EpzeroDocumentStore(metric="cosine")
+store.write_documents(documents)       # Documents with embeddings
+pipeline = Pipeline()
+pipeline.add_component("retriever", EpzeroEmbeddingRetriever(document_store=store, top_k=5))
+pipeline.run({"retriever": {"query_embedding": embedding,
+                            "filters": {"field": "meta.lang", "operator": "==", "value": "en"}}})
+```
+
+Filters use Haystack's syntax and run in the engine wherever that gives exactly Haystack's answer;
+the rest (ranges on dates, `None`) go to Haystack's reference implementation. The store passes
+Haystack's document-store test suite.
+
+## Retrieval evaluation
+
+```python
+from epzero import ir
+
+corpus, queries, qrels = ir.load_beir("scifact")              # a BEIR dataset folder
+collection = ir.Collection.build(list(corpus), doc_vectors, texts=list(corpus.values()), metric="cosine")
+run = collection.retrieve(list(queries), vectors=query_vectors, texts=list(queries.values()),
+                          k=100, mode="hybrid")                # or "dense", "keyword"
+ir.write_run(run, "run.txt", tag="my-system")                  # TREC format, for CLEF or TREC
+print(ir.evaluate(run, qrels))                                 # nDCG@10, R@100, MAP, MRR@10
+```
+
+`evaluate` matches trec_eval (tested against it, ties included) and also takes `per_query=True` and
+measures like `P@5` or `nDCG@20`. Documents can be added, replaced and removed by id, and a collection
+saves to a folder.
+
+## Platforms
+
+Wheels for Linux (x86-64 and ARM64, glibc 2.28+), macOS (Apple silicon and Intel) and, from 0.5,
+Windows (x64), Python 3.10 to 3.13. Elsewhere, `pip install epzero` builds from source (CMake 3.20+, a C++20 compiler).
+
+Live demo, benchmarks and design notes: https://epzero.dev/ · MIT license.

{epzero-0.3.0 → epzero-0.5.0}/bench/bench_recall.cpp

@@ -40,6 +40,16 @@ struct Options {
   std::size_t repeat = 1;          // passes over the query set, so many-thread runs last long enough to time
   Quantization quantization = Quantization::None;
   bool keep_vectors = true;
+  std::size_t nav_dim = 0;
+  bool fill_links = false;
+  bool fast_build = false;
+  bool half = false;
+  bool fill_after = false;
+  std::size_t refine = 0;
+  bool refine_exact = false;
+  std::uint64_t seed = 42;
+  float rerank_z = 0.0f;
+  std::size_t refine_ef = 0;
   std::size_t rerank = 0;
   std::size_t prefetch_depth = 0, prefetch_lines = 0;  // 0: the index's default
   std::vector<std::size_t> ef = {10, 20, 40, 80, 160, 320};
@@ -86,6 +96,16 @@ Options parse(int argc, char** argv) {
     }
     else if (arg == "--no-keep") o.keep_vectors = false;
     else if (arg == "--rerank") o.rerank = std::stoul(value());
+    else if (arg == "--nav-dim") o.nav_dim = std::stoul(value());
+    else if (arg == "--fill-links") o.fill_links = true;
+    else if (arg == "--fast-build") o.fast_build = true;
+    else if (arg == "--half") o.half = true;
+    else if (arg == "--fill-after") o.fill_after = true;
+    else if (arg == "--refine") o.refine = std::stoul(value());
+    else if (arg == "--refine-exact") o.refine_exact = true;
+    else if (arg == "--seed") o.seed = std::stoull(value());
+    else if (arg == "--rerank-z") o.rerank_z = std::stof(value());
+    else if (arg == "--refine-ef") o.refine_ef = std::stoul(value());
     else if (arg == "--prefetch") {  // depth,lines
       const auto v = parse_list(value());
       if (v.size() != 2) throw std::invalid_argument("--prefetch takes depth,lines");
@@ -171,8 +191,9 @@ int main(int argc, char** argv) try {
     std::printf("load      %s in %.2fs\n", opt.load.c_str(), seconds_since(t0));
   } else {
     HnswParams params{.dim = dim, .metric = opt.metric, .max_elements = base.n, .M = opt.M,
-                      .ef_construction = opt.ef_construction, .quantization = opt.quantization,
-                      .keep_vectors = opt.keep_vectors};
+                      .ef_construction = opt.ef_construction, .seed = opt.seed, .quantization = opt.quantization,
+                      .keep_vectors = opt.keep_vectors, .nav_dim = opt.nav_dim, .fill_links = opt.fill_links,
+                      .fast_build = opt.fast_build, .half_vectors = opt.half};
     loaded = std::make_unique<HnswIndex>(params);
     t0 = Clock::now();
     loaded->train(base.data.data(), std::min<std::size_t>(base.n, 100'000));  // no-op unquantized
@@ -184,6 +205,16 @@ int main(int argc, char** argv) try {
       });
     }
     for (auto& b : builders) b.join();
+    for (std::size_t pass = 0; pass < opt.refine; ++pass) {
+      const double refine_at = seconds_since(t0);
+      loaded->refine_links(opt.threads, opt.refine_ef, opt.refine_exact);
+      std::printf("refined   pass %zu in %.2fs\n", pass + 1, seconds_since(t0) - refine_at);
+    }
+    if (opt.fill_after) {
+      const double links_at = seconds_since(t0);
+      const std::size_t added = loaded->fill_free_links(opt.threads);
+      std::printf("filled    %zu links in %.2fs\n", added, seconds_since(t0) - links_at);
+    }
     build_s = seconds_since(t0);
     std::printf("build     %.2fs, %.0f inserts/s (%zu threads, M=%zu, efc=%zu)\n", build_s, base.n / build_s,
                 opt.threads, opt.M, opt.ef_construction);
@@ -230,7 +261,7 @@ int main(int argc, char** argv) try {
         const std::size_t q = i % queries.n;
         SearchStats stats;
         const auto start = Clock::now();
-        auto found = index.search(queries.row(q), opt.k, {.ef = ef, .rerank = opt.rerank, .stats = &stats});
+        auto found = index.search(queries.row(q), opt.k, {.ef = ef, .rerank = opt.rerank, .rerank_z = opt.rerank_z, .stats = &stats});
         latencies[i] = std::chrono::duration<double, std::micro>(Clock::now() - start).count();
         distances_per_query[i] = stats.distance_computations;
         full_per_query[i] = stats.full_distance_computations;

epzero-0.5.0/bench/regression.json

@@ -0,0 +1,58 @@
+{
+  "float-l2": {
+    "4": {
+      "recall": 0.9068,
+      "distances": 209.99
+    },
+    "8": {
+      "recall": 0.9068,
+      "distances": 209.99
+    },
+    "16": {
+      "recall": 0.9664,
+      "distances": 247.212
+    }
+  },
+  "float-cosine": {
+    "4": {
+      "recall": 0.9042,
+      "distances": 222.262
+    },
+    "8": {
+      "recall": 0.9042,
+      "distances": 222.262
+    },
+    "16": {
+      "recall": 0.967,
+      "distances": 263.986
+    }
+  },
+  "sq8-second-pass": {
+    "4": {
+      "recall": 0.9132,
+      "distances": 234.102
+    },
+    "8": {
+      "recall": 0.9132,
+      "distances": 234.102
+    },
+    "16": {
+      "recall": 0.981,
+      "distances": 275.388
+    }
+  },
+  "sq8-directions": {
+    "32": {
+      "recall": 0.4988,
+      "distances": 323.592
+    },
+    "64": {
+      "recall": 0.7316,
+      "distances": 390.994
+    },
+    "128": {
+      "recall": 0.9518,
+      "distances": 508.41
+    }
+  }
+}

epzero-0.5.0/fuzz/fuzz_metadata.cpp

@@ -0,0 +1,23 @@
+// Fuzzes MetadataIndex::load: arbitrary bytes as a metadata file must either load into an index that is
+// safe to query, change and save, or throw. Never crash, read out of bounds or hit undefined behavior.
+#include <exception>
+#include <string>
+
+#include "common.h"
+#include "epzero/metadata.h"
+
+extern "C" int LLVMFuzzerTestOneInput(const std::uint8_t* data, std::size_t size) {
+  static const auto path = epzero::fuzz::scratch("metadata") / "input.meta";
+  epzero::fuzz::write_bytes(path, data, size);
+  try {
+    epzero::MetadataIndex index = epzero::MetadataIndex::load(path.string());
+    (void)index.select({{.field = "source", .any_of = {std::string("faq")}}, {.field = "year", .min = 2000.0}});
+    (void)index.select({{.field = "year", .max = 1e9, .negate = true}});
+    index.set(1, {{"source", std::string("new")}});
+    index.remove(1);
+    index.save(path.string());
+  } catch (const std::exception&) {
+    // Rejected cleanly: the expected outcome for corrupt input.
+  }
+  return 0;
+}