epzero 0.3.0__tar.gz → 0.4.0__tar.gz

epzero-0.4.0/CHANGELOG.md

@@ -0,0 +1,54 @@
+# Changelog
+
+## 0.4.0
+
+**Automatic configuration.** `epzero.build(vectors, metric=...)` (C++: `epzero::build_auto` in
+`epzero/auto.h`) configures an index from its data and the machine it is built on:
+
+- From the data: 8-bit codes, whether to walk the graph on principal directions (at 256 dimensions or
+  more, when few directions hold most of the variance), the build passes, the re-rank, and the beam
+  width (`ef`) that reaches a recall target, calibrated on rows held out of the graph or on queries
+  you pass (`queries=`).
+- From the machine: the prefetch setting, timed on this CPU, and every core.
+- Measured on an Intel Xeon Gold 6226 and an AMD Threadripper PRO 9985WX: as fast as hand-tuned
+  settings (within 1-5%), recall 0.949-0.957 on real queries for a 0.95 target, and 1.3-4.1× Glass's
+  queries per second on GIST1M, OpenAI DBpedia and DEEP10M.
+- Every setting can be fixed instead (`epzero.build(vectors, nav_dim=0, M=32)`, or `epzero.plan()`,
+  edit, build). `index.plan` shows each setting and why it was chosen.
+- The choices are saved inside the index (`IndexProfile`, file format v4), and `search()` uses them
+  when called without settings. A loaded index re-applies its prefetch setting only on the same kind
+  of CPU (`epzero.machine()` names it).
+
+**Faster search and builds.**
+
+- `nav_dim`: walk the graph on 8-bit codes of principal directions, re-ranking with the full vectors.
+  With the other changes here, 2.8-4.2× Glass's queries per second on GIST1M and 1.6-2.7× on OpenAI
+  DBpedia embeddings at equal recall, on Intel and AMD servers.
+- `refine_links()`: a second build pass in which every node searches the finished graph and
+  reconnects; `fill_free_links()` gives free link slots to two-hop neighbours.
+- `fast_build`: build on the 8-bit codes.
+- `rerank_z`: stop the re-rank early once the remaining candidates can't reach the top k.
+- Queries are projected onto principal directions on the integer kernel; the AVX2 and AVX-VNNI integer
+  kernels finish with vector loads instead of a scalar tail.
+- The prefetch tuner can fetch a record's every cache line; on a Xeon Gold 6226 that was worth up to
+  1.7× on some datasets.
+- PCA runs as blocked matrix products (choosing directions 5× faster on GIST1M), training runs on
+  every core, and index memory is zeroed on every core, which spreads it across NUMA nodes.
+
+**Fixes.**
+
+- `Database`: a write that could not be applied (the database full, or not yet trained) was written to
+  the log first, after which every open failed replaying it. Writes are now checked before they are
+  logged, and replacing a vector in a full database frees its slot before compacting.
+- `RecallGuard` searched, and measured its reference, with the default re-rank depth; it now uses the
+  index's own settings and re-ranks the reference's whole beam. `explain()` and `trace()` likewise.
+- The LangChain store copied every stored vector on each add (quadratic in the store's size).
+- `pip install epzero` from source failed on machines with gRPC installed: it tried to build the
+  cluster node, whose protocol files aren't in the package.
+
+**Compatibility.** Indexes without a profile are still written in the format older versions read
+(v2, or v3 with principal directions); automatically configured ones are v4, which 0.3 cannot load.
+`search()`'s `k`, `ef`, `rerank` and `rerank_z` now default to the index's profile, which for an index
+built by hand is the old defaults (10, 64, 2 × k, off).
+
+Earlier releases are described by their git tags (`v0.2.0`, `v0.3.0`).

{epzero-0.3.0 → epzero-0.4.0}/CMakeLists.txt

@@ -48,6 +48,7 @@ endif()
 find_package(Threads REQUIRED)
 
 add_library(epzero_engine
+  src/auto.cpp
   src/distance.cpp
   src/distance_scalar.cpp
   src/hnsw.cpp
@@ -118,7 +119,7 @@ if(EPZERO_BUILD_TESTS)
   FetchContent_MakeAvailable(googletest)
 
   enable_testing()
-  add_executable(epzero_tests tests/distance_test.cpp tests/hnsw_test.cpp tests/recall_test.cpp tests/dataset_test.cpp tests/database_test.cpp tests/text_test.cpp)
+  add_executable(epzero_tests tests/auto_test.cpp tests/distance_test.cpp tests/hnsw_test.cpp tests/recall_test.cpp tests/dataset_test.cpp tests/database_test.cpp tests/text_test.cpp)
   target_link_libraries(epzero_tests PRIVATE epzero_engine GTest::gtest_main)
   if(EMSCRIPTEN)
     # Runs under Node (CMAKE_CROSSCOMPILING_EMULATOR); workers are pre-spawned for the

{epzero-0.3.0 → epzero-0.4.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.2
 Name: epzero
-Version: 0.3.0
-Summary: An HNSW vector database built from scratch in C++20: SIMD search, filtered search, recall monitoring, crash-safe storage
+Version: 0.4.0
+Summary: A fast HNSW vector database in C++20 that configures itself: SIMD search, filtered search, recall monitoring, crash-safe storage
 Keywords: vector database,hnsw,ann,nearest neighbor,embeddings,rag,langchain
 Author: Lajat
 License: MIT
@@ -31,9 +31,14 @@ Description-Content-Type: text/markdown
 
 An HNSW vector database written from scratch in C++20, with Python bindings.
 
-- **Fast search:** hand-written SIMD kernels (AVX2, AVX-512, NEON), chosen at runtime, and
-  lock-free reads. On SIFT1M it serves 1.13× hnswlib's and 1.17× FAISS's queries per second at
-  equal recall, on the same machine.
+- **Configures itself:** `epzero.build(vectors)` picks compression, the graph build and the search
+  effort for your data and recall target, times memory prefetching on your CPU, saves the choices
+  inside the index and tells you why it made each one. Any setting can be overridden.
+- **Fast search:** hand-written SIMD kernels (AVX2, AVX-512, VNNI, NEON) chosen at runtime, 8-bit
+  codes, an optional walk on principal directions, and lock-free reads. On dedicated Intel Xeon Gold
+  6226 and AMD EPYC 7513 servers, one thread, at equal recall: 1.4-4.2× the queries per second of
+  Glass, the fastest library we compared, on GIST1M, OpenAI DBpedia and DEEP10M, and 1.0-1.3× on
+  GloVe-100 (with each dataset's best settings; see the project's scoreboard for every loss).
 - **Filtered search that picks its own strategy:** an exact scan, a filtered graph walk, or a
   two-hop walk (the idea behind ACORN), chosen per query by a cost model.
 - **Recall you can see:** `RecallGuard` measures recall on a sample of live queries with a
@@ -43,7 +48,8 @@ An HNSW vector database written from scratch in C++20, with Python bindings.
 - **Hybrid search:** `TextIndex` (BM25 keyword search) and `hybrid_search()`, which merges it
   with vector search by reciprocal rank fusion.
 - **Durable:** `Database` adds a write-ahead log with crash recovery and checkpoints.
-- **8-bit quantization** with exact re-ranking.
+- **8-bit quantization** with exact re-ranking, stopped early per query once the rest can't reach
+  the top k.
 - **A LangChain vector store** in `epzero.langchain`.
 
 ```bash
@@ -51,12 +57,45 @@ pip install epzero                 # the engine
 pip install "epzero[langchain]"    # plus the LangChain vector store
 ```
 
-## Quick start
+## Quick start: let it configure itself
 
 ```python
 import numpy as np
 import epzero
 
+vectors = np.random.rand(100_000, 768).astype(np.float32)
+index = epzero.build(vectors, metric="cosine")       # chooses every setting (below)
+labels, distances = index.search(vectors[:5], k=10)  # at the effort calibrated for recall 0.95
+print(index.plan)                                    # what it chose, and why
+```
+
+`build` decides from the data: 8-bit compression, whether to walk on principal directions (high
+dimensions with a concentrated spectrum), the build passes, the re-rank, and the beam width that
+reaches your recall target, measured on rows held out of the graph. From the machine: the SIMD
+kernel, the prefetch setting (timed on this CPU) and every core. Change anything:
+
+```python
+index = epzero.build(vectors, metric="cosine", recall=0.99)           # a different target
+index = epzero.build(vectors, metric="cosine", queries=my_queries)    # calibrate on real queries
+index = epzero.build(vectors, metric="cosine", nav_dim=0, refine=False, M=32)  # override choices
+plan = epzero.plan(vectors, metric="cosine")         # look before building, edit, then pass it in
+index = epzero.build(vectors, plan=plan.replace(ef_construction=400))
+labels, distances = index.search(queries, k=50, ef=200)               # per-call overrides
+index.save("my.epz")                                 # the settings and reasons are saved inside it
+index = epzero.Index.load("my.epz")                  # search() uses them again
+```
+
+`build` returns an ordinary `epzero.Index`: filters, deletes, `explain()` and `RecallGuard` all use
+its settings. A saved index re-applies its prefetch setting only on the same kind of CPU (call
+`tune_prefetch()` on another). The decisions are made in the C++ engine (`epzero/auto.h`), so C++
+users get the same `build_auto()`.
+
+If your queries differ from your data (text queries against image vectors, say), pass a few
+hundred of them as `queries=`: recall calibrated on data rows is optimistic for those.
+
+## Configured by hand
+
+```python
 vectors = np.random.rand(100_000, 128).astype(np.float32)
 index = epzero.Index(dim=128, metric="l2", max_elements=100_000, M=16, ef_construction=200)
 index.add(vectors)                                   # labels 0..n-1 (or pass labels=)

{epzero-0.3.0 → epzero-0.4.0}/bench/bench_recall.cpp

@@ -40,6 +40,15 @@ struct Options {
   std::size_t repeat = 1;          // passes over the query set, so many-thread runs last long enough to time
   Quantization quantization = Quantization::None;
   bool keep_vectors = true;
+  std::size_t nav_dim = 0;
+  bool fill_links = false;
+  bool fast_build = false;
+  bool fill_after = false;
+  std::size_t refine = 0;
+  bool refine_exact = false;
+  std::uint64_t seed = 42;
+  float rerank_z = 0.0f;
+  std::size_t refine_ef = 0;
   std::size_t rerank = 0;
   std::size_t prefetch_depth = 0, prefetch_lines = 0;  // 0: the index's default
   std::vector<std::size_t> ef = {10, 20, 40, 80, 160, 320};
@@ -86,6 +95,15 @@ Options parse(int argc, char** argv) {
     }
     else if (arg == "--no-keep") o.keep_vectors = false;
     else if (arg == "--rerank") o.rerank = std::stoul(value());
+    else if (arg == "--nav-dim") o.nav_dim = std::stoul(value());
+    else if (arg == "--fill-links") o.fill_links = true;
+    else if (arg == "--fast-build") o.fast_build = true;
+    else if (arg == "--fill-after") o.fill_after = true;
+    else if (arg == "--refine") o.refine = std::stoul(value());
+    else if (arg == "--refine-exact") o.refine_exact = true;
+    else if (arg == "--seed") o.seed = std::stoull(value());
+    else if (arg == "--rerank-z") o.rerank_z = std::stof(value());
+    else if (arg == "--refine-ef") o.refine_ef = std::stoul(value());
     else if (arg == "--prefetch") {  // depth,lines
       const auto v = parse_list(value());
       if (v.size() != 2) throw std::invalid_argument("--prefetch takes depth,lines");
@@ -171,8 +189,9 @@ int main(int argc, char** argv) try {
     std::printf("load      %s in %.2fs\n", opt.load.c_str(), seconds_since(t0));
   } else {
     HnswParams params{.dim = dim, .metric = opt.metric, .max_elements = base.n, .M = opt.M,
-                      .ef_construction = opt.ef_construction, .quantization = opt.quantization,
-                      .keep_vectors = opt.keep_vectors};
+                      .ef_construction = opt.ef_construction, .seed = opt.seed, .quantization = opt.quantization,
+                      .keep_vectors = opt.keep_vectors, .nav_dim = opt.nav_dim, .fill_links = opt.fill_links,
+                      .fast_build = opt.fast_build};
     loaded = std::make_unique<HnswIndex>(params);
     t0 = Clock::now();
     loaded->train(base.data.data(), std::min<std::size_t>(base.n, 100'000));  // no-op unquantized
@@ -184,6 +203,16 @@ int main(int argc, char** argv) try {
       });
     }
     for (auto& b : builders) b.join();
+    for (std::size_t pass = 0; pass < opt.refine; ++pass) {
+      const double refine_at = seconds_since(t0);
+      loaded->refine_links(opt.threads, opt.refine_ef, opt.refine_exact);
+      std::printf("refined   pass %zu in %.2fs\n", pass + 1, seconds_since(t0) - refine_at);
+    }
+    if (opt.fill_after) {
+      const double links_at = seconds_since(t0);
+      const std::size_t added = loaded->fill_free_links(opt.threads);
+      std::printf("filled    %zu links in %.2fs\n", added, seconds_since(t0) - links_at);
+    }
     build_s = seconds_since(t0);
     std::printf("build     %.2fs, %.0f inserts/s (%zu threads, M=%zu, efc=%zu)\n", build_s, base.n / build_s,
                 opt.threads, opt.M, opt.ef_construction);
@@ -230,7 +259,7 @@ int main(int argc, char** argv) try {
         const std::size_t q = i % queries.n;
         SearchStats stats;
         const auto start = Clock::now();
-        auto found = index.search(queries.row(q), opt.k, {.ef = ef, .rerank = opt.rerank, .stats = &stats});
+        auto found = index.search(queries.row(q), opt.k, {.ef = ef, .rerank = opt.rerank, .rerank_z = opt.rerank_z, .stats = &stats});
         latencies[i] = std::chrono::duration<double, std::micro>(Clock::now() - start).count();
         distances_per_query[i] = stats.distance_computations;
         full_per_query[i] = stats.full_distance_computations;

{epzero-0.3.0 → epzero-0.4.0}/fuzz/make_seeds.cpp

@@ -29,6 +29,23 @@ int main(int argc, char** argv) {
   for (std::size_t i = 0; i < 40; ++i) index.add(i, vectors.data() + i * kDim);
   index.save((out / "snapshot" / "valid").string());
 
+  // Format v4: 8-bit codes and a profile (search settings, reasons, a machine name).
+  epzero::HnswParams quantized = small_params();
+  quantized.quantization = epzero::Quantization::SQ8;
+  epzero::HnswIndex profiled(quantized);
+  profiled.train(vectors.data(), 40);
+  for (std::size_t i = 0; i < 40; ++i) profiled.add(i, vectors.data() + i * kDim);
+  epzero::IndexProfile profile;
+  profile.ef = 24;
+  profile.rerank = 30;
+  profile.rerank_z = 3.0f;
+  profile.recall_target = 0.95f;
+  profile.refined = true;
+  profile.machine = "fuzz/scalar";
+  profile.notes = {{"ef", "calibrated"}, {"nav_dim", "off"}};
+  profiled.set_profile(profile);
+  profiled.save((out / "snapshot" / "valid-profile").string());
+
   const auto graph = index.export_graph();
   std::ofstream(out / "graph" / "valid", std::ios::binary)
       .write(reinterpret_cast<const char*>(graph.data()), static_cast<std::streamsize>(graph.size()));

epzero-0.4.0/include/epzero/auto.h

@@ -0,0 +1,70 @@
+#pragma once
+
+// Automatic configuration: an index set up for its data and for the machine it is built on, with
+// every choice overridable. The rules come from measurements in docs/performance.md.
+//
+//   epzero::AutoOptions options;
+//   options.metric = epzero::Metric::Cosine;
+//   options.fixed.nav_dim = 0;                       // override any choice
+//   auto index = epzero::build_auto(data, nullptr, n, dim, options);
+//   index->profile();                                // what it chose, and why
+//
+// From the data: 8-bit codes, whether to walk on principal directions, the build passes, the
+// re-rank, and the beam width (ef) that reaches the recall target, calibrated on rows held out of
+// the graph (or on queries you pass). From the machine: the prefetch setting, timed here, and every
+// core. All of it is saved with the index; see IndexProfile.
+
+#include <cstddef>
+#include <cstdint>
+#include <memory>
+#include <optional>
+#include <utility>
+
+#include "epzero/hnsw.h"
+
+namespace epzero {
+
+// Settings build_auto() would otherwise choose. Any that is set is used as given.
+struct AutoFixed {
+  std::optional<Quantization> quantization;
+  std::optional<std::size_t> nav_dim;
+  std::optional<bool> fast_build;
+  std::optional<bool> refine;      // refine_links() after the build
+  std::optional<bool> fill_after;  // fill_free_links() after the build
+  std::optional<std::size_t> rerank;
+  std::optional<float> rerank_z;
+  std::optional<std::size_t> ef;   // skips calibration
+  std::optional<std::pair<std::size_t, std::size_t>> prefetch;  // (depth, lines); skips timing
+};
+
+struct AutoOptions {
+  Metric metric = Metric::L2;
+  std::size_t M = 16;
+  std::size_t ef_construction = 200;
+  std::size_t k = 10;            // the k the beam width is calibrated for
+  float recall = 0.95f;          // the recall target
+  std::size_t threads = 0;       // 0: every core
+  std::size_t capacity = 0;      // vectors the index can hold (0: the rows given)
+  std::uint64_t seed = 42;
+  // Optional typical queries (query_count rows of dim floats) to calibrate on. Rows of the data are
+  // accurate when queries look like the data, and optimistic when they don't.
+  const float* queries = nullptr;
+  std::size_t query_count = 0;
+  AutoFixed fixed;
+};
+
+struct AutoPlan {
+  HnswParams params;
+  IndexProfile profile;  // ef is 0 while it still has to be calibrated
+  bool calibrate = true;  // build_auto will calibrate ef
+  bool tune = true;       // build_auto will time prefetch settings
+};
+
+// What build_auto() would do with `data` (n rows of dim floats), without building.
+AutoPlan plan_auto(const float* data, std::size_t n, std::size_t dim, const AutoOptions& options = {});
+
+// Builds an index of `data` with labels[i] for row i (labels null: 0..n-1), configured automatically.
+std::unique_ptr<HnswIndex> build_auto(const float* data, const std::uint64_t* labels, std::size_t n, std::size_t dim,
+                                      const AutoOptions& options = {});
+
+}  // namespace epzero

{epzero-0.3.0 → epzero-0.4.0}/include/epzero/database.h

@@ -107,6 +107,7 @@ class Database {
  private:
   Database(std::string dir, const HnswParams& params, const DatabaseOptions& options);
   void recover();
+  void check_upserts(const std::uint64_t* labels, std::size_t n) const;
   void append(const std::vector<std::uint8_t>& bytes, bool sync);
   void apply_upsert(std::uint64_t label, const float* vector);
   void apply_record(const LogRecord& record);

{epzero-0.3.0 → epzero-0.4.0}/include/epzero/distance.h

@@ -2,6 +2,7 @@
 
 #include <cstddef>
 #include <cstdint>
+#include <string>
 #include <string_view>
 #include <utility>
 #include <vector>
@@ -29,6 +30,10 @@ bool simd_level_supported(SimdLevel level) noexcept;
 // forces a specific (supported) level, which is how the benchmarks measure SIMD speedups.
 SimdLevel active_simd_level() noexcept;
 
+// The running CPU's model name ("" if unknown) and the active SIMD level, e.g. "AMD EPYC 7513 32-Core
+// Processor/avx2": what machine-specific settings, such as prefetching, were measured on.
+std::string machine_name();
+
 // Returns nullptr if `level` is not supported on this machine.
 DistanceFn get_distance_fn(Metric metric, SimdLevel level) noexcept;
 DistanceFn get_distance_fn(Metric metric) noexcept;

{epzero-0.3.0 → epzero-0.4.0}/include/epzero/hnsw.h

@@ -40,6 +40,18 @@ struct HnswParams {
   // SQ8: also keep full-precision vectors, used to build a better graph and to re-rank the final
   // candidates exactly. Without them the index stores ~4x less vector data. RaBitQ requires them.
   bool keep_vectors = true;
+  // SQ8: walk the graph on 8-bit codes of this many principal directions, learned by train(), instead
+  // of one code per dimension (0 = off). Each scored node then reads and computes nav_dim bytes, not
+  // dim, and the final re-rank with full vectors recovers exact distances. For embeddings whose
+  // variance sits in far fewer directions than they have dimensions. Needs keep_vectors, and
+  // usually a larger SearchOptions::rerank than the default.
+  std::size_t nav_dim = 0;
+  // Building: when the diversity heuristic leaves a node's link list short, fill the free slots with
+  // the closest candidates it pruned. The slots are allocated either way. Not saved with the index.
+  bool fill_links = false;
+  // Building, with SQ8: search for neighbours and prune them on the codes instead of the full
+  // vectors. Not saved with the index: later inserts use full vectors.
+  bool fast_build = false;
 };
 
 struct SearchResult {
@@ -114,6 +126,10 @@ struct SearchOptions {
   // and as re-scoring all ef of them, to within 0.0003, and 3-13% more queries per second than 3 x k
   // where full vectors are large). Ignored otherwise.
   std::size_t rerank = 0;
+  // Stops the re-rank early (0 = off): candidates are scored in blocks, and scoring ends once the
+  // next one, placed by this query's gap between estimated and exact distances, is more than this
+  // many standard deviations past the k-th best. `rerank` is then only a cap. 3 is a safe value.
+  float rerank_z = 0.0f;
   SearchStats* stats = nullptr;     // optional
   SearchTrace* trace = nullptr;     // optional; costs nothing when null
   // Optional: only labels the filter allows are returned. Must come from this index's make_filter().
@@ -141,6 +157,27 @@ std::string_view to_string(MissExplanation::Cause cause) noexcept;
 
 // Hierarchical Navigable Small World index (Malkov & Yashunin, 2016).
 //
+// How an index was configured, and why, saved in its file. search() callers that pass no settings
+// use k, ef, rerank and rerank_z from here (bindings fill them in; the C++ SearchOptions defaults are
+// unchanged). build_auto() fills it in; set_profile() sets it by hand.
+struct IndexProfile {
+  std::size_t k = 10;
+  std::size_t ef = 64;
+  std::size_t rerank = 0;
+  float rerank_z = 0.0f;
+  float recall_target = 0.0f;    // the recall ef was calibrated for (0: not calibrated)
+  float recall_measured = 0.0f;  // what calibration reached
+  bool built_on_codes = false;   // the build searched and pruned on 8-bit codes (fast_build)
+  bool refined = false;          // refine_links ran after the build
+  bool filled = false;           // fill_free_links ran after the build
+  // The prefetch setting chosen on `machine` (see machine_name()); a loaded index applies it only
+  // on the same kind of machine.
+  std::size_t prefetch_depth = std::numeric_limits<std::size_t>::max();
+  std::size_t prefetch_lines = 1;
+  std::string machine;
+  std::vector<std::pair<std::string, std::string>> notes;  // setting, why it was chosen
+};
+
 // Thread safety: add(), remove() and search() may be called concurrently from any number of
 // threads. train(), save() and compacted() must not run concurrently with add() or remove().
 //
@@ -199,6 +236,16 @@ class HnswIndex {
   // where the decoded approximations are used instead).
   std::unique_ptr<HnswIndex> compacted(std::size_t threads = 0) const;
 
+  // Once the index is built: gives each node's free layer-0 link slots to its closest two-hop
+  // neighbours. A cheaper way than fill_links to the same end, as links added here are never pruned
+  // again. Must not run concurrently with add() or remove(). Returns the number of links added.
+  std::size_t fill_free_links(std::size_t threads = 0);
+
+  // Once the index is built: every node searches the finished graph and reconnects, as its first
+  // neighbours were chosen from a sparser one. Must not run concurrently with add() or remove().
+  // `exact` searches and prunes on full vectors even on an index built with fast_build.
+  void refine_links(std::size_t threads = 0, std::size_t ef = 0, bool exact = false);  // ef 0: ef_construction
+
   void save(const std::string& path) const;
   // Loads a snapshot; corrupt or truncated files throw. With max_bytes, a snapshot whose index
   // would allocate more than that is refused before allocating (for untrusted files).
@@ -242,6 +289,10 @@ class HnswIndex {
   std::pair<std::size_t, std::size_t> tune_prefetch(const float* queries, std::size_t n, std::size_t k = 10,
                                                     std::size_t ef = 64);
 
+  const IndexProfile& profile() const noexcept { return profile_; }
+  // Not thread-safe against concurrent searches that read it; set it before serving.
+  void set_profile(IndexProfile profile) { profile_ = std::move(profile); }
+
  private:
   using id_t = std::uint32_t;
   struct Candidate {
@@ -270,6 +321,11 @@ class HnswIndex {
   // Full-precision vector: inside the layer-0 record (None) or in full_ (quantized + keep_vectors).
   const float* full_vector(id_t id) const noexcept;
   bool has_full_vectors() const noexcept { return !quantized() || params_.keep_vectors; }
+  // Bytes of SQ8 code per node, and the projection onto the walk's directions (nav_dim only).
+  std::size_t code_dim() const noexcept { return params_.nav_dim != 0 ? params_.nav_dim : params_.dim; }
+  void project(const float* v, float* out) const noexcept;
+  void project_query(const float* query, float* out) const;
+  void prepare_query_projection();
   const std::uint8_t* codes_of(id_t id) const noexcept {
     return reinterpret_cast<const std::uint8_t*>(record(id) + sizeof(float));
   }
@@ -342,6 +398,15 @@ class HnswIndex {
   // SQ8: per-dimension x = min + code * scale, and the kept full-precision vectors.
   std::vector<float> sq8_min_;
   std::vector<float> sq8_scale_;
+  bool walk_full_ = false;
+  bool build_exact_ = false;  // refine_links(exact): pair_distance uses full vectors for now
+  std::vector<float> nav_basis_;  // nav_dim: the directions, nav_dim rows of dim floats
+  DistanceFn dot_ = nullptr;      // nav_dim: 1 - dot, to project a vector
+  // nav_dim, for queries: nav_basis_ rounded to int8, one row at a time, with each row's scale and
+  // sum. Not saved: rebuilt from nav_basis_.
+  std::vector<std::int8_t> nav_ibasis_;
+  std::vector<float> nav_iscale_;
+  std::vector<std::int64_t> nav_isum_;
   std::unique_ptr<float[], AlignedFree> full_;
 
   // RaBitQ: the centroid, the rotation's random signs, and the padded (power of two) dimension.
@@ -371,6 +436,7 @@ class HnswIndex {
 
   std::size_t prefetch_depth_ = std::numeric_limits<std::size_t>::max();  // everything up front
   std::size_t prefetch_lines_ = 1;
+  IndexProfile profile_;
 };
 
 }  // namespace epzero

{epzero-0.3.0 → epzero-0.4.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "scikit_build_core.build"
 
 [project]
 name = "epzero"
-version = "0.3.0"
-description = "An HNSW vector database built from scratch in C++20: SIMD search, filtered search, recall monitoring, crash-safe storage"
+version = "0.4.0"
+description = "A fast HNSW vector database in C++20 that configures itself: SIMD search, filtered search, recall monitoring, crash-safe storage"
 readme = "python/README.md"
 requires-python = ">=3.10"
 license = { text = "MIT" }
@@ -41,6 +41,9 @@ wheel.packages = ["python/epzero"]
 EPZERO_BUILD_TESTS = "OFF"
 EPZERO_BUILD_BENCH = "OFF"
 EPZERO_BUILD_PYTHON = "ON"
+# The cluster node needs ../proto, outside the package: built from an sdist on a machine with gRPC
+# installed, it failed the whole install.
+EPZERO_BUILD_NODE = "OFF"
 
 [tool.cibuildwheel]
 build = "cp310-* cp311-* cp312-* cp313-*"

{epzero-0.3.0 → epzero-0.4.0}/python/README.md

@@ -2,9 +2,14 @@
 
 An HNSW vector database written from scratch in C++20, with Python bindings.
 
-- **Fast search:** hand-written SIMD kernels (AVX2, AVX-512, NEON), chosen at runtime, and
-  lock-free reads. On SIFT1M it serves 1.13× hnswlib's and 1.17× FAISS's queries per second at
-  equal recall, on the same machine.
+- **Configures itself:** `epzero.build(vectors)` picks compression, the graph build and the search
+  effort for your data and recall target, times memory prefetching on your CPU, saves the choices
+  inside the index and tells you why it made each one. Any setting can be overridden.
+- **Fast search:** hand-written SIMD kernels (AVX2, AVX-512, VNNI, NEON) chosen at runtime, 8-bit
+  codes, an optional walk on principal directions, and lock-free reads. On dedicated Intel Xeon Gold
+  6226 and AMD EPYC 7513 servers, one thread, at equal recall: 1.4-4.2× the queries per second of
+  Glass, the fastest library we compared, on GIST1M, OpenAI DBpedia and DEEP10M, and 1.0-1.3× on
+  GloVe-100 (with each dataset's best settings; see the project's scoreboard for every loss).
 - **Filtered search that picks its own strategy:** an exact scan, a filtered graph walk, or a
   two-hop walk (the idea behind ACORN), chosen per query by a cost model.
 - **Recall you can see:** `RecallGuard` measures recall on a sample of live queries with a
@@ -14,7 +19,8 @@ An HNSW vector database written from scratch in C++20, with Python bindings.
 - **Hybrid search:** `TextIndex` (BM25 keyword search) and `hybrid_search()`, which merges it
   with vector search by reciprocal rank fusion.
 - **Durable:** `Database` adds a write-ahead log with crash recovery and checkpoints.
-- **8-bit quantization** with exact re-ranking.
+- **8-bit quantization** with exact re-ranking, stopped early per query once the rest can't reach
+  the top k.
 - **A LangChain vector store** in `epzero.langchain`.
 
 ```bash
@@ -22,12 +28,45 @@ pip install epzero                 # the engine
 pip install "epzero[langchain]"    # plus the LangChain vector store
 ```
 
-## Quick start
+## Quick start: let it configure itself
 
 ```python
 import numpy as np
 import epzero
 
+vectors = np.random.rand(100_000, 768).astype(np.float32)
+index = epzero.build(vectors, metric="cosine")       # chooses every setting (below)
+labels, distances = index.search(vectors[:5], k=10)  # at the effort calibrated for recall 0.95
+print(index.plan)                                    # what it chose, and why
+```
+
+`build` decides from the data: 8-bit compression, whether to walk on principal directions (high
+dimensions with a concentrated spectrum), the build passes, the re-rank, and the beam width that
+reaches your recall target, measured on rows held out of the graph. From the machine: the SIMD
+kernel, the prefetch setting (timed on this CPU) and every core. Change anything:
+
+```python
+index = epzero.build(vectors, metric="cosine", recall=0.99)           # a different target
+index = epzero.build(vectors, metric="cosine", queries=my_queries)    # calibrate on real queries
+index = epzero.build(vectors, metric="cosine", nav_dim=0, refine=False, M=32)  # override choices
+plan = epzero.plan(vectors, metric="cosine")         # look before building, edit, then pass it in
+index = epzero.build(vectors, plan=plan.replace(ef_construction=400))
+labels, distances = index.search(queries, k=50, ef=200)               # per-call overrides
+index.save("my.epz")                                 # the settings and reasons are saved inside it
+index = epzero.Index.load("my.epz")                  # search() uses them again
+```
+
+`build` returns an ordinary `epzero.Index`: filters, deletes, `explain()` and `RecallGuard` all use
+its settings. A saved index re-applies its prefetch setting only on the same kind of CPU (call
+`tune_prefetch()` on another). The decisions are made in the C++ engine (`epzero/auto.h`), so C++
+users get the same `build_auto()`.
+
+If your queries differ from your data (text queries against image vectors, say), pass a few
+hundred of them as `queries=`: recall calibrated on data rows is optimistic for those.
+
+## Configured by hand
+
+```python
 vectors = np.random.rand(100_000, 128).astype(np.float32)
 index = epzero.Index(dim=128, metric="l2", max_elements=100_000, M=16, ef_construction=200)
 index.add(vectors)                                   # labels 0..n-1 (or pass labels=)

{epzero-0.3.0 → epzero-0.4.0}/python/epzero/__init__.py

@@ -3,11 +3,14 @@
     import numpy as np
     import epzero
 
-    index = epzero.Index(dim=128, metric="cosine")
-    index.add(vectors)                      # labels default to 0..n-1
-    labels, distances = index.search(queries, k=10, ef=64)
+    index = epzero.build(vectors, metric="cosine")   # configured for the data and this machine
+    labels, distances = index.search(queries, k=10)  # at the effort calibrated for recall 0.95
+    print(index.plan)                                # every setting, and why; pass any to override
 
-`Index` is the in-memory HNSW index (filtered search, 8-bit quantization, exact search and miss
+`build` chooses compression, the build passes, the re-rank and the beam width from the data, and
+times prefetching on this CPU (see epzero.auto); the settings are saved with the index. `Index` is
+the same engine configured by hand:
+the in-memory HNSW index (filtered search, 8-bit quantization, exact search and miss
 explanations); `Database` adds a write-ahead log and crash recovery; `RecallGuard` measures recall
 on a sample of live queries and moves the search effort to hold a target. `TextIndex` is keyword
 search (BM25), and `hybrid_search` merges it with vector search. A LangChain vector store is in
@@ -16,7 +19,8 @@ search (BM25), and `hybrid_search` merges it with vector search. A LangChain vec
 
 from importlib.metadata import PackageNotFoundError, version
 
-from ._core import Database, Filter, Index, RecallGuard, TextIndex, fuse_rankings, hybrid_search, simd_level
+from ._core import Database, Filter, Index, RecallGuard, TextIndex, fuse_rankings, hybrid_search, machine, simd_level
+from .auto import Plan, build, plan
 
 try:
     __version__ = version("epzero")
@@ -27,10 +31,14 @@ __all__ = [
     "Database",
     "Filter",
     "Index",
+    "Plan",
     "RecallGuard",
     "TextIndex",
+    "build",
     "fuse_rankings",
     "hybrid_search",
+    "machine",
+    "plan",
     "simd_level",
     "__version__",
 ]