numpy-vector-store 0.2.0__tar.gz → 0.3.0__tar.gz

{numpy_vector_store-0.2.0 → numpy_vector_store-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: numpy-vector-store
-Version: 0.2.0
+Version: 0.3.0
 Summary: A fast, lightweight, and zero-setup in-memory vector store powered by NumPy
 Project-URL: Homepage, https://github.com/tvanreenen/numpy-vector-store
 Project-URL: Repository, https://github.com/tvanreenen/numpy-vector-store
@@ -27,7 +27,7 @@ Description-Content-Type: text/markdown
 A fast, lightweight, zero-setup in-memory vector store powered by NumPy.
 
 - **Tiny local vector search** for projects that do not need a vector database
-- **Fast exact cosine search** using vectorized NumPy operations
+- **Fast exact vector search** using vectorized NumPy operations
 - **Simple typed API** returning `VectorHit(index, value, metadata)`
 - **Composable filtering** by passing prefiltered row indexes with `within_rows`
 - **Portable persistence** as trusted local `.npz` files with `vectors` + `metadata`
@@ -93,6 +93,58 @@ for hit in hits:
 dataclass, string, integer row ID, or any other Python object that fits your
 application.
 
+## Normalization
+
+`VectorStore` defaults to `normalize=True`, which scales each stored vector to
+length `1`. Normalization preserves vector direction while discarding magnitude:
+
+```python
+[3.0, 4.0] -> [0.6, 0.8]
+```
+
+This is the default because it makes cosine similarity fast and direction-only,
+which is the common case for semantic embeddings. Use `normalize=False` when
+vector length matters, such as when magnitude encodes strength, confidence,
+counts, scale, or raw geometry.
+
+Zero vectors are rejected in both modes because cosine similarity is undefined
+for zero-norm vectors.
+
+| Method | `normalize=True` default | `normalize=False` |
+|---|---|---|
+| `cosine_search` | True cosine similarity over stored unit vectors; fastest/default path for embeddings | True cosine similarity over raw vectors; computes vector norms during search |
+| `dot_search` | Dot product of unit vectors, effectively equivalent to cosine similarity | True dot product over original vectors; use when magnitude should affect ranking |
+| `euclidean_search` | Distance between normalized directions; useful only when direction-normalized distance is intended | True Euclidean distance over original vectors; use for geometric/feature-space nearest neighbors |
+| `get` | Returns normalized vectors | Returns original vectors |
+| `save` | Saves normalized vectors | Saves raw vectors |
+| `load` | Loads and normalizes vectors | Loads vectors exactly as stored |
+
+## Search Methods
+
+Use `cosine_search` for semantic embeddings and direction-only similarity:
+
+```python
+hits = store.cosine_search(query, top_k=10, min_value=0.75)
+```
+
+Use `dot_search` with `normalize=False` when larger-magnitude vectors should
+rank higher:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=3, normalize=False)
+store.add(vectors, metadata)
+hits = store.dot_search(query, top_k=10, min_value=0.0)
+```
+
+Use `euclidean_search` with `normalize=False` for raw coordinate or feature-space
+nearest-neighbor search:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=3, normalize=False)
+store.add(vectors, metadata)
+hits = store.euclidean_search(query, top_k=10, max_value=1.5)
+```
+
 ## Prefiltering
 
 The store does not implement a metadata query language. To filter by metadata,
@@ -146,39 +198,45 @@ loaded = VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz")
 loaded.load()
 ```
 
-Context manager usage auto-saves on exit:
+If you save with `normalize=False`, load with `normalize=False` too:
 
 ```python
-with VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz") as store:
-    store.add(embeddings, metadata)
-```
-
-Persistence uses a minimal NumPy `.npz` contract with `vectors` and `metadata`
-arrays. Vectors are normalized when added or loaded, and similarity search only
-normalizes the query vector. Loading validates shape, dimensions, row counts, and
-zero-norm vectors. It also uses `allow_pickle=True` for flexible Python metadata
-payloads, so only load files generated by your own application or another trusted
-local process. Loading untrusted `.npz` files is not a supported security model.
-
-## Migrating from 0.1
+store = VectorStore[dict[str, str]](
+    dimensions=1536,
+    file_path="raw-vectors.npz",
+    normalize=False,
+)
+store.add(raw_vectors, metadata)
+store.save()
 
-The preferred 0.2 API is `add(...)` and `cosine_search(...)`.
+loaded = VectorStore[dict[str, str]](
+    dimensions=1536,
+    file_path="raw-vectors.npz",
+    normalize=False,
+)
+loaded.load()
+```
 
-The 0.1 methods remain temporarily available, but emit `DeprecationWarning` and
-will be removed in a future 0.x release:
+Context manager usage auto-saves on exit:
 
 ```python
-store.add_vectors(vectors_2d, metadata_array)
-results = store.search(query, top_k=3, score_cutoff=0.5)
-for index, value, metadata in results:
-    ...
+with VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz") as store:
+    store.add(embeddings, metadata)
 ```
 
-Legacy `search(...)` keeps returning tuples. New `cosine_search(...)` returns
-`VectorHit` objects.
-
-`metadata_schema` was removed. For vectorized metadata filtering, keep metadata
-in a sidecar table and pass matching row indexes with `within_rows`.
+Persistence uses a minimal NumPy `.npz` contract with `vectors` and `metadata`
+arrays. The `.npz` file does not encode the `normalize` setting; choose the same
+setting when loading that you used when saving. Loading validates shape,
+dimensions, row counts, and zero-norm vectors. It also uses `allow_pickle=True`
+for flexible Python metadata payloads, so only load files generated by your own
+application or another trusted local process. Loading untrusted `.npz` files is
+not a supported security model.
+
+## Compatibility
+
+This project is still pre-1.0, so occasional breaking changes are expected while
+the API stabilizes. Breaking changes are documented in GitHub release notes.
+Deprecated APIs will keep warning for at least one point release before removal.
 
 ## Contributing
 

{numpy_vector_store-0.2.0 → numpy_vector_store-0.3.0}/README.md

@@ -3,7 +3,7 @@
 A fast, lightweight, zero-setup in-memory vector store powered by NumPy.
 
 - **Tiny local vector search** for projects that do not need a vector database
-- **Fast exact cosine search** using vectorized NumPy operations
+- **Fast exact vector search** using vectorized NumPy operations
 - **Simple typed API** returning `VectorHit(index, value, metadata)`
 - **Composable filtering** by passing prefiltered row indexes with `within_rows`
 - **Portable persistence** as trusted local `.npz` files with `vectors` + `metadata`
@@ -69,6 +69,58 @@ for hit in hits:
 dataclass, string, integer row ID, or any other Python object that fits your
 application.
 
+## Normalization
+
+`VectorStore` defaults to `normalize=True`, which scales each stored vector to
+length `1`. Normalization preserves vector direction while discarding magnitude:
+
+```python
+[3.0, 4.0] -> [0.6, 0.8]
+```
+
+This is the default because it makes cosine similarity fast and direction-only,
+which is the common case for semantic embeddings. Use `normalize=False` when
+vector length matters, such as when magnitude encodes strength, confidence,
+counts, scale, or raw geometry.
+
+Zero vectors are rejected in both modes because cosine similarity is undefined
+for zero-norm vectors.
+
+| Method | `normalize=True` default | `normalize=False` |
+|---|---|---|
+| `cosine_search` | True cosine similarity over stored unit vectors; fastest/default path for embeddings | True cosine similarity over raw vectors; computes vector norms during search |
+| `dot_search` | Dot product of unit vectors, effectively equivalent to cosine similarity | True dot product over original vectors; use when magnitude should affect ranking |
+| `euclidean_search` | Distance between normalized directions; useful only when direction-normalized distance is intended | True Euclidean distance over original vectors; use for geometric/feature-space nearest neighbors |
+| `get` | Returns normalized vectors | Returns original vectors |
+| `save` | Saves normalized vectors | Saves raw vectors |
+| `load` | Loads and normalizes vectors | Loads vectors exactly as stored |
+
+## Search Methods
+
+Use `cosine_search` for semantic embeddings and direction-only similarity:
+
+```python
+hits = store.cosine_search(query, top_k=10, min_value=0.75)
+```
+
+Use `dot_search` with `normalize=False` when larger-magnitude vectors should
+rank higher:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=3, normalize=False)
+store.add(vectors, metadata)
+hits = store.dot_search(query, top_k=10, min_value=0.0)
+```
+
+Use `euclidean_search` with `normalize=False` for raw coordinate or feature-space
+nearest-neighbor search:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=3, normalize=False)
+store.add(vectors, metadata)
+hits = store.euclidean_search(query, top_k=10, max_value=1.5)
+```
+
 ## Prefiltering
 
 The store does not implement a metadata query language. To filter by metadata,
@@ -122,39 +174,45 @@ loaded = VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz")
 loaded.load()
 ```
 
-Context manager usage auto-saves on exit:
+If you save with `normalize=False`, load with `normalize=False` too:
 
 ```python
-with VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz") as store:
-    store.add(embeddings, metadata)
-```
-
-Persistence uses a minimal NumPy `.npz` contract with `vectors` and `metadata`
-arrays. Vectors are normalized when added or loaded, and similarity search only
-normalizes the query vector. Loading validates shape, dimensions, row counts, and
-zero-norm vectors. It also uses `allow_pickle=True` for flexible Python metadata
-payloads, so only load files generated by your own application or another trusted
-local process. Loading untrusted `.npz` files is not a supported security model.
-
-## Migrating from 0.1
+store = VectorStore[dict[str, str]](
+    dimensions=1536,
+    file_path="raw-vectors.npz",
+    normalize=False,
+)
+store.add(raw_vectors, metadata)
+store.save()
 
-The preferred 0.2 API is `add(...)` and `cosine_search(...)`.
+loaded = VectorStore[dict[str, str]](
+    dimensions=1536,
+    file_path="raw-vectors.npz",
+    normalize=False,
+)
+loaded.load()
+```
 
-The 0.1 methods remain temporarily available, but emit `DeprecationWarning` and
-will be removed in a future 0.x release:
+Context manager usage auto-saves on exit:
 
 ```python
-store.add_vectors(vectors_2d, metadata_array)
-results = store.search(query, top_k=3, score_cutoff=0.5)
-for index, value, metadata in results:
-    ...
+with VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz") as store:
+    store.add(embeddings, metadata)
 ```
 
-Legacy `search(...)` keeps returning tuples. New `cosine_search(...)` returns
-`VectorHit` objects.
-
-`metadata_schema` was removed. For vectorized metadata filtering, keep metadata
-in a sidecar table and pass matching row indexes with `within_rows`.
+Persistence uses a minimal NumPy `.npz` contract with `vectors` and `metadata`
+arrays. The `.npz` file does not encode the `normalize` setting; choose the same
+setting when loading that you used when saving. Loading validates shape,
+dimensions, row counts, and zero-norm vectors. It also uses `allow_pickle=True`
+for flexible Python metadata payloads, so only load files generated by your own
+application or another trusted local process. Loading untrusted `.npz` files is
+not a supported security model.
+
+## Compatibility
+
+This project is still pre-1.0, so occasional breaking changes are expected while
+the API stabilizes. Breaking changes are documented in GitHub release notes.
+Deprecated APIs will keep warning for at least one point release before removal.
 
 ## Contributing
 

{numpy_vector_store-0.2.0 → numpy_vector_store-0.3.0}/src/numpy_vector_store/__init__.py

@@ -5,7 +5,7 @@ This package provides a lightweight vector store implementation for storing
 and searching vector embeddings using NumPy arrays.
 """
 
-__version__ = "0.2.0"
+__version__ = "0.3.0"
 __author__ = "Tim VanReenen"
 
 from .vector_store import VectorHit, VectorStore

{numpy_vector_store-0.2.0 → numpy_vector_store-0.3.0}/src/numpy_vector_store/vector_store.py

@@ -1,7 +1,6 @@
 from __future__ import annotations
 
-import warnings
-from collections.abc import Sequence
+from collections.abc import Callable, Sequence
 from dataclasses import dataclass
 from pathlib import Path
 from typing import Any, Generic, TypeVar
@@ -23,17 +22,19 @@ class VectorHit(Generic[TMetadata]):
 
 class VectorStore(Generic[TMetadata]):
     """
-    A small in-memory vector store using NumPy cosine similarity.
+    A small in-memory vector store using NumPy exact vector search.
 
     Metadata is an opaque row payload returned with vector hits. The store does
     not implement a metadata query language; pass row indexes to within_rows for
-    prefiltered similarity search.
+    prefiltered vector search.
     """
 
     def __init__(
         self,
         dimensions: int,
         file_path: str | Path | None = None,
+        *,
+        normalize: bool = True,
     ) -> None:
         """
         Initialize the vector store.
@@ -41,12 +42,14 @@ class VectorStore(Generic[TMetadata]):
         Args:
             dimensions: The number of dimensions for vectors to be stored.
             file_path: Optional path to save/load vectors from.
+            normalize: Whether to store vectors normalized to unit length.
         """
         if dimensions <= 0:
             raise ValueError("dimensions must be greater than 0")
 
         self.dimensions = dimensions
         self.file_path = Path(file_path) if file_path else None
+        self.normalize = normalize
         self.vectors: npt.NDArray[np.float32] = np.empty(
             (0, dimensions), dtype=np.float32
         )
@@ -59,8 +62,8 @@ class VectorStore(Generic[TMetadata]):
         """
         Add vectors and row metadata payloads.
 
-        Vectors are normalized at insert time. Metadata items are stored as
-        opaque payloads and returned with vector hits.
+        When normalize=True, vectors are normalized at insert time. Metadata
+        items are stored as opaque payloads and returned with vector hits.
         """
         vectors_2d = np.asarray(vectors, dtype=np.float32)
         if vectors_2d.ndim != 2:
@@ -78,40 +81,20 @@ class VectorStore(Generic[TMetadata]):
         if len(vectors_2d) != len(metadata_array):
             raise ValueError("Number of vectors must match number of metadata items")
 
-        normalized_vectors = self._normalize_vectors(
+        vectors_to_store = self._prepare_vectors_for_storage(
             vectors_2d,
             error_message="Cannot add zero-norm vectors",
         )
 
         if len(self.vectors) == 0:
-            self.vectors = normalized_vectors
+            self.vectors = vectors_to_store
         else:
-            self.vectors = np.vstack([self.vectors, normalized_vectors]).astype(
+            self.vectors = np.vstack([self.vectors, vectors_to_store]).astype(
                 np.float32, copy=False
             )
 
         self.metadata = np.append(self.metadata, metadata_array)
 
-    def add_vectors(self, vectors_2d: np.ndarray, metadata_array: np.ndarray) -> None:
-        """
-        Deprecated API for adding vectors and metadata arrays.
-
-        Use add(...) instead.
-        """
-        warnings.warn(
-            "VectorStore.add_vectors() is deprecated and will be removed in a future 0.x release. "
-            "Use VectorStore.add() instead.",
-            DeprecationWarning,
-            stacklevel=2,
-        )
-        if vectors_2d.ndim != 2:
-            raise ValueError("vectors_2d must be a 2D NumPy array")
-
-        if metadata_array.ndim != 1:
-            raise ValueError("metadata_array must be a 1D NumPy array")
-
-        self.add(vectors_2d, metadata_array.tolist())
-
     def load(self) -> None:
         """Load vectors from file if file_path is specified and exists."""
         if self._loaded or not self.file_path:
@@ -126,7 +109,7 @@ class VectorStore(Generic[TMetadata]):
                 loaded_metadata = np.array(data["metadata"], copy=True)
 
             self._validate_loaded_arrays(loaded_vectors, loaded_metadata)
-            loaded_vectors = self._normalize_vectors(
+            loaded_vectors = self._prepare_vectors_for_storage(
                 loaded_vectors,
                 error_message="Loaded vectors contain zero-norm vectors",
             )
@@ -164,87 +147,98 @@ class VectorStore(Generic[TMetadata]):
             min_value: Optional minimum cosine similarity value.
             within_rows: Optional row indexes to restrict search to.
         """
-        query_vector = self._validate_query(query)
-
-        if top_k <= 0:
-            raise ValueError("top_k must be greater than 0")
-
-        if len(self.vectors) == 0:
-            return []
+        return self._metric_search(
+            query,
+            top_k=top_k,
+            within_rows=within_rows,
+            values_fn=self._cosine_values,
+            descending=True,
+            min_value=min_value,
+            max_value=None,
+        )
 
-        row_indices = self._normalize_within_rows(within_rows)
-        if len(row_indices) == 0:
-            return []
+    def dot_search(
+        self,
+        query: npt.ArrayLike,
+        *,
+        top_k: int = 10,
+        min_value: float | None = None,
+        within_rows: Sequence[int] | npt.NDArray[np.integer[Any]] | None = None,
+    ) -> list[VectorHit[TMetadata]]:
+        """
+        Return rows ranked by dot product.
 
-        similarities = self._cosine_similarity_numpy(
-            query_vector, self.vectors[row_indices]
+        With normalize=True, this is the dot product of unit vectors. With
+        normalize=False, this is the true dot product over original vectors.
+        """
+        return self._metric_search(
+            query,
+            top_k=top_k,
+            within_rows=within_rows,
+            values_fn=self._dot_values,
+            descending=True,
+            min_value=min_value,
+            max_value=None,
         )
 
-        if min_value is not None:
-            valid_local_indices = np.flatnonzero(similarities >= min_value)
-        else:
-            valid_local_indices = np.arange(len(similarities))
-
-        if len(valid_local_indices) == 0:
-            return []
+    def euclidean_search(
+        self,
+        query: npt.ArrayLike,
+        *,
+        top_k: int = 10,
+        max_value: float | None = None,
+        within_rows: Sequence[int] | npt.NDArray[np.integer[Any]] | None = None,
+    ) -> list[VectorHit[TMetadata]]:
+        """
+        Return rows ranked by Euclidean distance.
 
-        valid_similarities = similarities[valid_local_indices]
-        result_count = min(top_k, len(valid_local_indices))
-        if result_count < len(valid_local_indices):
-            top_local_unsorted = np.argpartition(valid_similarities, -result_count)[
-                -result_count:
-            ]
-        else:
-            top_local_unsorted = np.arange(len(valid_similarities))
+        With normalize=True, this is distance between normalized directions.
+        With normalize=False, this is true Euclidean distance over original
+        vectors.
+        """
+        return self._metric_search(
+            query,
+            top_k=top_k,
+            within_rows=within_rows,
+            values_fn=self._euclidean_values,
+            descending=False,
+            min_value=None,
+            max_value=max_value,
+        )
 
-        top_local_sorted = top_local_unsorted[
-            np.argsort(valid_similarities[top_local_unsorted])[::-1]
-        ]
-        local_indices = valid_local_indices[top_local_sorted]
-        original_indices = row_indices[local_indices]
+    def _cosine_values(
+        self, query: npt.NDArray[np.float32], vectors: npt.NDArray[np.float32]
+    ) -> npt.NDArray[np.float32]:
+        """Compute cosine similarity values."""
+        query_norm = self._normalize_query(query)
+        values = np.dot(vectors, query_norm)
 
-        return [
-            VectorHit(
-                index=int(original_idx),
-                value=float(similarities[local_idx]),
-                metadata=self.metadata[original_idx],
-            )
-            for local_idx, original_idx in zip(
-                local_indices, original_indices, strict=True
-            )
-        ]
+        if not self.normalize:
+            vector_norms = np.linalg.norm(vectors, axis=1)
+            if np.any(vector_norms == 0):
+                raise ValueError("Cannot cosine search zero-norm stored vectors")
+            values = values / vector_norms
 
-    def search(
-        self, query_vector: np.ndarray, top_k: int = 10, score_cutoff: float = 0.0
-    ) -> list[tuple[int, float, TMetadata]]:
-        """
-        Deprecated tuple-returning similarity search.
+        return np.asarray(values, dtype=np.float32)
 
-        Use cosine_search(...), which returns VectorHit objects and uses
-        min_value instead.
-        """
-        warnings.warn(
-            "VectorStore.search() is deprecated and will be removed in a future 0.x release. "
-            "Use VectorStore.cosine_search() instead.",
-            DeprecationWarning,
-            stacklevel=2,
-        )
-        hits = self.cosine_search(query_vector, top_k=top_k, min_value=score_cutoff)
-        return [(hit.index, hit.value, hit.metadata) for hit in hits]
+    def _dot_values(
+        self, query: npt.NDArray[np.float32], vectors: npt.NDArray[np.float32]
+    ) -> npt.NDArray[np.float32]:
+        """Compute dot product values."""
+        if self.normalize:
+            query = self._normalize_query(query)
+        return np.asarray(np.dot(vectors, query), dtype=np.float32)
 
-    def _cosine_similarity_numpy(
+    def _euclidean_values(
         self, query: npt.NDArray[np.float32], vectors: npt.NDArray[np.float32]
     ) -> npt.NDArray[np.float32]:
-        """Compute cosine similarity against normalized vectors."""
-        query_magnitude = np.linalg.norm(query)
-        if query_magnitude == 0:
-            raise ValueError("Cannot search with zero-norm query vector")
-        query_norm = query / query_magnitude
-        similarities = np.asarray(np.dot(vectors, query_norm), dtype=np.float32)
-        return similarities
+        """Compute Euclidean distance values."""
+        if self.normalize:
+            query = self._normalize_query(query)
+        return np.asarray(np.linalg.norm(vectors - query, axis=1), dtype=np.float32)
 
     def get(self, index: int) -> tuple[npt.NDArray[np.float32], TMetadata] | None:
-        """Get a normalized vector and metadata payload by row index."""
+        """Get a stored vector and metadata payload by row index."""
         if 0 <= index < len(self.vectors):
             return (self.vectors[index], self.metadata[index])
         return None
@@ -272,13 +266,21 @@ class VectorStore(Generic[TMetadata]):
     ) -> npt.NDArray[Any]:
         return np.asarray(metadata, dtype=object)
 
-    def _normalize_vectors(
+    def _prepare_vectors_for_storage(
         self, vectors: npt.NDArray[np.float32], *, error_message: str
     ) -> npt.NDArray[np.float32]:
-        norms = np.linalg.norm(vectors, axis=1, keepdims=True)
+        norms = self._validate_non_zero_vectors(vectors, error_message=error_message)
+        if self.normalize:
+            return np.asarray(vectors / norms[:, np.newaxis], dtype=np.float32)
+        return vectors.astype(np.float32, copy=True)
+
+    def _validate_non_zero_vectors(
+        self, vectors: npt.NDArray[np.float32], *, error_message: str
+    ) -> npt.NDArray[np.float32]:
+        norms = np.linalg.norm(vectors, axis=1)
         if np.any(norms == 0):
             raise ValueError(error_message)
-        return np.asarray(vectors / norms, dtype=np.float32)
+        return np.asarray(norms, dtype=np.float32)
 
     def _validate_query(self, query: npt.ArrayLike) -> npt.NDArray[np.float32]:
         query_vector = np.asarray(query, dtype=np.float32)
@@ -290,6 +292,14 @@ class VectorStore(Generic[TMetadata]):
             )
         return query_vector
 
+    def _normalize_query(
+        self, query: npt.NDArray[np.float32]
+    ) -> npt.NDArray[np.float32]:
+        query_magnitude = np.linalg.norm(query)
+        if query_magnitude == 0:
+            raise ValueError("Cannot search with zero-norm query vector")
+        return np.asarray(query / query_magnitude, dtype=np.float32)
+
     def _normalize_within_rows(
         self, within_rows: Sequence[int] | npt.NDArray[np.integer[Any]] | None
     ) -> npt.NDArray[np.intp]:
@@ -309,6 +319,80 @@ class VectorStore(Generic[TMetadata]):
             raise IndexError("within_rows contains row indexes outside the store")
         return rows
 
+    def _metric_search(
+        self,
+        query: npt.ArrayLike,
+        *,
+        top_k: int,
+        within_rows: Sequence[int] | npt.NDArray[np.integer[Any]] | None,
+        values_fn: Callable[
+            [npt.NDArray[np.float32], npt.NDArray[np.float32]],
+            npt.NDArray[np.float32],
+        ],
+        descending: bool,
+        min_value: float | None,
+        max_value: float | None,
+    ) -> list[VectorHit[TMetadata]]:
+        query_vector = self._validate_query(query)
+
+        if top_k <= 0:
+            raise ValueError("top_k must be greater than 0")
+
+        if len(self.vectors) == 0:
+            return []
+
+        row_indices = self._normalize_within_rows(within_rows)
+        if len(row_indices) == 0:
+            return []
+
+        values = values_fn(query_vector, self.vectors[row_indices])
+
+        valid_local_indices = np.arange(len(values))
+        if min_value is not None:
+            valid_local_indices = valid_local_indices[
+                values[valid_local_indices] >= min_value
+            ]
+        if max_value is not None:
+            valid_local_indices = valid_local_indices[
+                values[valid_local_indices] <= max_value
+            ]
+
+        if len(valid_local_indices) == 0:
+            return []
+
+        valid_values = values[valid_local_indices]
+        result_count = min(top_k, len(valid_local_indices))
+        if result_count < len(valid_local_indices):
+            if descending:
+                top_valid_unsorted = np.argpartition(valid_values, -result_count)[
+                    -result_count:
+                ]
+            else:
+                top_valid_unsorted = np.argpartition(valid_values, result_count - 1)[
+                    :result_count
+                ]
+        else:
+            top_valid_unsorted = np.arange(len(valid_values))
+
+        sort_order = np.argsort(valid_values[top_valid_unsorted])
+        if descending:
+            sort_order = sort_order[::-1]
+
+        top_valid_sorted = top_valid_unsorted[sort_order]
+        local_indices = valid_local_indices[top_valid_sorted]
+        original_indices = row_indices[local_indices]
+
+        return [
+            VectorHit(
+                index=int(original_idx),
+                value=float(values[local_idx]),
+                metadata=self.metadata[original_idx],
+            )
+            for local_idx, original_idx in zip(
+                local_indices, original_indices, strict=True
+            )
+        ]
+
     def _validate_required_fields(self, files: set[str]) -> None:
         if "vectors" not in files:
             raise ValueError("Persisted vector store is missing vectors")

{numpy_vector_store-0.2.0 → numpy_vector_store-0.3.0}/tests/test_vector_store.py

@@ -22,9 +22,16 @@ class TestVectorStore:
         """Test VectorStore initialization."""
         store = VectorStore(dimensions=3)
         assert store.dimensions == 3
+        assert store.normalize is True
         assert len(store.vectors) == 0
         assert len(store) == 0
 
+    def test_init_allows_raw_vector_storage(self):
+        """Test VectorStore can preserve raw vectors."""
+        store = VectorStore(dimensions=3, normalize=False)
+
+        assert store.normalize is False
+
     def test_init_rejects_non_positive_dimensions(self):
         """Test VectorStore rejects invalid dimensions."""
         with pytest.raises(ValueError, match="dimensions"):
@@ -47,6 +54,15 @@ class TestVectorStore:
 
         np.testing.assert_array_almost_equal(store.vectors[0], np.array([0.6, 0.8]))
 
+    def test_add_preserves_raw_vectors_when_normalize_false(self):
+        """Test add preserves original vectors when normalize=False."""
+        store = VectorStore[dict[str, str]](dimensions=2, normalize=False)
+
+        store.add([[3.0, 4.0]], [{"id": "v"}])
+
+        np.testing.assert_array_equal(store.vectors[0], np.array([3.0, 4.0]))
+        np.testing.assert_array_equal(store.get(0)[0], np.array([3.0, 4.0]))
+
     def test_add_rejects_wrong_dimension(self):
         """Test add rejects vectors with wrong dimensions."""
         store = VectorStore(dimensions=3)
@@ -82,6 +98,13 @@ class TestVectorStore:
         with pytest.raises(ValueError, match="zero-norm"):
             store.add([[0.0, 0.0, 0.0]], [{"id": "zero"}])
 
+    def test_add_rejects_zero_norm_vector_when_normalize_false(self):
+        """Test add rejects zero vectors because cosine search is always available."""
+        store = VectorStore(dimensions=3, normalize=False)
+
+        with pytest.raises(ValueError, match="zero-norm"):
+            store.add([[0.0, 0.0, 0.0]], [{"id": "zero"}])
+
     def test_cosine_search_returns_vector_hits(self):
         """Test preferred cosine_search returns typed vector hits."""
         store = VectorStore[dict[str, str]](dimensions=3)
@@ -217,6 +240,116 @@ class TestVectorStore:
 
         assert results == []
 
+    def test_cosine_search_with_raw_vectors(self):
+        """Test cosine_search computes true cosine when normalize=False."""
+        store = VectorStore[dict[str, str]](dimensions=2, normalize=False)
+        store.add(
+            [[10.0, 0.0], [0.0, 2.0], [3.0, 4.0]],
+            [{"id": "x"}, {"id": "y"}, {"id": "diag"}],
+        )
+
+        results = store.cosine_search([1.0, 0.0], top_k=3)
+
+        assert [hit.index for hit in results] == [0, 2, 1]
+        assert [hit.value for hit in results] == pytest.approx([1.0, 0.6, 0.0])
+
+    def test_dot_search_with_raw_vectors(self):
+        """Test dot_search ranks by true dot product when normalize=False."""
+        store = VectorStore[dict[str, str]](dimensions=2, normalize=False)
+        store.add(
+            [[10.0, 0.0], [0.0, 2.0], [3.0, 4.0]],
+            [{"id": "x"}, {"id": "y"}, {"id": "diag"}],
+        )
+
+        results = store.dot_search([1.0, 0.0], top_k=3)
+
+        assert all(isinstance(hit, VectorHit) for hit in results)
+        assert [hit.index for hit in results] == [0, 2, 1]
+        assert [hit.value for hit in results] == pytest.approx([10.0, 3.0, 0.0])
+
+    def test_dot_search_with_normalized_vectors(self):
+        """Test dot_search uses stored unit vectors when normalize=True."""
+        store = VectorStore[dict[str, str]](dimensions=2)
+        store.add(
+            [[10.0, 0.0], [3.0, 4.0], [0.0, 2.0]],
+            [{"id": "x"}, {"id": "diag"}, {"id": "y"}],
+        )
+
+        results = store.dot_search([2.0, 0.0], top_k=3, min_value=0.5)
+
+        assert [hit.index for hit in results] == [0, 1]
+        assert [hit.value for hit in results] == pytest.approx([1.0, 0.6])
+
+    def test_euclidean_search_with_raw_vectors(self):
+        """Test euclidean_search ranks by nearest raw vector when normalize=False."""
+        store = VectorStore[dict[str, str]](dimensions=2, normalize=False)
+        store.add(
+            [[1.0, 1.0], [4.0, 5.0], [2.0, 1.0]],
+            [{"id": "near"}, {"id": "far"}, {"id": "also-near"}],
+        )
+
+        results = store.euclidean_search([1.0, 2.0], top_k=3)
+
+        assert all(isinstance(hit, VectorHit) for hit in results)
+        assert [hit.index for hit in results] == [0, 2, 1]
+        assert [hit.value for hit in results] == pytest.approx(
+            [1.0, np.sqrt(2.0), np.sqrt(18.0)]
+        )
+
+    def test_euclidean_search_with_max_value(self):
+        """Test euclidean_search max_value filters by distance."""
+        store = VectorStore[dict[str, str]](dimensions=2, normalize=False)
+        store.add(
+            [[1.0, 1.0], [4.0, 5.0], [2.0, 1.0]],
+            [{"id": "near"}, {"id": "far"}, {"id": "also-near"}],
+        )
+
+        results = store.euclidean_search([1.0, 2.0], top_k=3, max_value=1.5)
+
+        assert [hit.index for hit in results] == [0, 2]
+
+    def test_euclidean_search_with_normalized_vectors(self):
+        """Test euclidean_search uses normalized stored vectors by default."""
+        store = VectorStore[dict[str, str]](dimensions=2)
+        store.add(
+            [[10.0, 0.0], [3.0, 4.0]],
+            [{"id": "x"}, {"id": "diag"}],
+        )
+
+        results = store.euclidean_search([2.0, 0.0], top_k=2)
+
+        assert [hit.index for hit in results] == [0, 1]
+        assert [hit.value for hit in results] == pytest.approx([0.0, np.sqrt(0.8)])
+
+    def test_metric_searches_support_within_rows(self):
+        """Test dot and Euclidean search support prefiltered row indexes."""
+        store = VectorStore[dict[str, str]](dimensions=2, normalize=False)
+        store.add(
+            [[10.0, 0.0], [0.0, 2.0], [3.0, 4.0]],
+            [{"id": "x"}, {"id": "y"}, {"id": "diag"}],
+        )
+
+        dot_results = store.dot_search([1.0, 0.0], within_rows=[1, 2])
+        euclidean_results = store.euclidean_search([1.0, 0.0], within_rows=[1, 2])
+
+        assert [hit.index for hit in dot_results] == [2, 1]
+        assert [hit.index for hit in euclidean_results] == [1, 2]
+
+    def test_metric_searches_validate_common_inputs(self):
+        """Test added search methods share common validation behavior."""
+        store = VectorStore(dimensions=2)
+        store.add([[1.0, 0.0]], [{"id": "x"}])
+
+        with pytest.raises(ValueError, match="top_k"):
+            store.dot_search([1.0, 0.0], top_k=0)
+
+        with pytest.raises(ValueError, match="Query vector dimension"):
+            store.euclidean_search([1.0])
+
+        assert store.dot_search([1.0, 0.0], within_rows=[]) == []
+        with pytest.raises(IndexError, match="outside"):
+            store.euclidean_search([1.0, 0.0], within_rows=[1])
+
     def test_get(self):
         """Test retrieving vector and metadata by index."""
         store = VectorStore(dimensions=2)
@@ -442,6 +575,27 @@ class TestVectorStore:
         finally:
             Path(file_path).unlink(missing_ok=True)
 
+    def test_load_preserves_raw_vectors_when_normalize_false(self):
+        """Test load preserves valid vectors when normalize=False."""
+        with tempfile.NamedTemporaryFile(suffix=".npz", delete=False) as tmp:
+            file_path = tmp.name
+
+        try:
+            np.savez_compressed(
+                file_path,
+                vectors=np.array([[2.0, 0.0]], dtype=np.float32),
+                metadata=np.array([{"id": "test"}], dtype=object),
+            )
+
+            store = VectorStore(dimensions=2, file_path=file_path, normalize=False)
+            store.load()
+
+            np.testing.assert_array_equal(store.vectors[0], np.array([2.0, 0.0]))
+            assert store.cosine_search([1.0, 0.0])[0].value == pytest.approx(1.0)
+            assert store.dot_search([1.0, 0.0])[0].value == pytest.approx(2.0)
+        finally:
+            Path(file_path).unlink(missing_ok=True)
+
     def test_load_raises_on_zero_norm_vectors(self):
         """Test load rejects persisted zero-norm vectors."""
         with tempfile.NamedTemporaryFile(suffix=".npz", delete=False) as tmp:
@@ -460,6 +614,24 @@ class TestVectorStore:
         finally:
             Path(file_path).unlink(missing_ok=True)
 
+    def test_load_raises_on_zero_norm_vectors_when_normalize_false(self):
+        """Test raw stores reject zero rows because cosine search is always available."""
+        with tempfile.NamedTemporaryFile(suffix=".npz", delete=False) as tmp:
+            file_path = tmp.name
+
+        try:
+            np.savez_compressed(
+                file_path,
+                vectors=np.array([[0.0, 0.0]], dtype=np.float32),
+                metadata=np.array([{"id": "zero"}], dtype=object),
+            )
+
+            store = VectorStore(dimensions=2, file_path=file_path, normalize=False)
+            with pytest.raises(ValueError, match="zero-norm"):
+                store.load()
+        finally:
+            Path(file_path).unlink(missing_ok=True)
+
     def test_save_no_file_path(self):
         """Test save with no file path."""
         store = VectorStore(dimensions=2)
@@ -498,56 +670,24 @@ class TestVectorStore:
         finally:
             Path(file_path).unlink(missing_ok=True)
 
-    def test_add_vectors_deprecated_compatibility_api(self):
-        """Test deprecated add_vectors remains compatible temporarily."""
-        store = VectorStore(dimensions=2)
-        vectors_2d = np.array([[1.0, 2.0], [3.0, 4.0]])
-        metadata_array = np.array([{"id": 1}, {"id": 2}])
-
-        with pytest.deprecated_call(match="add_vectors"):
-            store.add_vectors(vectors_2d, metadata_array)
-
-        assert len(store) == 2
-        assert store.get(0)[1] == {"id": 1}
-
-    def test_add_vectors_deprecated_validation(self):
-        """Test deprecated add_vectors still preserves validation."""
-        store = VectorStore(dimensions=2)
-
-        with pytest.raises(ValueError, match="2D"):
-            with pytest.deprecated_call(match="add_vectors"):
-                store.add_vectors(np.array([1.0, 2.0]), np.array([{"id": 1}]))
-
-        with pytest.raises(ValueError, match="1D"):
-            with pytest.deprecated_call(match="add_vectors"):
-                store.add_vectors(np.array([[1.0, 2.0]]), np.array([[{"id": 1}]]))
-
-    def test_search_deprecated_compatibility_api(self):
-        """Test deprecated search remains compatible temporarily."""
-        store = VectorStore(dimensions=3)
-        store.add(
-            [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]],
-            [{"name": "x-axis"}, {"name": "y-axis"}],
-        )
-
-        with pytest.deprecated_call(match="search"):
-            results = store.search(np.array([0.9, 0.1, 0.0]), top_k=2)
+    def test_save_and_load_raw_vectors_round_trip(self):
+        """Test normalize=False saves and loads raw vectors without mode metadata."""
+        with tempfile.NamedTemporaryFile(suffix=".npz", delete=False) as tmp:
+            file_path = tmp.name
 
-        assert len(results) == 2
-        assert results[0][0] == 0
-        assert results[0][1] > 0.9
-        assert results[0][2] == {"name": "x-axis"}
+        try:
+            store1 = VectorStore(dimensions=2, file_path=file_path, normalize=False)
+            store1.add([[3.0, 4.0]], [{"id": "raw"}])
+            store1.save()
 
-    def test_search_deprecated_score_cutoff(self):
-        """Test deprecated search still supports score_cutoff."""
-        store = VectorStore(dimensions=3)
-        store.add(
-            [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]],
-            [{"name": "x-axis"}, {"name": "y-axis"}],
-        )
+            with np.load(file_path, allow_pickle=True) as data:
+                assert set(data.files) == {"vectors", "metadata"}
+                np.testing.assert_array_equal(data["vectors"][0], np.array([3.0, 4.0]))
 
-        with pytest.deprecated_call(match="search"):
-            results = store.search(np.array([0.9, 0.1, 0.0]), score_cutoff=0.8)
+            store2 = VectorStore(dimensions=2, file_path=file_path, normalize=False)
+            store2.load()
 
-        assert len(results) == 1
-        assert results[0][0] == 0
+            np.testing.assert_array_equal(store2.get(0)[0], np.array([3.0, 4.0]))
+            assert store2.get(0)[1] == {"id": "raw"}
+        finally:
+            Path(file_path).unlink(missing_ok=True)