numpy-vector-store 0.1.0__tar.gz → 0.3.0__tar.gz

numpy_vector_store-0.3.0/PKG-INFO

@@ -0,0 +1,258 @@
+Metadata-Version: 2.4
+Name: numpy-vector-store
+Version: 0.3.0
+Summary: A fast, lightweight, and zero-setup in-memory vector store powered by NumPy
+Project-URL: Homepage, https://github.com/tvanreenen/numpy-vector-store
+Project-URL: Repository, https://github.com/tvanreenen/numpy-vector-store
+Project-URL: Issues, https://github.com/tvanreenen/numpy-vector-store/issues
+Author: Tim VanReenen
+License: MIT
+License-File: LICENSE
+Keywords: embeddings,numpy,search,store,vector
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Python: >=3.10
+Requires-Dist: numpy>=1.20.0
+Description-Content-Type: text/markdown
+
+# NumPy Vector Store
+
+A fast, lightweight, zero-setup in-memory vector store powered by NumPy.
+
+- **Tiny local vector search** for projects that do not need a vector database
+- **Fast exact vector search** using vectorized NumPy operations
+- **Simple typed API** returning `VectorHit(index, value, metadata)`
+- **Composable filtering** by passing prefiltered row indexes with `within_rows`
+- **Portable persistence** as trusted local `.npz` files with `vectors` + `metadata`
+- **No framework opinions**: bring your own embeddings, chunking, async, and metadata model
+
+## Why?
+
+This library is purpose-built for small to medium-scale vector search tasks and
+offers a simple alternative to heavyweight vector databases when you do not need
+network services, indexing infrastructure, ingestion pipelines, or domain-specific
+metadata filtering.
+
+## When/Where?
+
+Below are benchmark results for cosine similarity search to help you assess its
+suitability for your use case.
+
+| Embedding Type | Dimensions | ~5ms | ~25ms | ~100ms | ~500ms |
+|----------------|------------|------|--------|---------|---------|
+| **Sentence Transformers** | 384 | 1K vectors<br/>1.5MB | 10K vectors<br/>15MB | 100K vectors<br/>147MB | 500K vectors<br/>732MB |
+| **OpenAI Small** | 1536 | 500 vectors<br/>3MB | 5K vectors<br/>29MB | 25K vectors<br/>147MB | 100K vectors<br/>586MB |
+| **OpenAI Large** | 3072 | 200 vectors<br/>2MB | 2.5K vectors<br/>29MB | 5K vectors<br/>59MB | 25K vectors<br/>293MB |
+
+*Benchmarks performed on Apple M2 hardware.*
+
+## Installation
+
+```bash
+uv add numpy-vector-store
+```
+
+## Quick Start
+
+```python
+import numpy as np
+from numpy_vector_store import VectorStore
+
+store = VectorStore[dict[str, str]](dimensions=3)
+
+store.add(
+    vectors=np.array([
+        [1.0, 0.0, 0.0],
+        [0.0, 1.0, 0.0],
+        [0.0, 0.0, 1.0],
+    ]),
+    metadata=[
+        {"title": "x-axis"},
+        {"title": "y-axis"},
+        {"title": "z-axis"},
+    ],
+)
+
+hits = store.cosine_search(
+    query=np.array([0.9, 0.1, 0.0]),
+    top_k=2,
+)
+
+for hit in hits:
+    print(f"{hit.metadata['title']}: {hit.value:.3f}")
+```
+
+`metadata` is an opaque row payload returned with hits. It can be a dict,
+dataclass, string, integer row ID, or any other Python object that fits your
+application.
+
+## Normalization
+
+`VectorStore` defaults to `normalize=True`, which scales each stored vector to
+length `1`. Normalization preserves vector direction while discarding magnitude:
+
+```python
+[3.0, 4.0] -> [0.6, 0.8]
+```
+
+This is the default because it makes cosine similarity fast and direction-only,
+which is the common case for semantic embeddings. Use `normalize=False` when
+vector length matters, such as when magnitude encodes strength, confidence,
+counts, scale, or raw geometry.
+
+Zero vectors are rejected in both modes because cosine similarity is undefined
+for zero-norm vectors.
+
+| Method | `normalize=True` default | `normalize=False` |
+|---|---|---|
+| `cosine_search` | True cosine similarity over stored unit vectors; fastest/default path for embeddings | True cosine similarity over raw vectors; computes vector norms during search |
+| `dot_search` | Dot product of unit vectors, effectively equivalent to cosine similarity | True dot product over original vectors; use when magnitude should affect ranking |
+| `euclidean_search` | Distance between normalized directions; useful only when direction-normalized distance is intended | True Euclidean distance over original vectors; use for geometric/feature-space nearest neighbors |
+| `get` | Returns normalized vectors | Returns original vectors |
+| `save` | Saves normalized vectors | Saves raw vectors |
+| `load` | Loads and normalizes vectors | Loads vectors exactly as stored |
+
+## Search Methods
+
+Use `cosine_search` for semantic embeddings and direction-only similarity:
+
+```python
+hits = store.cosine_search(query, top_k=10, min_value=0.75)
+```
+
+Use `dot_search` with `normalize=False` when larger-magnitude vectors should
+rank higher:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=3, normalize=False)
+store.add(vectors, metadata)
+hits = store.dot_search(query, top_k=10, min_value=0.0)
+```
+
+Use `euclidean_search` with `normalize=False` for raw coordinate or feature-space
+nearest-neighbor search:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=3, normalize=False)
+store.add(vectors, metadata)
+hits = store.euclidean_search(query, top_k=10, max_value=1.5)
+```
+
+## Prefiltering
+
+The store does not implement a metadata query language. To filter by metadata,
+produce row indexes first, then pass them with `within_rows`.
+
+```python
+rows = [
+    i
+    for i, metadata in enumerate(store.metadata)
+    if metadata["title"].startswith("x")
+]
+
+hits = store.cosine_search(query, top_k=10, within_rows=rows)
+```
+
+For structured NumPy metadata, use NumPy to produce the row indexes:
+
+```python
+metadata_table = np.array(
+    [
+        ("intro", "A", 2024),
+        ("setup", "A", 2023),
+        ("guide", "B", 2024),
+    ],
+    dtype=[("title", "U20"), ("product", "U10"), ("year", "i4")],
+)
+
+store = VectorStore[int](dimensions=3)
+store.add(vectors, metadata=np.arange(len(metadata_table)))
+
+mask = (metadata_table["product"] == "A") & (metadata_table["year"] >= 2024)
+rows = np.flatnonzero(mask)
+
+hits = store.cosine_search(query, within_rows=rows)
+
+for hit in hits:
+    row = metadata_table[hit.metadata]
+    print(row["title"], hit.value)
+```
+
+## Persistence
+
+Pass a `file_path` and call `save()` / `load()` explicitly:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz")
+store.add(embeddings, metadata)
+store.save()
+
+loaded = VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz")
+loaded.load()
+```
+
+If you save with `normalize=False`, load with `normalize=False` too:
+
+```python
+store = VectorStore[dict[str, str]](
+    dimensions=1536,
+    file_path="raw-vectors.npz",
+    normalize=False,
+)
+store.add(raw_vectors, metadata)
+store.save()
+
+loaded = VectorStore[dict[str, str]](
+    dimensions=1536,
+    file_path="raw-vectors.npz",
+    normalize=False,
+)
+loaded.load()
+```
+
+Context manager usage auto-saves on exit:
+
+```python
+with VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz") as store:
+    store.add(embeddings, metadata)
+```
+
+Persistence uses a minimal NumPy `.npz` contract with `vectors` and `metadata`
+arrays. The `.npz` file does not encode the `normalize` setting; choose the same
+setting when loading that you used when saving. Loading validates shape,
+dimensions, row counts, and zero-norm vectors. It also uses `allow_pickle=True`
+for flexible Python metadata payloads, so only load files generated by your own
+application or another trusted local process. Loading untrusted `.npz` files is
+not a supported security model.
+
+## Compatibility
+
+This project is still pre-1.0, so occasional breaking changes are expected while
+the API stabilizes. Breaking changes are documented in GitHub release notes.
+Deprecated APIs will keep warning for at least one point release before removal.
+
+## Contributing
+
+```bash
+git clone https://github.com/tvanreenen/numpy-vector-store.git
+cd numpy-vector-store
+uv sync --frozen --group dev
+```
+
+Before submitting a pull request:
+
+1. Run `uv run ruff check`
+2. Run `uv run ruff format --check`
+3. Run `uv run mypy src/`
+4. Run `uv run pytest`
+
+## License
+
+MIT License - see [LICENSE](LICENSE) file for details.

numpy_vector_store-0.3.0/README.md

@@ -0,0 +1,234 @@
+# NumPy Vector Store
+
+A fast, lightweight, zero-setup in-memory vector store powered by NumPy.
+
+- **Tiny local vector search** for projects that do not need a vector database
+- **Fast exact vector search** using vectorized NumPy operations
+- **Simple typed API** returning `VectorHit(index, value, metadata)`
+- **Composable filtering** by passing prefiltered row indexes with `within_rows`
+- **Portable persistence** as trusted local `.npz` files with `vectors` + `metadata`
+- **No framework opinions**: bring your own embeddings, chunking, async, and metadata model
+
+## Why?
+
+This library is purpose-built for small to medium-scale vector search tasks and
+offers a simple alternative to heavyweight vector databases when you do not need
+network services, indexing infrastructure, ingestion pipelines, or domain-specific
+metadata filtering.
+
+## When/Where?
+
+Below are benchmark results for cosine similarity search to help you assess its
+suitability for your use case.
+
+| Embedding Type | Dimensions | ~5ms | ~25ms | ~100ms | ~500ms |
+|----------------|------------|------|--------|---------|---------|
+| **Sentence Transformers** | 384 | 1K vectors<br/>1.5MB | 10K vectors<br/>15MB | 100K vectors<br/>147MB | 500K vectors<br/>732MB |
+| **OpenAI Small** | 1536 | 500 vectors<br/>3MB | 5K vectors<br/>29MB | 25K vectors<br/>147MB | 100K vectors<br/>586MB |
+| **OpenAI Large** | 3072 | 200 vectors<br/>2MB | 2.5K vectors<br/>29MB | 5K vectors<br/>59MB | 25K vectors<br/>293MB |
+
+*Benchmarks performed on Apple M2 hardware.*
+
+## Installation
+
+```bash
+uv add numpy-vector-store
+```
+
+## Quick Start
+
+```python
+import numpy as np
+from numpy_vector_store import VectorStore
+
+store = VectorStore[dict[str, str]](dimensions=3)
+
+store.add(
+    vectors=np.array([
+        [1.0, 0.0, 0.0],
+        [0.0, 1.0, 0.0],
+        [0.0, 0.0, 1.0],
+    ]),
+    metadata=[
+        {"title": "x-axis"},
+        {"title": "y-axis"},
+        {"title": "z-axis"},
+    ],
+)
+
+hits = store.cosine_search(
+    query=np.array([0.9, 0.1, 0.0]),
+    top_k=2,
+)
+
+for hit in hits:
+    print(f"{hit.metadata['title']}: {hit.value:.3f}")
+```
+
+`metadata` is an opaque row payload returned with hits. It can be a dict,
+dataclass, string, integer row ID, or any other Python object that fits your
+application.
+
+## Normalization
+
+`VectorStore` defaults to `normalize=True`, which scales each stored vector to
+length `1`. Normalization preserves vector direction while discarding magnitude:
+
+```python
+[3.0, 4.0] -> [0.6, 0.8]
+```
+
+This is the default because it makes cosine similarity fast and direction-only,
+which is the common case for semantic embeddings. Use `normalize=False` when
+vector length matters, such as when magnitude encodes strength, confidence,
+counts, scale, or raw geometry.
+
+Zero vectors are rejected in both modes because cosine similarity is undefined
+for zero-norm vectors.
+
+| Method | `normalize=True` default | `normalize=False` |
+|---|---|---|
+| `cosine_search` | True cosine similarity over stored unit vectors; fastest/default path for embeddings | True cosine similarity over raw vectors; computes vector norms during search |
+| `dot_search` | Dot product of unit vectors, effectively equivalent to cosine similarity | True dot product over original vectors; use when magnitude should affect ranking |
+| `euclidean_search` | Distance between normalized directions; useful only when direction-normalized distance is intended | True Euclidean distance over original vectors; use for geometric/feature-space nearest neighbors |
+| `get` | Returns normalized vectors | Returns original vectors |
+| `save` | Saves normalized vectors | Saves raw vectors |
+| `load` | Loads and normalizes vectors | Loads vectors exactly as stored |
+
+## Search Methods
+
+Use `cosine_search` for semantic embeddings and direction-only similarity:
+
+```python
+hits = store.cosine_search(query, top_k=10, min_value=0.75)
+```
+
+Use `dot_search` with `normalize=False` when larger-magnitude vectors should
+rank higher:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=3, normalize=False)
+store.add(vectors, metadata)
+hits = store.dot_search(query, top_k=10, min_value=0.0)
+```
+
+Use `euclidean_search` with `normalize=False` for raw coordinate or feature-space
+nearest-neighbor search:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=3, normalize=False)
+store.add(vectors, metadata)
+hits = store.euclidean_search(query, top_k=10, max_value=1.5)
+```
+
+## Prefiltering
+
+The store does not implement a metadata query language. To filter by metadata,
+produce row indexes first, then pass them with `within_rows`.
+
+```python
+rows = [
+    i
+    for i, metadata in enumerate(store.metadata)
+    if metadata["title"].startswith("x")
+]
+
+hits = store.cosine_search(query, top_k=10, within_rows=rows)
+```
+
+For structured NumPy metadata, use NumPy to produce the row indexes:
+
+```python
+metadata_table = np.array(
+    [
+        ("intro", "A", 2024),
+        ("setup", "A", 2023),
+        ("guide", "B", 2024),
+    ],
+    dtype=[("title", "U20"), ("product", "U10"), ("year", "i4")],
+)
+
+store = VectorStore[int](dimensions=3)
+store.add(vectors, metadata=np.arange(len(metadata_table)))
+
+mask = (metadata_table["product"] == "A") & (metadata_table["year"] >= 2024)
+rows = np.flatnonzero(mask)
+
+hits = store.cosine_search(query, within_rows=rows)
+
+for hit in hits:
+    row = metadata_table[hit.metadata]
+    print(row["title"], hit.value)
+```
+
+## Persistence
+
+Pass a `file_path` and call `save()` / `load()` explicitly:
+
+```python
+store = VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz")
+store.add(embeddings, metadata)
+store.save()
+
+loaded = VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz")
+loaded.load()
+```
+
+If you save with `normalize=False`, load with `normalize=False` too:
+
+```python
+store = VectorStore[dict[str, str]](
+    dimensions=1536,
+    file_path="raw-vectors.npz",
+    normalize=False,
+)
+store.add(raw_vectors, metadata)
+store.save()
+
+loaded = VectorStore[dict[str, str]](
+    dimensions=1536,
+    file_path="raw-vectors.npz",
+    normalize=False,
+)
+loaded.load()
+```
+
+Context manager usage auto-saves on exit:
+
+```python
+with VectorStore[dict[str, str]](dimensions=1536, file_path="vectors.npz") as store:
+    store.add(embeddings, metadata)
+```
+
+Persistence uses a minimal NumPy `.npz` contract with `vectors` and `metadata`
+arrays. The `.npz` file does not encode the `normalize` setting; choose the same
+setting when loading that you used when saving. Loading validates shape,
+dimensions, row counts, and zero-norm vectors. It also uses `allow_pickle=True`
+for flexible Python metadata payloads, so only load files generated by your own
+application or another trusted local process. Loading untrusted `.npz` files is
+not a supported security model.
+
+## Compatibility
+
+This project is still pre-1.0, so occasional breaking changes are expected while
+the API stabilizes. Breaking changes are documented in GitHub release notes.
+Deprecated APIs will keep warning for at least one point release before removal.
+
+## Contributing
+
+```bash
+git clone https://github.com/tvanreenen/numpy-vector-store.git
+cd numpy-vector-store
+uv sync --frozen --group dev
+```
+
+Before submitting a pull request:
+
+1. Run `uv run ruff check`
+2. Run `uv run ruff format --check`
+3. Run `uv run mypy src/`
+4. Run `uv run pytest`
+
+## License
+
+MIT License - see [LICENSE](LICENSE) file for details.

{numpy_vector_store-0.1.0 → numpy_vector_store-0.3.0}/pyproject.toml

@@ -7,7 +7,7 @@ name = "numpy-vector-store"
 dynamic = ["version"]
 description = "A fast, lightweight, and zero-setup in-memory vector store powered by NumPy"
 readme = "README.md"
-requires-python = ">=3.9"
+requires-python = ">=3.10"
 license = {text = "MIT"}
 authors = [
     {name = "Tim VanReenen"},
@@ -18,10 +18,11 @@ classifiers = [
     "Intended Audience :: Developers",
     "License :: OSI Approved :: MIT License",
     "Programming Language :: Python :: 3",
-    "Programming Language :: Python :: 3.9",
     "Programming Language :: Python :: 3.10",
     "Programming Language :: Python :: 3.11",
     "Programming Language :: Python :: 3.12",
+    "Programming Language :: Python :: 3.13",
+    "Programming Language :: Python :: 3.14",
 ]
 dependencies = [
     "numpy>=1.20.0",
@@ -41,7 +42,7 @@ path = "src/numpy_vector_store/__init__.py"
 
 [tool.ruff]
 line-length = 88
-target-version = "py39"
+target-version = "py310"
 
 [tool.ruff.lint]
 select = [
@@ -62,7 +63,7 @@ ignore = [
 known-first-party = ["numpy_vector_store"]
 
 [tool.mypy]
-python_version = "3.9"
+python_version = "3.10"
 warn_return_any = true
 warn_unused_configs = true
 disallow_untyped_defs = true

{numpy_vector_store-0.1.0 → numpy_vector_store-0.3.0}/src/numpy_vector_store/__init__.py

@@ -5,9 +5,9 @@ This package provides a lightweight vector store implementation for storing
 and searching vector embeddings using NumPy arrays.
 """
 
-__version__ = "0.1.0"
+__version__ = "0.3.0"
 __author__ = "Tim VanReenen"
 
-from .vector_store import VectorStore
+from .vector_store import VectorHit, VectorStore
 
-__all__ = ["VectorStore"]
+__all__ = ["VectorHit", "VectorStore"]