baked-vector 0.1.0__tar.gz

baked_vector-0.1.0/PKG-INFO

@@ -0,0 +1,225 @@
+Metadata-Version: 2.4
+Name: baked-vector
+Version: 0.1.0
+Summary: Zero-dependency vector store for small-scale embedding search.
+License: MIT
+Project-URL: Repository, https://github.com/sunnytuladhar/baked-vector
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+Provides-Extra: st
+Requires-Dist: sentence-transformers>=3.0; extra == "st"
+Provides-Extra: openai
+Requires-Dist: openai>=1.0; extra == "openai"
+Provides-Extra: bedrock
+Requires-Dist: boto3>=1.34; extra == "bedrock"
+Provides-Extra: google
+Requires-Dist: google-generativeai>=0.8; extra == "google"
+Provides-Extra: anthropic
+Requires-Dist: anthropic>=0.40; extra == "anthropic"
+
+# baked-vector
+
+Zero-dependency vector store for small-scale embedding search. No numpy, no server, no setup — just a binary file and a JSON file baked into your project.
+
+Designed for under 10,000 vectors. Bring your own embeddings.
+
+## Install
+
+```bash
+pip install baked-vector
+# or
+uv add baked-vector
+```
+
+## Usage
+
+```python
+import baked
+
+store = baked.VectorStore("mydata")
+
+# add vectors with arbitrary metadata
+store.add(embedding, {"source": "doc1.pdf", "page": 3, "text": "..."})
+store.save()
+
+# reload from disk automatically
+store = baked.VectorStore("mydata")
+
+# search
+results = store.search(query_embedding, top_k=5)
+# [(metadata_dict, score), ...]
+
+# filter by metadata
+results = store.search(query_embedding, top_k=5, filter={"source": "doc1.pdf"})
+
+# range search — all results with score >= threshold
+results = store.search(query_embedding, radius=0.85)
+
+# batch search
+results = store.search_many([vec1, vec2], top_k=5)
+```
+
+## API
+
+### `VectorStore(path)`
+
+Loads from `<path>.vecs` and `<path>.json` if they exist, otherwise starts empty.
+
+| Method | Description |
+|--------|-------------|
+| `add(embedding, metadata)` | Add a vector. Returns its id. |
+| `remove(id)` | Soft-delete. Takes effect on next `save()`. |
+| `save()` | Flush to disk, compacting deleted entries. |
+| `search(query, top_k, radius, filter)` | Find similar vectors. |
+| `search_many(queries, top_k, radius, filter)` | Batch search. |
+| `len(store)` | Number of active vectors. |
+
+### Search parameters
+
+- `top_k` — return top N results (default: 5)
+- `radius` — return all results with score ≥ threshold (overrides `top_k`)
+- `filter` — dict of key-value pairs that must match the metadata exactly
+
+Scores are cosine similarity in range [-1, 1]. Higher = more similar.
+
+## Storage format
+
+Two files:
+
+- `<path>.vecs` — binary packed floats. 8-byte header (dim, count), then N × dim × 4 bytes.
+- `<path>.json` — JSON array of metadata dicts, one per vector.
+
+Vectors are L2-normalized at insert time. IDs are positional and shift after `save()` following a `remove()`.
+
+## Performance
+
+Pure Python, brute-force scan. Rough query times:
+
+| Vectors | 384-dim | 768-dim | 1536-dim |
+|---------|---------|---------|----------|
+| 1k | <0.1s | <0.1s | ~0.1s |
+| 5k | ~0.1s | ~0.3s | ~0.5s |
+| 10k | ~0.3s | ~0.6s | ~1–2s |
+
+## EmbeddingStore
+
+`EmbeddingStore` wraps `VectorStore` and handles embedding automatically. Install an extra for your provider, pass the adapter, and work with text directly.
+
+```python
+import baked
+
+embed = baked.openai_embedder()  # see adapters below
+store = baked.EmbeddingStore("mydata", embedder=embed)
+
+store.add("Wealth is not money...", {"title": "How to Make Wealth"})
+store.save()
+
+results = store.search("how startups create value", top_k=3)
+# [({"text": "...", "title": "..."}, score), ...]
+```
+
+`EmbeddingStore` exposes the same `add`, `remove`, `save`, `search`, and `len` interface as `VectorStore`, but takes `str` instead of `list[float]`. The original text is stored in metadata under `"text"` automatically.
+
+### Adapters
+
+| Provider | Function | Install extra |
+|----------|----------|---------------|
+| OpenAI | `openai_embedder(model, api_key)` | `baked-vector[openai]` |
+| Azure OpenAI | `azure_openai_embedder(deployment, endpoint, api_key, api_version)` | `baked-vector[openai]` |
+| AWS Bedrock | `bedrock_embedder(model_id, region_name)` | `baked-vector[bedrock]` |
+| Google | `google_embedder(model, api_key)` | `baked-vector[google]` |
+| Local (sentence-transformers) | `sentence_transformer_embedder(model_name)` | `baked-vector[st]` |
+
+```bash
+pip install baked-vector[openai]
+pip install baked-vector[bedrock]
+pip install baked-vector[google]
+pip install baked-vector[st]
+```
+
+Defaults: OpenAI → `text-embedding-3-small`, Bedrock → `amazon.titan-embed-text-v2:0`, Google → `models/text-embedding-004`, local → `all-MiniLM-L6-v2`.
+
+You can also pass any `(str) -> list[float]` callable as `embedder` — no adapter required.
+
+---
+
+## Providers (manual)
+
+baked-vector stores and searches vectors — bring embeddings from any provider.
+
+### OpenAI
+
+```python
+from openai import OpenAI
+
+client = OpenAI()
+
+def embed(text: str) -> list[float]:
+    resp = client.embeddings.create(model="text-embedding-3-small", input=text)
+    return resp.data[0].embedding
+
+store.add(embed("hello world"), {"text": "hello world"})
+results = store.search(embed("hi there"), top_k=5)
+```
+
+### AWS Bedrock (Titan)
+
+```python
+import json
+import boto3
+
+bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
+
+def embed(text: str) -> list[float]:
+    resp = bedrock.invoke_model(
+        modelId="amazon.titan-embed-text-v2:0",
+        body=json.dumps({"inputText": text, "dimensions": 512}),
+    )
+    return json.loads(resp["body"].read())["embedding"]
+
+store.add(embed("hello world"), {"text": "hello world"})
+results = store.search(embed("hi there"), top_k=5)
+```
+
+### Azure OpenAI
+
+```python
+from openai import AzureOpenAI
+
+client = AzureOpenAI(
+    api_key="YOUR_KEY",
+    api_version="2024-02-01",
+    azure_endpoint="https://YOUR_RESOURCE.openai.azure.com",
+)
+
+def embed(text: str) -> list[float]:
+    resp = client.embeddings.create(model="YOUR_DEPLOYMENT_NAME", input=text)
+    return resp.data[0].embedding
+
+store.add(embed("hello world"), {"text": "hello world"})
+results = store.search(embed("hi there"), top_k=5)
+```
+
+### Cohere
+
+```python
+import cohere
+
+co = cohere.Client("YOUR_API_KEY")
+
+def embed(text: str) -> list[float]:
+    resp = co.embed(texts=[text], model="embed-english-v3.0", input_type="search_document")
+    return resp.embeddings[0]
+
+store.add(embed("hello world"), {"text": "hello world"})
+
+# use input_type="search_query" for query embeddings
+results = store.search(
+    co.embed(texts=["hi there"], model="embed-english-v3.0", input_type="search_query").embeddings[0],
+    top_k=5,
+)
+```
+
+## License
+
+MIT

baked_vector-0.1.0/README.md

@@ -0,0 +1,206 @@
+# baked-vector
+
+Zero-dependency vector store for small-scale embedding search. No numpy, no server, no setup — just a binary file and a JSON file baked into your project.
+
+Designed for under 10,000 vectors. Bring your own embeddings.
+
+## Install
+
+```bash
+pip install baked-vector
+# or
+uv add baked-vector
+```
+
+## Usage
+
+```python
+import baked
+
+store = baked.VectorStore("mydata")
+
+# add vectors with arbitrary metadata
+store.add(embedding, {"source": "doc1.pdf", "page": 3, "text": "..."})
+store.save()
+
+# reload from disk automatically
+store = baked.VectorStore("mydata")
+
+# search
+results = store.search(query_embedding, top_k=5)
+# [(metadata_dict, score), ...]
+
+# filter by metadata
+results = store.search(query_embedding, top_k=5, filter={"source": "doc1.pdf"})
+
+# range search — all results with score >= threshold
+results = store.search(query_embedding, radius=0.85)
+
+# batch search
+results = store.search_many([vec1, vec2], top_k=5)
+```
+
+## API
+
+### `VectorStore(path)`
+
+Loads from `<path>.vecs` and `<path>.json` if they exist, otherwise starts empty.
+
+| Method | Description |
+|--------|-------------|
+| `add(embedding, metadata)` | Add a vector. Returns its id. |
+| `remove(id)` | Soft-delete. Takes effect on next `save()`. |
+| `save()` | Flush to disk, compacting deleted entries. |
+| `search(query, top_k, radius, filter)` | Find similar vectors. |
+| `search_many(queries, top_k, radius, filter)` | Batch search. |
+| `len(store)` | Number of active vectors. |
+
+### Search parameters
+
+- `top_k` — return top N results (default: 5)
+- `radius` — return all results with score ≥ threshold (overrides `top_k`)
+- `filter` — dict of key-value pairs that must match the metadata exactly
+
+Scores are cosine similarity in range [-1, 1]. Higher = more similar.
+
+## Storage format
+
+Two files:
+
+- `<path>.vecs` — binary packed floats. 8-byte header (dim, count), then N × dim × 4 bytes.
+- `<path>.json` — JSON array of metadata dicts, one per vector.
+
+Vectors are L2-normalized at insert time. IDs are positional and shift after `save()` following a `remove()`.
+
+## Performance
+
+Pure Python, brute-force scan. Rough query times:
+
+| Vectors | 384-dim | 768-dim | 1536-dim |
+|---------|---------|---------|----------|
+| 1k | <0.1s | <0.1s | ~0.1s |
+| 5k | ~0.1s | ~0.3s | ~0.5s |
+| 10k | ~0.3s | ~0.6s | ~1–2s |
+
+## EmbeddingStore
+
+`EmbeddingStore` wraps `VectorStore` and handles embedding automatically. Install an extra for your provider, pass the adapter, and work with text directly.
+
+```python
+import baked
+
+embed = baked.openai_embedder()  # see adapters below
+store = baked.EmbeddingStore("mydata", embedder=embed)
+
+store.add("Wealth is not money...", {"title": "How to Make Wealth"})
+store.save()
+
+results = store.search("how startups create value", top_k=3)
+# [({"text": "...", "title": "..."}, score), ...]
+```
+
+`EmbeddingStore` exposes the same `add`, `remove`, `save`, `search`, and `len` interface as `VectorStore`, but takes `str` instead of `list[float]`. The original text is stored in metadata under `"text"` automatically.
+
+### Adapters
+
+| Provider | Function | Install extra |
+|----------|----------|---------------|
+| OpenAI | `openai_embedder(model, api_key)` | `baked-vector[openai]` |
+| Azure OpenAI | `azure_openai_embedder(deployment, endpoint, api_key, api_version)` | `baked-vector[openai]` |
+| AWS Bedrock | `bedrock_embedder(model_id, region_name)` | `baked-vector[bedrock]` |
+| Google | `google_embedder(model, api_key)` | `baked-vector[google]` |
+| Local (sentence-transformers) | `sentence_transformer_embedder(model_name)` | `baked-vector[st]` |
+
+```bash
+pip install baked-vector[openai]
+pip install baked-vector[bedrock]
+pip install baked-vector[google]
+pip install baked-vector[st]
+```
+
+Defaults: OpenAI → `text-embedding-3-small`, Bedrock → `amazon.titan-embed-text-v2:0`, Google → `models/text-embedding-004`, local → `all-MiniLM-L6-v2`.
+
+You can also pass any `(str) -> list[float]` callable as `embedder` — no adapter required.
+
+---
+
+## Providers (manual)
+
+baked-vector stores and searches vectors — bring embeddings from any provider.
+
+### OpenAI
+
+```python
+from openai import OpenAI
+
+client = OpenAI()
+
+def embed(text: str) -> list[float]:
+    resp = client.embeddings.create(model="text-embedding-3-small", input=text)
+    return resp.data[0].embedding
+
+store.add(embed("hello world"), {"text": "hello world"})
+results = store.search(embed("hi there"), top_k=5)
+```
+
+### AWS Bedrock (Titan)
+
+```python
+import json
+import boto3
+
+bedrock = boto3.client("bedrock-runtime", region_name="us-east-1")
+
+def embed(text: str) -> list[float]:
+    resp = bedrock.invoke_model(
+        modelId="amazon.titan-embed-text-v2:0",
+        body=json.dumps({"inputText": text, "dimensions": 512}),
+    )
+    return json.loads(resp["body"].read())["embedding"]
+
+store.add(embed("hello world"), {"text": "hello world"})
+results = store.search(embed("hi there"), top_k=5)
+```
+
+### Azure OpenAI
+
+```python
+from openai import AzureOpenAI
+
+client = AzureOpenAI(
+    api_key="YOUR_KEY",
+    api_version="2024-02-01",
+    azure_endpoint="https://YOUR_RESOURCE.openai.azure.com",
+)
+
+def embed(text: str) -> list[float]:
+    resp = client.embeddings.create(model="YOUR_DEPLOYMENT_NAME", input=text)
+    return resp.data[0].embedding
+
+store.add(embed("hello world"), {"text": "hello world"})
+results = store.search(embed("hi there"), top_k=5)
+```
+
+### Cohere
+
+```python
+import cohere
+
+co = cohere.Client("YOUR_API_KEY")
+
+def embed(text: str) -> list[float]:
+    resp = co.embed(texts=[text], model="embed-english-v3.0", input_type="search_document")
+    return resp.embeddings[0]
+
+store.add(embed("hello world"), {"text": "hello world"})
+
+# use input_type="search_query" for query embeddings
+results = store.search(
+    co.embed(texts=["hi there"], model="embed-english-v3.0", input_type="search_query").embeddings[0],
+    top_k=5,
+)
+```
+
+## License
+
+MIT

baked_vector-0.1.0/baked/__init__.py

@@ -0,0 +1,25 @@
+from .store import VectorStore
+from .embed import (
+    EmbeddingStore,
+    sentence_transformer_embedder,
+    openai_embedder,
+    azure_openai_embedder,
+    bedrock_embedder,
+    google_embedder,
+)
+from .chunking import chunk_fixed, chunk_sentences, chunk_paragraphs, chunk_recursive
+
+__all__ = [
+    "VectorStore",
+    "EmbeddingStore",
+    "sentence_transformer_embedder",
+    "openai_embedder",
+    "azure_openai_embedder",
+    "bedrock_embedder",
+    "google_embedder",
+    "chunk_fixed",
+    "chunk_sentences",
+    "chunk_paragraphs",
+    "chunk_recursive",
+]
+__version__ = "0.1.0"