beaver-db 0.2.0__tar.gz → 0.4.0__tar.gz

beaver_db-0.4.0/PKG-INFO

@@ -0,0 +1,129 @@
+Metadata-Version: 2.4
+Name: beaver-db
+Version: 0.4.0
+Summary: Asynchronous, embedded, modern DB based on SQLite.
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=2.3.3
+
+# beaver 🦫
+
+A fast, single-file, multi-modal database for Python, built with the standard sqlite3 library.
+
+`beaver` is the **B**ackend for **E**mbedded **A**synchronous **V**ector & **E**vent Retrieval. It's an industrious, all-in-one database designed to manage complex, modern data types without requiring a database server.
+
+## Design Philosophy
+
+`beaver` is built with a minimalistic philosophy for small, local use cases where a full-blown database server would be overkill.
+
+  - **Minimalistic & Zero-Dependency**: Uses only Python's standard libraries (`sqlite3`, `asyncio`) and `numpy`.
+  - **Async-First (When It Matters)**: The pub/sub system is fully asynchronous for high-performance, real-time messaging. Other features like key-value, list, and vector operations are synchronous for ease of use.
+  - **Built for Local Applications**: Perfect for local AI tools, RAG prototypes, chatbots, and desktop utilities that need persistent, structured data without network overhead.
+  - **Fast by Default**: It's built on SQLite, which is famously fast and reliable for local applications.
+
+## Core Features
+
+  - **Asynchronous Pub/Sub**: A fully asynchronous, Redis-like publish-subscribe system for real-time messaging.
+  - **Persistent Key-Value Store**: A simple `set`/`get` interface for storing any JSON-serializable object.
+  - **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
+  - **Vector Storage & Search**: Store vector embeddings and perform simple, brute-force k-nearest neighbor searches, ideal for small-scale RAG.
+  - **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
+
+## Installation
+
+```bash
+pip install beaver-db
+```
+
+## Quickstart & API Guide
+
+### Initialization
+
+All you need to do is import and instantiate the `BeaverDB` class with a file path.
+
+```python
+from beaver import BeaverDB, Document
+
+db = BeaverDB("my_application.db")
+```
+
+### Key-Value Store
+
+Use `set()` and `get()` for simple data storage. The value can be any JSON-encodable object.
+
+```python
+# Set a value
+db.set("app_config", {"theme": "dark", "user_id": 123})
+
+# Get a value
+config = db.get("app_config")
+print(f"Theme: {config['theme']}") # Output: Theme: dark
+```
+
+### List Management
+
+Get a list wrapper with `db.list()` and use Pythonic methods to manage it.
+
+```python
+tasks = db.list("daily_tasks")
+tasks.push("Write the project report")
+tasks.prepend("Plan the day's agenda")
+print(f"The first task is: {tasks[0]}")
+```
+
+### Vector Storage & Search
+
+Store `Document` objects containing vector embeddings and metadata. The search is a linear scan, which is sufficient for small-to-medium collections.
+
+```python
+# Get a handle to a collection
+docs = db.collection("my_documents")
+
+# Create and index a document (ID will be a UUID)
+doc1 = Document(embedding=[0.1, 0.2, 0.7], text="A cat sat on the mat.")
+docs.index(doc1)
+
+# Create and index a document with a specific ID (for upserting)
+doc2 = Document(id="article-42", embedding=[0.9, 0.1, 0.1], text="A dog chased a ball.")
+docs.index(doc2)
+
+# Search for the 2 most similar documents
+query_vector = [0.15, 0.25, 0.65]
+results = docs.search(vector=query_vector, top_k=2)
+
+# Results are a list of (Document, distance) tuples
+top_document, distance = results[0]
+print(f"Closest document: {top_document.text} (distance: {distance:.4f})")
+```
+
+### Asynchronous Pub/Sub
+
+Publish events from one part of your app and listen in another using `asyncio`.
+
+```python
+import asyncio
+
+async def listener():
+    async with db.subscribe("system_events") as sub:
+        async for message in sub:
+            print(f"LISTENER: Received event -> {message['event']}")
+
+async def publisher():
+    await asyncio.sleep(1)
+    await db.publish("system_events", {"event": "user_login", "user": "alice"})
+
+# To run them concurrently:
+# asyncio.run(asyncio.gather(listener(), publisher()))
+```
+
+## Roadmap
+
+`beaver` aims to be a complete, self-contained data toolkit. The following features are planned:
+
+  - **More Efficient Vector Search**: Integrate an approximate nearest neighbor (ANN) index like `scipy.spatial.cKDTree` to improve search speed on larger datasets.
+  - **JSON Document Store with Full-Text Search**: Store flexible JSON documents and get powerful full-text search across all text fields, powered by SQLite's FTS5 extension.
+  - **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks with standard SQL.
+
+## License
+
+This project is licensed under the MIT License.

beaver_db-0.4.0/README.md

@@ -0,0 +1,121 @@
+# beaver 🦫
+
+A fast, single-file, multi-modal database for Python, built with the standard sqlite3 library.
+
+`beaver` is the **B**ackend for **E**mbedded **A**synchronous **V**ector & **E**vent Retrieval. It's an industrious, all-in-one database designed to manage complex, modern data types without requiring a database server.
+
+## Design Philosophy
+
+`beaver` is built with a minimalistic philosophy for small, local use cases where a full-blown database server would be overkill.
+
+  - **Minimalistic & Zero-Dependency**: Uses only Python's standard libraries (`sqlite3`, `asyncio`) and `numpy`.
+  - **Async-First (When It Matters)**: The pub/sub system is fully asynchronous for high-performance, real-time messaging. Other features like key-value, list, and vector operations are synchronous for ease of use.
+  - **Built for Local Applications**: Perfect for local AI tools, RAG prototypes, chatbots, and desktop utilities that need persistent, structured data without network overhead.
+  - **Fast by Default**: It's built on SQLite, which is famously fast and reliable for local applications.
+
+## Core Features
+
+  - **Asynchronous Pub/Sub**: A fully asynchronous, Redis-like publish-subscribe system for real-time messaging.
+  - **Persistent Key-Value Store**: A simple `set`/`get` interface for storing any JSON-serializable object.
+  - **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
+  - **Vector Storage & Search**: Store vector embeddings and perform simple, brute-force k-nearest neighbor searches, ideal for small-scale RAG.
+  - **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
+
+## Installation
+
+```bash
+pip install beaver-db
+```
+
+## Quickstart & API Guide
+
+### Initialization
+
+All you need to do is import and instantiate the `BeaverDB` class with a file path.
+
+```python
+from beaver import BeaverDB, Document
+
+db = BeaverDB("my_application.db")
+```
+
+### Key-Value Store
+
+Use `set()` and `get()` for simple data storage. The value can be any JSON-encodable object.
+
+```python
+# Set a value
+db.set("app_config", {"theme": "dark", "user_id": 123})
+
+# Get a value
+config = db.get("app_config")
+print(f"Theme: {config['theme']}") # Output: Theme: dark
+```
+
+### List Management
+
+Get a list wrapper with `db.list()` and use Pythonic methods to manage it.
+
+```python
+tasks = db.list("daily_tasks")
+tasks.push("Write the project report")
+tasks.prepend("Plan the day's agenda")
+print(f"The first task is: {tasks[0]}")
+```
+
+### Vector Storage & Search
+
+Store `Document` objects containing vector embeddings and metadata. The search is a linear scan, which is sufficient for small-to-medium collections.
+
+```python
+# Get a handle to a collection
+docs = db.collection("my_documents")
+
+# Create and index a document (ID will be a UUID)
+doc1 = Document(embedding=[0.1, 0.2, 0.7], text="A cat sat on the mat.")
+docs.index(doc1)
+
+# Create and index a document with a specific ID (for upserting)
+doc2 = Document(id="article-42", embedding=[0.9, 0.1, 0.1], text="A dog chased a ball.")
+docs.index(doc2)
+
+# Search for the 2 most similar documents
+query_vector = [0.15, 0.25, 0.65]
+results = docs.search(vector=query_vector, top_k=2)
+
+# Results are a list of (Document, distance) tuples
+top_document, distance = results[0]
+print(f"Closest document: {top_document.text} (distance: {distance:.4f})")
+```
+
+### Asynchronous Pub/Sub
+
+Publish events from one part of your app and listen in another using `asyncio`.
+
+```python
+import asyncio
+
+async def listener():
+    async with db.subscribe("system_events") as sub:
+        async for message in sub:
+            print(f"LISTENER: Received event -> {message['event']}")
+
+async def publisher():
+    await asyncio.sleep(1)
+    await db.publish("system_events", {"event": "user_login", "user": "alice"})
+
+# To run them concurrently:
+# asyncio.run(asyncio.gather(listener(), publisher()))
+```
+
+## Roadmap
+
+`beaver` aims to be a complete, self-contained data toolkit. The following features are planned:
+
+  - **More Efficient Vector Search**: Integrate an approximate nearest neighbor (ANN) index like `scipy.spatial.cKDTree` to improve search speed on larger datasets.
+  - **JSON Document Store with Full-Text Search**: Store flexible JSON documents and get powerful full-text search across all text fields, powered by SQLite's FTS5 extension.
+  - **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks with standard SQL.
+
+## License
+
+This project is licensed under the MIT License.

beaver_db-0.4.0/beaver/__init__.py

@@ -0,0 +1 @@
+from .core import BeaverDB, Document

{beaver_db-0.2.0 → beaver_db-0.4.0}/beaver/core.py

@@ -1,4 +1,6 @@
 import asyncio
+import uuid
+import numpy as np
 import json
 import sqlite3
 import time
@@ -26,43 +28,83 @@ class BeaverDB:
         self._create_pubsub_table()
         self._create_kv_table()
         self._create_list_table()
+        self._create_collections_table()
+        self._create_fts_table()  # <-- Nueva llamada
+
+    def _create_fts_table(self):
+        """Creates the virtual FTS table for full text search."""
+        with self._conn:
+            self._conn.execute(
+                """
+                CREATE VIRTUAL TABLE IF NOT EXISTS beaver_fts_index USING fts5(
+                    collection,
+                    item_id,
+                    field_path,
+                    field_content,
+                    tokenize = 'porter'
+                )
+            """
+            )
 
     def _create_pubsub_table(self):
         """Creates the pub/sub log table if it doesn't exist."""
         with self._conn:
-            self._conn.execute("""
+            self._conn.execute(
+                """
                 CREATE TABLE IF NOT EXISTS beaver_pubsub_log (
                     timestamp REAL PRIMARY KEY,
                     channel_name TEXT NOT NULL,
                     message_payload TEXT NOT NULL
                 )
-            """)
-            self._conn.execute("""
+            """
+            )
+            self._conn.execute(
+                """
                 CREATE INDEX IF NOT EXISTS idx_pubsub_channel_timestamp
                 ON beaver_pubsub_log (channel_name, timestamp)
-            """)
+            """
+            )
 
     def _create_kv_table(self):
         """Creates the key-value store table if it doesn't exist."""
         with self._conn:
-            self._conn.execute("""
+            self._conn.execute(
+                """
                 CREATE TABLE IF NOT EXISTS _beaver_kv_store (
                     key TEXT PRIMARY KEY,
                     value TEXT NOT NULL
                 )
-            """)
+            """
+            )
 
     def _create_list_table(self):
         """Creates the lists table if it doesn't exist."""
         with self._conn:
-            self._conn.execute("""
+            self._conn.execute(
+                """
                 CREATE TABLE IF NOT EXISTS beaver_lists (
                     list_name TEXT NOT NULL,
                     item_order REAL NOT NULL,
                     item_value TEXT NOT NULL,
                     PRIMARY KEY (list_name, item_order)
                 )
-            """)
+            """
+            )
+
+    def _create_collections_table(self):
+        """Creates the collections table if it doesn't exist."""
+        with self._conn:
+            self._conn.execute(
+                """
+                CREATE TABLE IF NOT EXISTS beaver_collections (
+                    collection TEXT NOT NULL,
+                    item_id TEXT NOT NULL,
+                    item_vector BLOB,
+                    metadata TEXT,
+                    PRIMARY KEY (collection, item_id)
+                )
+            """
+            )
 
     def close(self):
         """Closes the database connection."""
@@ -94,7 +136,7 @@ class BeaverDB:
         with self._conn:
             self._conn.execute(
                 "INSERT OR REPLACE INTO _beaver_kv_store (key, value) VALUES (?, ?)",
-                (key, json_value)
+                (key, json_value),
             )
 
     def get(self, key: str) -> Any:
@@ -120,7 +162,7 @@ class BeaverDB:
         cursor.close()
 
         if result:
-            return json.loads(result['value'])
+            return json.loads(result["value"])
         return None
 
     # --- List Methods ---
@@ -139,6 +181,10 @@ class BeaverDB:
             raise TypeError("List name must be a non-empty string.")
         return ListWrapper(name, self._conn)
 
+    def collection(self, name: str) -> "CollectionWrapper":
+        """Returns a wrapper for interacting with a vector collection."""
+        return CollectionWrapper(name, self._conn)
+
     # --- Asynchronous Pub/Sub Methods ---
 
     async def publish(self, channel_name: str, payload: Any):
@@ -153,16 +199,14 @@ class BeaverDB:
         except TypeError as e:
             raise TypeError("Message payload must be JSON-serializable.") from e
 
-        await asyncio.to_thread(
-            self._write_publish_to_db, channel_name, json_payload
-        )
+        await asyncio.to_thread(self._write_publish_to_db, channel_name, json_payload)
 
     def _write_publish_to_db(self, channel_name, json_payload):
         """The synchronous part of the publish operation."""
         with self._conn:
             self._conn.execute(
                 "INSERT INTO beaver_pubsub_log (timestamp, channel_name, message_payload) VALUES (?, ?, ?)",
-                (time.time(), channel_name, json_payload)
+                (time.time(), channel_name, json_payload),
             )
 
     def subscribe(self, channel_name: str) -> "Subscriber":
@@ -182,7 +226,9 @@ class ListWrapper:
     def __len__(self) -> int:
         """Returns the number of items in the list (e.g., `len(my_list)`)."""
         cursor = self._conn.cursor()
-        cursor.execute("SELECT COUNT(*) FROM beaver_lists WHERE list_name = ?", (self._name,))
+        cursor.execute(
+            "SELECT COUNT(*) FROM beaver_lists WHERE list_name = ?", (self._name,)
+        )
         count = cursor.fetchone()[0]
         cursor.close()
         return count
@@ -203,9 +249,9 @@ class ListWrapper:
             cursor = self._conn.cursor()
             cursor.execute(
                 "SELECT item_value FROM beaver_lists WHERE list_name = ? ORDER BY item_order ASC LIMIT ? OFFSET ?",
-                (self._name, limit, start)
+                (self._name, limit, start),
             )
-            results = [json.loads(row['item_value']) for row in cursor.fetchall()]
+            results = [json.loads(row["item_value"]) for row in cursor.fetchall()]
             cursor.close()
             return results
 
@@ -219,11 +265,11 @@ class ListWrapper:
             cursor = self._conn.cursor()
             cursor.execute(
                 "SELECT item_value FROM beaver_lists WHERE list_name = ? ORDER BY item_order ASC LIMIT 1 OFFSET ?",
-                (self._name, offset)
+                (self._name, offset),
             )
             result = cursor.fetchone()
             cursor.close()
-            return json.loads(result['item_value']) if result else None
+            return json.loads(result["item_value"]) if result else None
 
         else:
             raise TypeError("List indices must be integers or slices.")
@@ -233,7 +279,7 @@ class ListWrapper:
         cursor = self._conn.cursor()
         cursor.execute(
             "SELECT item_order FROM beaver_lists WHERE list_name = ? ORDER BY item_order ASC LIMIT 1 OFFSET ?",
-            (self._name, index)
+            (self._name, index),
         )
         result = cursor.fetchone()
         cursor.close()
@@ -247,26 +293,32 @@ class ListWrapper:
         """Pushes an item to the end of the list."""
         with self._conn:
             cursor = self._conn.cursor()
-            cursor.execute("SELECT MAX(item_order) FROM beaver_lists WHERE list_name = ?", (self._name,))
+            cursor.execute(
+                "SELECT MAX(item_order) FROM beaver_lists WHERE list_name = ?",
+                (self._name,),
+            )
             max_order = cursor.fetchone()[0] or 0.0
             new_order = max_order + 1.0
 
             cursor.execute(
                 "INSERT INTO beaver_lists (list_name, item_order, item_value) VALUES (?, ?, ?)",
-                (self._name, new_order, json.dumps(value))
+                (self._name, new_order, json.dumps(value)),
             )
 
     def prepend(self, value: Any):
         """Prepends an item to the beginning of the list."""
         with self._conn:
             cursor = self._conn.cursor()
-            cursor.execute("SELECT MIN(item_order) FROM beaver_lists WHERE list_name = ?", (self._name,))
+            cursor.execute(
+                "SELECT MIN(item_order) FROM beaver_lists WHERE list_name = ?",
+                (self._name,),
+            )
             min_order = cursor.fetchone()[0] or 0.0
             new_order = min_order - 1.0
 
             cursor.execute(
                 "INSERT INTO beaver_lists (list_name, item_order, item_value) VALUES (?, ?, ?)",
-                (self._name, new_order, json.dumps(value))
+                (self._name, new_order, json.dumps(value)),
             )
 
     def insert(self, index: int, value: Any):
@@ -288,7 +340,7 @@ class ListWrapper:
         with self._conn:
             self._conn.execute(
                 "INSERT INTO beaver_lists (list_name, item_order, item_value) VALUES (?, ?, ?)",
-                (self._name, new_order, json.dumps(value))
+                (self._name, new_order, json.dumps(value)),
             )
 
     def pop(self) -> Any:
@@ -297,14 +349,16 @@ class ListWrapper:
             cursor = self._conn.cursor()
             cursor.execute(
                 "SELECT rowid, item_value FROM beaver_lists WHERE list_name = ? ORDER BY item_order DESC LIMIT 1",
-                (self._name,)
+                (self._name,),
             )
             result = cursor.fetchone()
             if not result:
                 return None
 
             rowid_to_delete, value_to_return = result
-            cursor.execute("DELETE FROM beaver_lists WHERE rowid = ?", (rowid_to_delete,))
+            cursor.execute(
+                "DELETE FROM beaver_lists WHERE rowid = ?", (rowid_to_delete,)
+            )
             return json.loads(value_to_return)
 
     def deque(self) -> Any:
@@ -313,14 +367,16 @@ class ListWrapper:
             cursor = self._conn.cursor()
             cursor.execute(
                 "SELECT rowid, item_value FROM beaver_lists WHERE list_name = ? ORDER BY item_order ASC LIMIT 1",
-                (self._name,)
+                (self._name,),
             )
             result = cursor.fetchone()
             if not result:
                 return None
 
             rowid_to_delete, value_to_return = result
-            cursor.execute("DELETE FROM beaver_lists WHERE rowid = ?", (rowid_to_delete,))
+            cursor.execute(
+                "DELETE FROM beaver_lists WHERE rowid = ?", (rowid_to_delete,)
+            )
             return json.loads(value_to_return)
 
 
@@ -330,7 +386,9 @@ class Subscriber(AsyncIterator):
     Designed to be used with 'async with'.
     """
 
-    def __init__(self, conn: sqlite3.Connection, channel_name: str, poll_interval: float = 0.1):
+    def __init__(
+        self, conn: sqlite3.Connection, channel_name: str, poll_interval: float = 0.1
+    ):
         self._conn = conn
         self._channel = channel_name
         self._poll_interval = poll_interval
@@ -342,9 +400,7 @@ class Subscriber(AsyncIterator):
         """Background task that polls the database for new messages."""
         while True:
             try:
-                new_messages = await asyncio.to_thread(
-                    self._fetch_new_messages_from_db
-                )
+                new_messages = await asyncio.to_thread(self._fetch_new_messages_from_db)
                 if new_messages:
                     for msg in new_messages:
                         payload = json.loads(msg["message_payload"])
@@ -362,7 +418,7 @@ class Subscriber(AsyncIterator):
         cursor = self._conn.cursor()
         cursor.execute(
             "SELECT timestamp, message_payload FROM beaver_pubsub_log WHERE channel_name = ? AND timestamp > ? ORDER BY timestamp ASC",
-            (self._channel, self._last_seen_timestamp)
+            (self._channel, self._last_seen_timestamp),
         )
         results = cursor.fetchall()
         cursor.close()
@@ -385,3 +441,198 @@ class Subscriber(AsyncIterator):
     async def __anext__(self) -> Any:
         """Allows 'async for' to pull messages from the internal queue."""
         return await self._queue.get()
+
+
+class Document:
+    """A data class for a vector and its metadata, with a unique ID."""
+
+    def __init__(
+        self, embedding: list[float] | None = None, id: str | None = None, **metadata
+    ):
+        self.id = id or str(uuid.uuid4())
+
+        if embedding is None:
+            self.embedding = None
+        else:
+            if not isinstance(embedding, list) or not all(
+                isinstance(x, (int, float)) for x in embedding
+            ):
+                raise TypeError("Embedding must be a list of numbers.")
+
+            self.embedding = np.array(embedding, dtype=np.float32)
+
+        for key, value in metadata.items():
+            setattr(self, key, value)
+
+    def to_dict(self) -> dict[str, Any]:
+        """Serializes metadata to a dictionary."""
+        metadata = self.__dict__.copy()
+        # Exclude internal attributes from the metadata payload
+        metadata.pop("embedding", None)
+        metadata.pop("id", None)
+        return metadata
+
+    def __repr__(self):
+        metadata_str = ", ".join(f"{k}={v!r}" for k, v in self.to_dict().items())
+        return f"Document(id='{self.id}', {metadata_str})"
+
+
+class CollectionWrapper:
+    """A wrapper for vector collection operations with upsert logic."""
+
+    def __init__(self, name: str, conn: sqlite3.Connection):
+        self._name = name
+        self._conn = conn
+
+    # Dentro de la clase CollectionWrapper en beaver/core.py
+
+    def _flatten_metadata(self, metadata: dict, prefix: str = "") -> dict[str, str]:
+        """
+        Aplana un diccionario anidado y filtra solo los valores de tipo string.
+        Ejemplo: {'a': {'b': 'c'}} -> {'a__b': 'c'}
+        """
+        flat_dict = {}
+        for key, value in metadata.items():
+            new_key = f"{prefix}__{key}" if prefix else key
+            if isinstance(value, dict):
+                flat_dict.update(self._flatten_metadata(value, new_key))
+            elif isinstance(value, str):
+                flat_dict[new_key] = value
+        return flat_dict
+
+    def index(self, document: Document, *, fts: bool = True):
+        """
+        Indexa un Document, realizando un upsert y actualizando el índice FTS.
+        """
+        with self._conn:
+            if fts:
+                self._conn.execute(
+                    "DELETE FROM beaver_fts_index WHERE collection = ? AND item_id = ?",
+                    (self._name, document.id),
+                )
+
+                string_fields = self._flatten_metadata(document.to_dict())
+
+                if string_fields:
+                    fts_data = [
+                        (self._name, document.id, path, content)
+                        for path, content in string_fields.items()
+                    ]
+                    self._conn.executemany(
+                        "INSERT INTO beaver_fts_index (collection, item_id, field_path, field_content) VALUES (?, ?, ?, ?)",
+                        fts_data,
+                    )
+
+            self._conn.execute(
+                "INSERT OR REPLACE INTO beaver_collections (collection, item_id, item_vector, metadata) VALUES (?, ?, ?, ?)",
+                (
+                    self._name,
+                    document.id,
+                    document.embedding.tobytes() if document.embedding is not None else None,
+                    json.dumps(document.to_dict()),
+                ),
+            )
+
+    def search(
+        self, vector: list[float], top_k: int = 10
+    ) -> list[tuple[Document, float]]:
+        """
+        Performs a vector search and returns Document objects.
+        """
+        query_vector = np.array(vector, dtype=np.float32)
+
+        cursor = self._conn.cursor()
+        cursor.execute(
+            "SELECT item_id, item_vector, metadata FROM beaver_collections WHERE collection = ?",
+            (self._name,),
+        )
+
+        all_docs_data = cursor.fetchall()
+        cursor.close()
+
+        if not all_docs_data:
+            return []
+
+        results = []
+        for row in all_docs_data:
+            if row["item_vector"] is None:
+                continue  # Skip documents without embeddings
+
+            doc_id = row["item_id"]
+            embedding = np.frombuffer(row["item_vector"], dtype=np.float32).tolist()
+            metadata = json.loads(row["metadata"])
+
+            distance = np.linalg.norm(embedding - query_vector)
+
+            # Reconstruct the Document object with its original ID
+            doc = Document(id=doc_id, embedding=list(embedding), **metadata)
+            results.append((doc, float(distance)))
+
+        results.sort(key=lambda x: x[1])
+        return results[:top_k]
+
+    def match(
+        self, query: str, on_field: str | None = None, top_k: int = 10
+    ) -> list[tuple[Document, float]]:
+        """
+        Realiza una búsqueda de texto completo en los campos de metadatos indexados.
+
+        Args:
+            query: La expresión de búsqueda (ej. "gato", "perro OR conejo").
+            on_field: Opcional, el campo específico donde buscar (ej. "details__title").
+            top_k: El número máximo de resultados a devolver.
+
+        Returns:
+            Una lista de tuplas (Documento, puntuación_de_relevancia).
+        """
+        cursor = self._conn.cursor()
+
+        sql_query = """
+            SELECT
+                t1.item_id, t1.item_vector, t1.metadata, fts.rank
+            FROM beaver_collections AS t1
+            JOIN (
+                SELECT DISTINCT item_id, rank
+                FROM beaver_fts_index
+                WHERE beaver_fts_index MATCH ?
+                ORDER BY rank
+                LIMIT ?
+            ) AS fts ON t1.item_id = fts.item_id
+            WHERE t1.collection = ?
+            ORDER BY fts.rank
+        """
+
+        params = []
+        field_filter_sql = ""
+
+        if on_field:
+            field_filter_sql = "AND field_path = ?"
+            params.append(on_field)
+        else:
+            # Búsqueda en todos los campos
+            params.append(query)
+
+        sql_query = sql_query.format(field_filter_sql)
+        params.extend([top_k, self._name])
+
+        cursor.execute(sql_query, tuple(params))
+
+        results = []
+        for row in cursor.fetchall():
+            doc_id = row["item_id"]
+
+            if row["item_vector"] is None:
+                embedding = None
+            else:
+                embedding = np.frombuffer(row["item_vector"], dtype=np.float32).tolist()
+
+            metadata = json.loads(row["metadata"])
+            rank = row["rank"]
+
+            doc = Document(id=doc_id, embedding=embedding, **metadata)
+            results.append((doc, rank))
+
+        results.sort(key=lambda x: x[1])
+        cursor.close()
+
+        return results

beaver_db-0.4.0/beaver_db.egg-info/PKG-INFO

@@ -0,0 +1,129 @@
+Metadata-Version: 2.4
+Name: beaver-db
+Version: 0.4.0
+Summary: Asynchronous, embedded, modern DB based on SQLite.
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+Requires-Dist: numpy>=2.3.3
+
+# beaver 🦫
+
+A fast, single-file, multi-modal database for Python, built with the standard sqlite3 library.
+
+`beaver` is the **B**ackend for **E**mbedded **A**synchronous **V**ector & **E**vent Retrieval. It's an industrious, all-in-one database designed to manage complex, modern data types without requiring a database server.
+
+## Design Philosophy
+
+`beaver` is built with a minimalistic philosophy for small, local use cases where a full-blown database server would be overkill.
+
+  - **Minimalistic & Zero-Dependency**: Uses only Python's standard libraries (`sqlite3`, `asyncio`) and `numpy`.
+  - **Async-First (When It Matters)**: The pub/sub system is fully asynchronous for high-performance, real-time messaging. Other features like key-value, list, and vector operations are synchronous for ease of use.
+  - **Built for Local Applications**: Perfect for local AI tools, RAG prototypes, chatbots, and desktop utilities that need persistent, structured data without network overhead.
+  - **Fast by Default**: It's built on SQLite, which is famously fast and reliable for local applications.
+
+## Core Features
+
+  - **Asynchronous Pub/Sub**: A fully asynchronous, Redis-like publish-subscribe system for real-time messaging.
+  - **Persistent Key-Value Store**: A simple `set`/`get` interface for storing any JSON-serializable object.
+  - **Pythonic List Management**: A fluent, Redis-like interface for managing persistent, ordered lists.
+  - **Vector Storage & Search**: Store vector embeddings and perform simple, brute-force k-nearest neighbor searches, ideal for small-scale RAG.
+  - **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
+
+## Installation
+
+```bash
+pip install beaver-db
+```
+
+## Quickstart & API Guide
+
+### Initialization
+
+All you need to do is import and instantiate the `BeaverDB` class with a file path.
+
+```python
+from beaver import BeaverDB, Document
+
+db = BeaverDB("my_application.db")
+```
+
+### Key-Value Store
+
+Use `set()` and `get()` for simple data storage. The value can be any JSON-encodable object.
+
+```python
+# Set a value
+db.set("app_config", {"theme": "dark", "user_id": 123})
+
+# Get a value
+config = db.get("app_config")
+print(f"Theme: {config['theme']}") # Output: Theme: dark
+```
+
+### List Management
+
+Get a list wrapper with `db.list()` and use Pythonic methods to manage it.
+
+```python
+tasks = db.list("daily_tasks")
+tasks.push("Write the project report")
+tasks.prepend("Plan the day's agenda")
+print(f"The first task is: {tasks[0]}")
+```
+
+### Vector Storage & Search
+
+Store `Document` objects containing vector embeddings and metadata. The search is a linear scan, which is sufficient for small-to-medium collections.
+
+```python
+# Get a handle to a collection
+docs = db.collection("my_documents")
+
+# Create and index a document (ID will be a UUID)
+doc1 = Document(embedding=[0.1, 0.2, 0.7], text="A cat sat on the mat.")
+docs.index(doc1)
+
+# Create and index a document with a specific ID (for upserting)
+doc2 = Document(id="article-42", embedding=[0.9, 0.1, 0.1], text="A dog chased a ball.")
+docs.index(doc2)
+
+# Search for the 2 most similar documents
+query_vector = [0.15, 0.25, 0.65]
+results = docs.search(vector=query_vector, top_k=2)
+
+# Results are a list of (Document, distance) tuples
+top_document, distance = results[0]
+print(f"Closest document: {top_document.text} (distance: {distance:.4f})")
+```
+
+### Asynchronous Pub/Sub
+
+Publish events from one part of your app and listen in another using `asyncio`.
+
+```python
+import asyncio
+
+async def listener():
+    async with db.subscribe("system_events") as sub:
+        async for message in sub:
+            print(f"LISTENER: Received event -> {message['event']}")
+
+async def publisher():
+    await asyncio.sleep(1)
+    await db.publish("system_events", {"event": "user_login", "user": "alice"})
+
+# To run them concurrently:
+# asyncio.run(asyncio.gather(listener(), publisher()))
+```
+
+## Roadmap
+
+`beaver` aims to be a complete, self-contained data toolkit. The following features are planned:
+
+  - **More Efficient Vector Search**: Integrate an approximate nearest neighbor (ANN) index like `scipy.spatial.cKDTree` to improve search speed on larger datasets.
+  - **JSON Document Store with Full-Text Search**: Store flexible JSON documents and get powerful full-text search across all text fields, powered by SQLite's FTS5 extension.
+  - **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks with standard SQL.
+
+## License
+
+This project is licensed under the MIT License.

{beaver_db-0.2.0 → beaver_db-0.4.0}/beaver_db.egg-info/SOURCES.txt

@@ -5,4 +5,5 @@ beaver/core.py
 beaver_db.egg-info/PKG-INFO
 beaver_db.egg-info/SOURCES.txt
 beaver_db.egg-info/dependency_links.txt
+beaver_db.egg-info/requires.txt
 beaver_db.egg-info/top_level.txt

beaver_db-0.4.0/beaver_db.egg-info/requires.txt

@@ -0,0 +1 @@
+numpy>=2.3.3

{beaver_db-0.2.0 → beaver_db-0.4.0}/pyproject.toml

@@ -1,10 +1,12 @@
 [project]
 name = "beaver-db"
-version = "0.2.0"
+version = "0.4.0"
 description = "Asynchronous, embedded, modern DB based on SQLite."
 readme = "README.md"
 requires-python = ">=3.13"
-dependencies = []
+dependencies = [
+    "numpy>=2.3.3",
+]
 
 [tool.hatch.build.targets.wheel]
 packages = ["beaver"]

beaver_db-0.2.0/PKG-INFO

@@ -1,109 +0,0 @@
-Metadata-Version: 2.4
-Name: beaver-db
-Version: 0.2.0
-Summary: Asynchronous, embedded, modern DB based on SQLite.
-Requires-Python: >=3.13
-Description-Content-Type: text/markdown
-
-# beaver 🦫
-
-A fast, single-file, multi-modal database for Python, built with the standard sqlite3 library.
-
-`beaver` is the Backend for Embedded Asynchronous Vector & Event Retrieval. It's an industrious, all-in-one database designed to manage complex, modern data types without requiring a database server.
-
-Design Philosophy
-`beaver` is built with a minimalistic philosophy for small, local use cases where a full-blown database server would be overkill.
-
-- **Minimalistic & Zero-Dependency**: Uses only Python's standard libraries (sqlite3, asyncio). No external packages are required, making it incredibly lightweight and portable.
-- **Async-First (When It Matters)**: The pub/sub system is fully asynchronous for high-performance, real-time messaging. Simpler features like key-value and list operations remain synchronous for ease of use.
-- **Built for Local Applications**: Perfect for local AI tools, chatbots (streaming tokens), task management apps, desktop utilities, and prototypes that need persistent, structured data without network overhead.
-- **Fast by Default**: It's built on SQLite, which is famously fast, reliable, and will likely serve your needs for a long way before you need a "professional" database.
-
-## Core Features
-
-- **Asynchronous Pub/Sub**: A fully asynchronous, Redis-like publish-subscribe system for real-time messaging.
-- **Persistent Key-Value Store**: A simple set/get interface for storing configuration, session data, or any other JSON-serializable object.
-- **Pythonic List Management**: A fluent, Redis-like interface (db.list("name").push()) for managing persistent, ordered lists with support for indexing and slicing.
-- **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
-
-## Installation
-
-```bash
-pip install beaver-db
-```
-
-## Quickstart & API Guide
-
-### 1. Initialization
-
-All you need to do is import and instantiate the BeaverDB class with a file path.
-
-```python
-from beaver import BeaverDB
-
-db = BeaverDB("my_application.db")
-```
-
-### 2. Key-Value Store
-
-Use `set()` and `get()` for simple data storage. The value can be any JSON-encodable object.
-
-```python
-# Set a value
-db.set("app_config", {"theme": "dark", "user_id": 123})
-
-# Get a value
-config = db.get("app_config")
-print(f"Theme: {config['theme']}") # Output: Theme: dark
-```
-
-### 3. List Management
-
-Get a list wrapper with `db.list()` and use Pythonic methods to manage it.
-
-```python
-# Get a wrapper for the 'tasks' list
-tasks = db.list("daily_tasks")
-
-# Push items to the list
-tasks.push("Write the project report")
-tasks.push("Send follow-up emails")
-tasks.prepend("Plan the day's agenda") # Push to the front
-
-# Use len() and indexing (including slices!)
-print(f"There are {len(tasks)} tasks.")
-print(f"The first task is: {tasks[0]}")
-print(f"The rest is: {tasks[1:]}")
-```
-
-### 4. Asynchronous Pub/Sub
-
-Publish events from one part of your app and listen in another using asyncio.
-
-```python
-import asyncio
-
-async def listener():
-    async with db.subscribe("system_events") as sub:
-        async for message in sub:
-            print(f"LISTENER: Received event -> {message['event']}")
-
-async def publisher():
-    await asyncio.sleep(1)
-    await db.publish("system_events", {"event": "user_login", "user": "alice"})
-
-# To run them concurrently:
-# asyncio.run(asyncio.gather(listener(), publisher()))
-```
-
-## Roadmap
-
-`beaver` aims to be a complete, self-contained data toolkit. The following features are planned:
-
-- **Vector Storage & Search**: Store NumPy vector embeddings and perform efficient k-nearest neighbor (k-NN) searches using `scipy.spatial.cKDTree`.
-- **JSON Document Store with Full-Text Search**: Store flexible JSON documents and get powerful full-text search across all text fields, powered by SQLite's FTS5 extension.
-- **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks (e.g., managing users, products) with standard SQL.
-
-## License
-
-This project is licensed under the MIT License.

beaver_db-0.2.0/README.md

@@ -1,102 +0,0 @@
-# beaver 🦫
-
-A fast, single-file, multi-modal database for Python, built with the standard sqlite3 library.
-
-`beaver` is the Backend for Embedded Asynchronous Vector & Event Retrieval. It's an industrious, all-in-one database designed to manage complex, modern data types without requiring a database server.
-
-Design Philosophy
-`beaver` is built with a minimalistic philosophy for small, local use cases where a full-blown database server would be overkill.
-
-- **Minimalistic & Zero-Dependency**: Uses only Python's standard libraries (sqlite3, asyncio). No external packages are required, making it incredibly lightweight and portable.
-- **Async-First (When It Matters)**: The pub/sub system is fully asynchronous for high-performance, real-time messaging. Simpler features like key-value and list operations remain synchronous for ease of use.
-- **Built for Local Applications**: Perfect for local AI tools, chatbots (streaming tokens), task management apps, desktop utilities, and prototypes that need persistent, structured data without network overhead.
-- **Fast by Default**: It's built on SQLite, which is famously fast, reliable, and will likely serve your needs for a long way before you need a "professional" database.
-
-## Core Features
-
-- **Asynchronous Pub/Sub**: A fully asynchronous, Redis-like publish-subscribe system for real-time messaging.
-- **Persistent Key-Value Store**: A simple set/get interface for storing configuration, session data, or any other JSON-serializable object.
-- **Pythonic List Management**: A fluent, Redis-like interface (db.list("name").push()) for managing persistent, ordered lists with support for indexing and slicing.
-- **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
-
-## Installation
-
-```bash
-pip install beaver-db
-```
-
-## Quickstart & API Guide
-
-### 1. Initialization
-
-All you need to do is import and instantiate the BeaverDB class with a file path.
-
-```python
-from beaver import BeaverDB
-
-db = BeaverDB("my_application.db")
-```
-
-### 2. Key-Value Store
-
-Use `set()` and `get()` for simple data storage. The value can be any JSON-encodable object.
-
-```python
-# Set a value
-db.set("app_config", {"theme": "dark", "user_id": 123})
-
-# Get a value
-config = db.get("app_config")
-print(f"Theme: {config['theme']}") # Output: Theme: dark
-```
-
-### 3. List Management
-
-Get a list wrapper with `db.list()` and use Pythonic methods to manage it.
-
-```python
-# Get a wrapper for the 'tasks' list
-tasks = db.list("daily_tasks")
-
-# Push items to the list
-tasks.push("Write the project report")
-tasks.push("Send follow-up emails")
-tasks.prepend("Plan the day's agenda") # Push to the front
-
-# Use len() and indexing (including slices!)
-print(f"There are {len(tasks)} tasks.")
-print(f"The first task is: {tasks[0]}")
-print(f"The rest is: {tasks[1:]}")
-```
-
-### 4. Asynchronous Pub/Sub
-
-Publish events from one part of your app and listen in another using asyncio.
-
-```python
-import asyncio
-
-async def listener():
-    async with db.subscribe("system_events") as sub:
-        async for message in sub:
-            print(f"LISTENER: Received event -> {message['event']}")
-
-async def publisher():
-    await asyncio.sleep(1)
-    await db.publish("system_events", {"event": "user_login", "user": "alice"})
-
-# To run them concurrently:
-# asyncio.run(asyncio.gather(listener(), publisher()))
-```
-
-## Roadmap
-
-`beaver` aims to be a complete, self-contained data toolkit. The following features are planned:
-
-- **Vector Storage & Search**: Store NumPy vector embeddings and perform efficient k-nearest neighbor (k-NN) searches using `scipy.spatial.cKDTree`.
-- **JSON Document Store with Full-Text Search**: Store flexible JSON documents and get powerful full-text search across all text fields, powered by SQLite's FTS5 extension.
-- **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks (e.g., managing users, products) with standard SQL.
-
-## License
-
-This project is licensed under the MIT License.

beaver_db-0.2.0/beaver/__init__.py

@@ -1 +0,0 @@
-from .core import BeaverDB, Subscriber

beaver_db-0.2.0/beaver_db.egg-info/PKG-INFO

@@ -1,109 +0,0 @@
-Metadata-Version: 2.4
-Name: beaver-db
-Version: 0.2.0
-Summary: Asynchronous, embedded, modern DB based on SQLite.
-Requires-Python: >=3.13
-Description-Content-Type: text/markdown
-
-# beaver 🦫
-
-A fast, single-file, multi-modal database for Python, built with the standard sqlite3 library.
-
-`beaver` is the Backend for Embedded Asynchronous Vector & Event Retrieval. It's an industrious, all-in-one database designed to manage complex, modern data types without requiring a database server.
-
-Design Philosophy
-`beaver` is built with a minimalistic philosophy for small, local use cases where a full-blown database server would be overkill.
-
-- **Minimalistic & Zero-Dependency**: Uses only Python's standard libraries (sqlite3, asyncio). No external packages are required, making it incredibly lightweight and portable.
-- **Async-First (When It Matters)**: The pub/sub system is fully asynchronous for high-performance, real-time messaging. Simpler features like key-value and list operations remain synchronous for ease of use.
-- **Built for Local Applications**: Perfect for local AI tools, chatbots (streaming tokens), task management apps, desktop utilities, and prototypes that need persistent, structured data without network overhead.
-- **Fast by Default**: It's built on SQLite, which is famously fast, reliable, and will likely serve your needs for a long way before you need a "professional" database.
-
-## Core Features
-
-- **Asynchronous Pub/Sub**: A fully asynchronous, Redis-like publish-subscribe system for real-time messaging.
-- **Persistent Key-Value Store**: A simple set/get interface for storing configuration, session data, or any other JSON-serializable object.
-- **Pythonic List Management**: A fluent, Redis-like interface (db.list("name").push()) for managing persistent, ordered lists with support for indexing and slicing.
-- **Single-File & Portable**: All data is stored in a single SQLite file, making it incredibly easy to move, back up, or embed in your application.
-
-## Installation
-
-```bash
-pip install beaver-db
-```
-
-## Quickstart & API Guide
-
-### 1. Initialization
-
-All you need to do is import and instantiate the BeaverDB class with a file path.
-
-```python
-from beaver import BeaverDB
-
-db = BeaverDB("my_application.db")
-```
-
-### 2. Key-Value Store
-
-Use `set()` and `get()` for simple data storage. The value can be any JSON-encodable object.
-
-```python
-# Set a value
-db.set("app_config", {"theme": "dark", "user_id": 123})
-
-# Get a value
-config = db.get("app_config")
-print(f"Theme: {config['theme']}") # Output: Theme: dark
-```
-
-### 3. List Management
-
-Get a list wrapper with `db.list()` and use Pythonic methods to manage it.
-
-```python
-# Get a wrapper for the 'tasks' list
-tasks = db.list("daily_tasks")
-
-# Push items to the list
-tasks.push("Write the project report")
-tasks.push("Send follow-up emails")
-tasks.prepend("Plan the day's agenda") # Push to the front
-
-# Use len() and indexing (including slices!)
-print(f"There are {len(tasks)} tasks.")
-print(f"The first task is: {tasks[0]}")
-print(f"The rest is: {tasks[1:]}")
-```
-
-### 4. Asynchronous Pub/Sub
-
-Publish events from one part of your app and listen in another using asyncio.
-
-```python
-import asyncio
-
-async def listener():
-    async with db.subscribe("system_events") as sub:
-        async for message in sub:
-            print(f"LISTENER: Received event -> {message['event']}")
-
-async def publisher():
-    await asyncio.sleep(1)
-    await db.publish("system_events", {"event": "user_login", "user": "alice"})
-
-# To run them concurrently:
-# asyncio.run(asyncio.gather(listener(), publisher()))
-```
-
-## Roadmap
-
-`beaver` aims to be a complete, self-contained data toolkit. The following features are planned:
-
-- **Vector Storage & Search**: Store NumPy vector embeddings and perform efficient k-nearest neighbor (k-NN) searches using `scipy.spatial.cKDTree`.
-- **JSON Document Store with Full-Text Search**: Store flexible JSON documents and get powerful full-text search across all text fields, powered by SQLite's FTS5 extension.
-- **Standard Relational Interface**: While `beaver` provides high-level features, you can always use the same SQLite file for normal relational tasks (e.g., managing users, products) with standard SQL.
-
-## License
-
-This project is licensed under the MIT License.