cognee-community-vector-adapter-valkey 0.1.1__tar.gz

cognee_community_vector_adapter_valkey-0.1.1/.gitignore

@@ -0,0 +1,24 @@
+# Python-generated files
+__pycache__/
+*.py[oc]
+build/
+dist/
+wheels/
+*.egg-info
+
+# Virtual environments
+.venv
+.idea
+# Env config
+.env
+
+# Cognee system
+# for backward compatibility
+.cognee-data
+.cognee-system
+
+# for new version
+.cognee_data
+.cognee_system
+.data_storage
+*.log

cognee_community_vector_adapter_valkey-0.1.1/PKG-INFO

@@ -0,0 +1,160 @@
+Metadata-Version: 2.4
+Name: cognee-community-vector-adapter-valkey
+Version: 0.1.1
+Summary: Valkey vector database adapter for cognee
+Requires-Python: <=3.13,>=3.11
+Requires-Dist: cognee==0.5.2
+Requires-Dist: numpy>=1.24.0
+Requires-Dist: valkey-glide>=2.1.0
+Provides-Extra: dev
+Requires-Dist: anyio>=4.0; extra == 'dev'
+Requires-Dist: mypy>=1.17.1; extra == 'dev'
+Requires-Dist: pytest-asyncio>=0.23; extra == 'dev'
+Requires-Dist: pytest-cov>=4.0.0; extra == 'dev'
+Requires-Dist: pytest>=7.4; extra == 'dev'
+Provides-Extra: test
+Requires-Dist: pytest-asyncio>=0.23; extra == 'test'
+Requires-Dist: pytest>=7.4; extra == 'test'
+Requires-Dist: valkey-glide>=2.1.0; extra == 'test'
+Description-Content-Type: text/markdown
+
+# Cognee Valkey Vector Adapter
+
+A Valkey vector database adapter for Cognee using Valkey Glide, providing high-performance vector storage and retrieval for AI memory applications. Compared to the Redis adapter, Valkey offers a fully open-source, community-driven architecture without the licensing restrictions of Redis. Using Valkey Glide ensures efficient async operations and native support for Valkey’s enhancements, providing optimal compatibility and performance when running on Valkey, making it the best choice for teams adopting Valkey as their primary in-memory vector solution.
+
+## Features
+
+- Full support for vector embeddings storage and retrieval
+- Batch / pipeline operations for efficient processing
+- Automatic embedding generation via configurable embedding engines
+- Comprehensive error handling
+
+## Installation
+
+If published, the package can be simply installed via pip:
+
+```bash
+pip install cognee-community-vector-adapter-valkey
+```
+
+In case it is not published yet, you can use poetry to locally build the adapter package:
+
+```bash
+pip install uv
+uv sync --all-extras
+```
+
+## Prerequisites
+
+You need a Valkey instance with the Valkey Search module enabled. You can use:
+
+1. **Valkey**:
+   ```bash
+   docker run -d --name valkey -p 6379:6379 valkey/valkey-bundle
+   ```
+   
+## Examples
+Checkout the `examples/` folder!
+
+```bash
+uv run examples/example.py
+```
+
+>You will need an OpenAI API key to run the example script.
+
+## Configuration
+
+Configure Valkey as your vector database in cognee:
+
+- `vector_db_provider`: Set to "valkey"
+- `vector_db_url`: Valkey connection URL (e.g., "valkey://localhost:6379")
+
+### Environment Variables
+
+Set the following environment variables or pass them directly in the config:
+
+```bash
+export VECTOR_DB_URL="valkey://localhost:6379"
+```
+
+### Connection URL Examples
+
+```python
+# Local Valkey
+config.set_vector_db_config({
+    "vector_db_provider": "valkey",
+    "vector_db_url": "valkey://localhost:6379"
+})
+
+# Valkey with authentication
+config.set_vector_db_config({
+    "vector_db_provider": "valkey", 
+    "vector_db_url": "valkey://user:password@localhost:6379"
+})
+```
+
+## Requirements
+
+- Python >= 3.11, <= 3.13
+- valkey-glide >= 2.1.0
+- cognee >= 0.4.0
+
+## Advanced Usage
+
+For direct adapter usage (advanced users only):
+
+```python
+from cognee.infrastructure.databases.vector.embeddings.EmbeddingEngine import EmbeddingEngine
+from cognee_community_vector_adapter_valkey import ValkeyAdapter
+from cognee.infrastructure.engine import DataPoint
+
+# Initialize embedding engine and adapter
+embedding_engine = EmbeddingEngine(model="your-model")
+valkey_adapter = ValkeyAdapter(
+    url="valkey://localhost:6379",
+    embedding_engine=embedding_engine
+)
+
+# Direct adapter operations
+await valkey_adapter.create_collection("my_collection")
+data_points = [DataPoint(id="1", text="Hello", metadata={"index_fields": ["text"]})]
+await valkey_adapter.create_data_points("my_collection", data_points)
+results = await valkey_adapter.search("my_collection", query_text="Hello", limit=10)
+```
+
+## Error Handling
+
+The adapter includes comprehensive error handling:
+
+- `VectorEngineInitializationError`: Raised when required parameters are missing
+- `CollectionNotFoundError`: Raised when attempting operations on non-existent collections
+- `InvalidValueError`: Raised for invalid query parameters
+- Graceful handling of connection failures and embedding errors
+
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Connection Errors**: Ensure Valkey is running and accessible at the specified URL
+2. **Search Module Missing**: Make sure Valkey has the Search module enabled
+3. **Embedding Dimension Mismatch**: Verify embedding engine dimensions match index configuration
+4. **Collection Not Found**: Always create collections before adding data points
+
+### Debug Logging
+
+The adapter uses Cognee's logging system. Enable debug logging to see detailed operation logs:
+
+```python
+import logging
+logging.getLogger("ValkeyAdapter").setLevel(logging.DEBUG)
+```
+
+## Development
+
+To contribute or modify the adapter:
+
+1. Clone the repository and `cd` into the `valkey` folder
+2. Install dependencies: `uv sync --all-extras`
+3. Make sure a Valkey instance is running (see above)
+5. Make your changes, test, and submit a PR

cognee_community_vector_adapter_valkey-0.1.1/README.md

@@ -0,0 +1,140 @@
+# Cognee Valkey Vector Adapter
+
+A Valkey vector database adapter for Cognee using Valkey Glide, providing high-performance vector storage and retrieval for AI memory applications. Compared to the Redis adapter, Valkey offers a fully open-source, community-driven architecture without the licensing restrictions of Redis. Using Valkey Glide ensures efficient async operations and native support for Valkey’s enhancements, providing optimal compatibility and performance when running on Valkey, making it the best choice for teams adopting Valkey as their primary in-memory vector solution.
+
+## Features
+
+- Full support for vector embeddings storage and retrieval
+- Batch / pipeline operations for efficient processing
+- Automatic embedding generation via configurable embedding engines
+- Comprehensive error handling
+
+## Installation
+
+If published, the package can be simply installed via pip:
+
+```bash
+pip install cognee-community-vector-adapter-valkey
+```
+
+In case it is not published yet, you can use poetry to locally build the adapter package:
+
+```bash
+pip install uv
+uv sync --all-extras
+```
+
+## Prerequisites
+
+You need a Valkey instance with the Valkey Search module enabled. You can use:
+
+1. **Valkey**:
+   ```bash
+   docker run -d --name valkey -p 6379:6379 valkey/valkey-bundle
+   ```
+   
+## Examples
+Checkout the `examples/` folder!
+
+```bash
+uv run examples/example.py
+```
+
+>You will need an OpenAI API key to run the example script.
+
+## Configuration
+
+Configure Valkey as your vector database in cognee:
+
+- `vector_db_provider`: Set to "valkey"
+- `vector_db_url`: Valkey connection URL (e.g., "valkey://localhost:6379")
+
+### Environment Variables
+
+Set the following environment variables or pass them directly in the config:
+
+```bash
+export VECTOR_DB_URL="valkey://localhost:6379"
+```
+
+### Connection URL Examples
+
+```python
+# Local Valkey
+config.set_vector_db_config({
+    "vector_db_provider": "valkey",
+    "vector_db_url": "valkey://localhost:6379"
+})
+
+# Valkey with authentication
+config.set_vector_db_config({
+    "vector_db_provider": "valkey", 
+    "vector_db_url": "valkey://user:password@localhost:6379"
+})
+```
+
+## Requirements
+
+- Python >= 3.11, <= 3.13
+- valkey-glide >= 2.1.0
+- cognee >= 0.4.0
+
+## Advanced Usage
+
+For direct adapter usage (advanced users only):
+
+```python
+from cognee.infrastructure.databases.vector.embeddings.EmbeddingEngine import EmbeddingEngine
+from cognee_community_vector_adapter_valkey import ValkeyAdapter
+from cognee.infrastructure.engine import DataPoint
+
+# Initialize embedding engine and adapter
+embedding_engine = EmbeddingEngine(model="your-model")
+valkey_adapter = ValkeyAdapter(
+    url="valkey://localhost:6379",
+    embedding_engine=embedding_engine
+)
+
+# Direct adapter operations
+await valkey_adapter.create_collection("my_collection")
+data_points = [DataPoint(id="1", text="Hello", metadata={"index_fields": ["text"]})]
+await valkey_adapter.create_data_points("my_collection", data_points)
+results = await valkey_adapter.search("my_collection", query_text="Hello", limit=10)
+```
+
+## Error Handling
+
+The adapter includes comprehensive error handling:
+
+- `VectorEngineInitializationError`: Raised when required parameters are missing
+- `CollectionNotFoundError`: Raised when attempting operations on non-existent collections
+- `InvalidValueError`: Raised for invalid query parameters
+- Graceful handling of connection failures and embedding errors
+
+
+## Troubleshooting
+
+### Common Issues
+
+1. **Connection Errors**: Ensure Valkey is running and accessible at the specified URL
+2. **Search Module Missing**: Make sure Valkey has the Search module enabled
+3. **Embedding Dimension Mismatch**: Verify embedding engine dimensions match index configuration
+4. **Collection Not Found**: Always create collections before adding data points
+
+### Debug Logging
+
+The adapter uses Cognee's logging system. Enable debug logging to see detailed operation logs:
+
+```python
+import logging
+logging.getLogger("ValkeyAdapter").setLevel(logging.DEBUG)
+```
+
+## Development
+
+To contribute or modify the adapter:
+
+1. Clone the repository and `cd` into the `valkey` folder
+2. Install dependencies: `uv sync --all-extras`
+3. Make sure a Valkey instance is running (see above)
+5. Make your changes, test, and submit a PR

cognee_community_vector_adapter_valkey-0.1.1/__init__.py

@@ -0,0 +1,7 @@
+from .cognee_community_vector_adapter_valkey import ValkeyAdapter
+from .cognee_community_vector_adapter_valkey.exceptions import (
+    CollectionNotFoundError,
+    ValkeyVectorEngineInitializationError,
+)
+
+__all__ = ["ValkeyAdapter", "ValkeyVectorEngineInitializationError", "CollectionNotFoundError"]

cognee_community_vector_adapter_valkey-0.1.1/cognee_community_vector_adapter_valkey/__init__.py

@@ -0,0 +1,4 @@
+from .exceptions import CollectionNotFoundError, ValkeyVectorEngineInitializationError
+from .valkey_adapter import ValkeyAdapter
+
+__all__ = ["ValkeyAdapter", "ValkeyVectorEngineInitializationError", "CollectionNotFoundError"]

cognee_community_vector_adapter_valkey-0.1.1/cognee_community_vector_adapter_valkey/exceptions.py

@@ -0,0 +1,10 @@
+class ValkeyVectorEngineInitializationError(Exception):
+    """Exception raised when vector engine initialization fails."""
+
+    pass
+
+
+class CollectionNotFoundError(Exception):
+    """Exception raised when a collection is not found."""
+
+    pass

cognee_community_vector_adapter_valkey-0.1.1/cognee_community_vector_adapter_valkey/register.py

@@ -0,0 +1,5 @@
+from cognee.infrastructure.databases.vector import use_vector_adapter
+
+from .valkey_adapter import ValkeyAdapter
+
+use_vector_adapter("valkey", ValkeyAdapter)

cognee_community_vector_adapter_valkey-0.1.1/cognee_community_vector_adapter_valkey/utils.py

@@ -0,0 +1,180 @@
+from __future__ import annotations
+
+import json
+import struct
+from functools import singledispatch
+from typing import Any
+from urllib.parse import urlparse
+from uuid import UUID
+
+from cognee.infrastructure.databases.vector.models.ScoredResult import ScoredResult
+
+"""
+Internal helper function. Not part of the public API.
+"""
+
+
+def _parse_host_port(url: str) -> tuple[str, int]:
+    """
+    Parse a url and extract the host and port.
+
+    Args:
+        url (str): The connection URL, e.g., "valkey://localhost:6379".
+
+    Returns:
+        tuple[str, int]: A tuple containing:
+            - host (str): The hostname from the URL, defaults to "localhost" if missing.
+            - port (int): The port number from the URL, defaults to 6379 if missing.
+    """
+
+    parsed = urlparse(url)
+    host = parsed.hostname or "localhost"
+    port = parsed.port or 6379
+    return host, port
+
+
+def _to_float32_bytes(vec) -> bytes:
+    """
+    Convert a sequence of numeric values into a bytes representation using 32-bit floats.
+
+    Args:
+        vec (Iterable[float]): A sequence of numbers (e.g., list, tuple) to be converted.
+
+    Returns:
+        bytes: A binary representation of the input values packed as consecutive 32-bit floats.
+
+    Notes:
+        - Uses `struct.pack` with the format string `"{len(vec)}f"`, which packs all values as
+          IEEE 754 single-precision floats.
+        - Ensures compatibility with vector databases or embedding engines that require raw
+          float32 byte arrays.
+    """
+
+    return struct.pack(f"{len(vec)}f", *map(float, vec))
+
+
+@singledispatch
+def _serialize_for_json(obj: Any) -> Any:
+    """Convert objects to JSON-serializable format.
+    This id default serialization: return the object as-is.
+
+    Args:
+        obj: Object to serialize (UUID, dict, list, or any other type).
+
+    Returns:
+        JSON-serializable representation of the object.
+    """
+    return obj
+
+
+@_serialize_for_json.register
+def _(obj: UUID) -> str:
+    return str(obj)
+
+
+@_serialize_for_json.register
+def _(obj: dict) -> dict:
+    return {k: _serialize_for_json(v) for k, v in obj.items()}
+
+
+@_serialize_for_json.register
+def _(obj: list) -> list:
+    return [_serialize_for_json(item) for item in obj]
+
+
+def _b2s(x: bytes | bytearray | str) -> str:
+    """Convert bytes or bytearray to a UTF-8 string if possible,
+        otherwise return a string representation.
+
+    Args:
+        x (Any): The input value, which may be bytes, bytearray, or any other type.
+
+    Returns:
+        Any: A decoded UTF-8 string if `x` is bytes or bytearray; otherwise, returns `x` unchanged.
+            If decoding fails, returns the string representation of `x`.
+    """
+
+    if isinstance(x, (bytes, bytearray)):
+        try:
+            return x.decode("utf-8")
+        except Exception:
+            return str(x)
+    return x
+
+
+def _build_scored_results_from_ft(
+    raw: Any,
+    *,
+    use_key_suffix_when_missing_id: bool = True,
+) -> list[ScoredResult]:
+    """Build a list of `ScoredResult` objects from raw FT (Full-Text) search response.
+
+    Args:
+        raw (Any): The raw response from Valkey's FT search command, expected to be a list or tuple
+                   where the second element is a mapping of keys to field dictionaries.
+        use_key_suffix_when_missing_id (bool): If True, use the key string as the ID when the `id`
+                   field is missing in the response.
+
+    Returns:
+        list[ScoredResult]: A list of scored results, each containing:
+            - id (str): Extracted from `id` field or fallback to key.
+            - payload (dict): Parsed JSON from `payload_data` field, or raw string if malformed.
+            - score (float | None): Extracted from `__vector_score` field if present.
+
+    Notes:
+        - Handles both byte keys and string keys by decoding them.
+        - Gracefully falls back when fields are missing or payload is invalid JSON.
+    """
+    if not isinstance(raw, (list, tuple)) or len(raw) < 2 or not isinstance(raw[1], dict):
+        return []
+
+    mapping: dict[Any, dict[Any, Any]] = raw[1]  # the { key -> fields } dict
+    scored: list[ScoredResult] = []
+
+    for key_bytes, fields in mapping.items():
+        key_str = _b2s(key_bytes)
+
+        # Extract id
+        raw_id = fields.get(b"id") if b"id" in fields else fields.get("id")
+        if raw_id is not None:
+            result_id = _b2s(raw_id)
+        else:
+            result_id = key_str
+
+        # Extrat score
+        score = (
+            fields.get(b"__vector_score")
+            if b"__vector_score" in fields
+            else fields.get("__vector_score")
+        )
+        if score is not None:
+            score = float(score)
+
+        # Extract and parse payload_data
+        payload_raw = (
+            fields.get(b"payload_data") if b"payload_data" in fields else fields.get("payload_data")
+        )
+        payload: dict[str, Any] = {}
+        if payload_raw is not None:
+            payload_str = _b2s(payload_raw)
+            if isinstance(payload_str, str):
+                try:
+                    obj = json.loads(payload_str)
+                    if isinstance(obj, dict):
+                        payload = obj
+                    else:
+                        # If it's not a dict (e.g., list), wrap it
+                        payload = {"_payload": obj}
+                except json.JSONDecodeError:
+                    # Keep the raw string if malformed
+                    payload = {"_payload_raw": payload_str}
+
+        scored.append(
+            ScoredResult(
+                id=result_id,
+                payload=payload,
+                score=score,
+            )
+        )
+
+    return scored