qdrant-haystack 8.0.0__tar.gz → 9.0.0__tar.gz

{qdrant_haystack-8.0.0 → qdrant_haystack-9.0.0}/CHANGELOG.md

@@ -1,22 +1,58 @@
 # Changelog
 
+## [integrations/qdrant-v8.1.0] - 2025-03-07
+
+### 🧹 Chores
+
+- Remove Python 3.8 support (#1421)
+
+### 🌀 Miscellaneous
+
+- Docs: Update document store descriptions for deepset Pipeline Builder (#1447)
+- Refactor: Qdrant - raise error if existing collection is not compatible with Haystack (#1481)
+
+## [integrations/qdrant-v8.0.0] - 2025-02-19
+
+### 🧹 Chores
+
+- Fix linting/isort (#1215)
+- Inherit from `FilterDocumentsTestWithDataframe` in Document Stores (#1290)
+- [**breaking**] Qdrant - remove dataframe support (#1403)
+
+
 ## [integrations/qdrant-v7.0.0] - 2024-10-29
 
-### ⚙️ Miscellaneous Tasks
+### ⚙️ CI
 
-- Update ruff linting scripts and settings (#1105)
 - Adopt uv as installer (#1142)
 
+### 🧹 Chores
+
+- Update ruff linting scripts and settings (#1105)
+
+### 🌀 Miscellaneous
+
+- Refactor!: Qdrant - remove `index` parameter from methods (#1160)
+
 ## [integrations/qdrant-v6.0.0] - 2024-09-13
 
+### 🌀 Miscellaneous
+
+- Remove support for deprecated legacy filters in Qdrant (#1084)
+
 ## [integrations/qdrant-v5.1.0] - 2024-09-12
 
 ### 🚀 Features
 
 - Qdrant - Add group_by and group_size optional parameters to Retrievers (#1054)
 
+
 ## [integrations/qdrant-v5.0.0] - 2024-09-02
 
+### 🌀 Miscellaneous
+
+- Fix!: fix type errors in `QdrantDocumentStore`; rename `ids` (parameter of `delete_documents`) to `document_ids` (#1041)
+
 ## [integrations/qdrant-v4.2.0] - 2024-08-27
 
 ### 🚜 Refactor
@@ -27,12 +63,18 @@
 
 - Do not retry tests in `hatch run test` command (#954)
 
+### 🌀 Miscellaneous
+
+- Chore: Update Qdrant tests for the new `apply_filter_policy` usage (#969)
+- Chore: qdrant - ruff update, don't ruff tests (#989)
+
 ## [integrations/qdrant-v4.1.2] - 2024-07-15
 
 ### 🐛 Bug Fixes
 
 - `qdrant` - Fallback to default filter policy when deserializing retrievers without the init parameter (#902)
 
+
 ## [integrations/qdrant-v4.1.1] - 2024-07-10
 
 ### 🚀 Features
@@ -43,6 +85,10 @@
 
 - Errors in convert_filters_to_qdrant (#870)
 
+### 🌀 Miscellaneous
+
+- Chore: Minor retriever pydoc fix (#884)
+
 ## [integrations/qdrant-v4.1.0] - 2024-07-03
 
 ### 🚀 Features
@@ -50,6 +96,7 @@
 - Add `score_threshold` to Qdrant Retrievers (#860)
 - Qdrant - add support for BM42 (#864)
 
+
 ## [integrations/qdrant-v4.0.0] - 2024-07-02
 
 ### 🚜 Refactor
@@ -57,17 +104,25 @@
 - [**breaking**] Qdrant - remove unused init parameters: `content_field`, `name_field`, `embedding_field`, and `duplicate_documents` (#861)
 - [**breaking**] Qdrant - set `scale_score` default value to `False` (#862)
 
-### ⚙️ Miscellaneous Tasks
+### ⚙️ CI
 
 - Retry tests to reduce flakyness (#836)
+
+### 🧹 Chores
+
 - Update ruff invocation to include check parameter (#853)
 
+### 🌀 Miscellaneous
+
+- Ci: install `pytest-rerunfailures` where needed; add retry config to `test-cov` script (#845)
+
 ## [integrations/qdrant-v3.8.1] - 2024-06-20
 
 ### 📚 Documentation
 
 - Added docstrings for QdrantDocumentStore (#808)
 
+
 ## [integrations/qdrant-v3.8.0] - 2024-06-06
 
 ### 🚀 Features
@@ -86,17 +141,29 @@
 
 - Defer database connection to the first usage (#748)
 
+### 🌀 Miscellaneous
+
+- Qdrant - improve docstrings for retrievers (#687)
+- Chore: change the pydoc renderer class (#718)
+- Allow vanilla qdrant filters (#692)
+
 ## [integrations/qdrant-v3.5.0] - 2024-04-24
 
+### 🌀 Miscellaneous
+
+- Chore: add license classifiers (#680)
+- Qdrant - add hybrid retriever (#675)
+
 ## [integrations/qdrant-v3.4.0] - 2024-04-23
 
-### Qdrant
+### 🌀 Miscellaneous
 
 - Add embedding retrieval example (#666)
+- Rename `QdrantSparseRetriever` to `QdrantSparseEmbeddingRetriever` (#681)
 
 ## [integrations/qdrant-v3.3.1] - 2024-04-12
 
-### Qdrant
+### 🌀 Miscellaneous
 
 - Add migration utility function for Sparse Embedding support (#659)
 
@@ -110,7 +177,7 @@
 
 ### 🐛 Bug Fixes
 
-- Fix haystack-ai pin (#649)
+- Fix `haystack-ai` pins (#649)
 
 ## [integrations/qdrant-v3.2.0] - 2024-03-27
 
@@ -121,11 +188,9 @@
 
 ### 🐛 Bug Fixes
 
-- Fix linter errors (#282)
+- Fix: fix linter errors (#282)
 - Fix order of API docs (#447)
-
-This PR will also push the docs to Readme
-- Fixes (#518)
+- Doc: fixing docstrings for qdrant (#518)
 
 ### 🚜 Refactor
 
@@ -137,13 +202,19 @@ This PR will also push the docs to Readme
 - Small consistency improvements (#536)
 - Disable-class-def (#556)
 
-### ⚙️ Miscellaneous Tasks
+### ⚙️ CI
 
 - Generate API docs for Qdrant (#361)
 
+### 🌀 Miscellaneous
+
+- Make tests show coverage (#566)
+- Remove references to Python 3.7 (#601)
+- Make Document Stores initially skip `SparseEmbedding` (#606)
+
 ## [integrations/qdrant-v3.0.0] - 2024-01-22
 
-### Refact
+### 🌀 Miscellaneous
 
 - [**breaking**] Change import paths (#255)
 
@@ -161,4 +232,8 @@ This PR will also push the docs to Readme
 
 - Use `hatch_vcs` to manage integrations versioning (#103)
 
+### 🌀 Miscellaneous
+
+- Renamed QdrntRetriever to QdrntEmbeddingRetriever (#174)
+
 <!-- generated by git-cliff -->

{qdrant_haystack-8.0.0 → qdrant_haystack-9.0.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: qdrant-haystack
-Version: 8.0.0
+Version: 9.0.0
 Summary: An integration of Qdrant ANN vector database backend with Haystack
 Project-URL: Source, https://github.com/deepset-ai/haystack-core-integrations
 Project-URL: Documentation, https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/qdrant/README.md
@@ -11,14 +11,14 @@ License-File: LICENSE.txt
 Classifier: Development Status :: 4 - Beta
 Classifier: License :: OSI Approved :: Apache Software License
 Classifier: Programming Language :: Python
-Classifier: Programming Language :: Python :: 3.8
 Classifier: Programming Language :: Python :: 3.9
 Classifier: Programming Language :: Python :: 3.10
 Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: Implementation :: CPython
 Classifier: Programming Language :: Python :: Implementation :: PyPy
-Requires-Python: >=3.8
-Requires-Dist: haystack-ai
+Requires-Python: >=3.9
+Requires-Dist: haystack-ai>=2.11.0
 Requires-Dist: qdrant-client>=1.10.0
 Description-Content-Type: text/markdown
 

{qdrant_haystack-8.0.0 → qdrant_haystack-9.0.0}/pyproject.toml

@@ -7,7 +7,7 @@ name = "qdrant-haystack"
 dynamic = ["version"]
 description = 'An integration of Qdrant ANN vector database backend with Haystack'
 readme = "README.md"
-requires-python = ">=3.8"
+requires-python = ">=3.9"
 license = "Apache-2.0"
 keywords = []
 authors = [
@@ -18,14 +18,14 @@ classifiers = [
   "License :: OSI Approved :: Apache Software License",
   "Development Status :: 4 - Beta",
   "Programming Language :: Python",
-  "Programming Language :: Python :: 3.8",
   "Programming Language :: Python :: 3.9",
   "Programming Language :: Python :: 3.10",
   "Programming Language :: Python :: 3.11",
+  "Programming Language :: Python :: 3.12",
   "Programming Language :: Python :: Implementation :: CPython",
   "Programming Language :: Python :: Implementation :: PyPy",
 ]
-dependencies = ["haystack-ai", "qdrant-client>=1.10.0"]
+dependencies = ["haystack-ai>=2.11.0", "qdrant-client>=1.10.0"]
 
 [project.urls]
 Source = "https://github.com/deepset-ai/haystack-core-integrations"
@@ -56,7 +56,7 @@ cov-retry = ["test-cov-retry", "cov-report"]
 docs = ["pydoc-markdown pydoc/config.yml"]
 
 [[tool.hatch.envs.all.matrix]]
-python = ["3.8", "3.9", "3.10", "3.11"]
+python = [ "3.9", "3.10", "3.11"]
 
 [tool.hatch.envs.lint]
 installer = "uv"

{qdrant_haystack-8.0.0 → qdrant_haystack-9.0.0}/src/haystack_integrations/document_stores/qdrant/converters.py

@@ -1,7 +1,7 @@
-import logging
 import uuid
 from typing import List, Union
 
+from haystack import logging
 from haystack.dataclasses import Document
 from qdrant_client.http import models as rest
 
@@ -23,14 +23,6 @@ def convert_haystack_documents_to_qdrant_points(
     for document in documents:
         payload = document.to_dict(flatten=False)
 
-        if payload.pop("dataframe", None):
-            logger.warning(
-                "Document %s has the `dataframe` field set,"
-                "QdrantDocumentStore no longer supports dataframes and this field will be ignored. "
-                "The `dataframe` field will soon be removed from Haystack Document.",
-                document.id,
-            )
-
         if use_sparse_embeddings:
             vector = {}
 
@@ -73,14 +65,6 @@ def convert_qdrant_point_to_haystack_document(point: QdrantPoint, use_sparse_emb
     payload = {**point.payload}
     payload["score"] = point.score if hasattr(point, "score") else None
 
-    if payload.pop("dataframe", None):
-        logger.warning(
-            "Document %s has the `dataframe` field set,"
-            "QdrantDocumentStore no longer supports dataframes and this field will be ignored. "
-            "The `dataframe` field will soon be removed from Haystack Document.",
-            payload["id"],
-        )
-
     if not use_sparse_embeddings:
         payload["embedding"] = point.vector if hasattr(point, "vector") else None
     elif hasattr(point, "vector") and point.vector is not None:

{qdrant_haystack-8.0.0 → qdrant_haystack-9.0.0}/src/haystack_integrations/document_stores/qdrant/document_store.py

@@ -1,11 +1,10 @@
 import inspect
-import logging
 from itertools import islice
 from typing import Any, ClassVar, Dict, Generator, List, Optional, Set, Union
 
 import numpy as np
 import qdrant_client
-from haystack import default_from_dict, default_to_dict
+from haystack import default_from_dict, default_to_dict, logging
 from haystack.dataclasses import Document
 from haystack.dataclasses.sparse_embedding import SparseEmbedding
 from haystack.document_stores.errors import DocumentStoreError, DuplicateDocumentError
@@ -48,8 +47,8 @@ def get_batches_from_generator(iterable, n):
 
 class QdrantDocumentStore:
     """
-    QdrantDocumentStore is a Document Store for Qdrant.
-    It can be used with any Qdrant instance: in-memory, disk-persisted, Docker-based,
+    A QdrantDocumentStore implementation that you
+    can use with any Qdrant instance: in-memory, disk-persisted, Docker-based,
     and Qdrant Cloud Cluster deployments.
 
     Usage example by creating an in-memory instance:
@@ -866,10 +865,18 @@ class QdrantDocumentStore:
 
         collection_info = self.client.get_collection(collection_name)
 
-        has_named_vectors = (
-            isinstance(collection_info.config.params.vectors, dict)
-            and DENSE_VECTORS_NAME in collection_info.config.params.vectors
-        )
+        has_named_vectors = isinstance(collection_info.config.params.vectors, dict)
+
+        if has_named_vectors and DENSE_VECTORS_NAME not in collection_info.config.params.vectors:
+            msg = (
+                f"Collection '{collection_name}' already exists in Qdrant, "
+                f"but it has been originally created outside of Haystack and is not supported. "
+                f"If possible, you should create a new Document Store with Haystack. "
+                f"In case you want to migrate the existing collection, see an example script in "
+                f"https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/qdrant/src/"
+                f"haystack_integrations/document_stores/qdrant/migrate_to_sparse.py."
+            )
+            raise QdrantStoreError(msg)
 
         if self.use_sparse_embeddings and not has_named_vectors:
             msg = (
@@ -882,7 +889,7 @@ class QdrantDocumentStore:
             )
             raise QdrantStoreError(msg)
 
-        elif not self.use_sparse_embeddings and has_named_vectors:
+        if not self.use_sparse_embeddings and has_named_vectors:
             msg = (
                 f"Collection '{collection_name}' already exists in Qdrant, "
                 f"but it has been originally created with sparse embedding vectors."

{qdrant_haystack-8.0.0 → qdrant_haystack-9.0.0}/src/haystack_integrations/document_stores/qdrant/migrate_to_sparse.py

@@ -1,13 +1,14 @@
-import logging
+import logging as python_logging
 import time
 
+from haystack import logging
 from qdrant_client.http import models
 
 from haystack_integrations.document_stores.qdrant import QdrantDocumentStore
 
 logger = logging.getLogger(__name__)
-logger.addHandler(logging.StreamHandler())
-logger.setLevel(logging.INFO)
+logger.addHandler(python_logging.StreamHandler())
+logger.setLevel(python_logging.INFO)
 
 
 def migrate_to_sparse_embeddings_support(old_document_store: QdrantDocumentStore, new_index: str):

{qdrant_haystack-8.0.0 → qdrant_haystack-9.0.0}/tests/test_converters.py

@@ -1,10 +1,7 @@
 import numpy as np
-from haystack import Document
-from pandas import DataFrame
 from qdrant_client.http import models as rest
 
 from haystack_integrations.document_stores.qdrant.converters import (
-    convert_haystack_documents_to_qdrant_points,
     convert_id,
     convert_qdrant_point_to_haystack_document,
 )
@@ -65,44 +62,3 @@ def test_point_to_document_reverts_proper_structure_from_record_without_sparse()
     assert document.sparse_embedding is None
     assert {"test_field": 1} == document.meta
     assert 0.0 == np.sum(np.array([1.0, 0.0, 0.0, 0.0]) - document.embedding)
-
-
-def test_point_to_document_skips_dataframe():
-
-    point = rest.Record(
-        id="c7c62e8e-02b9-4ec6-9f88-46bd97b628b7",
-        payload={
-            "id": "my-id",
-            "content": "Lorem ipsum",
-            "content_type": "text",
-            "meta": {
-                "test_field": 1,
-            },
-            "dataframe": {"a": [1, 2, 3]},
-        },
-        vector=[1.0, 0.0, 0.0, 0.0],
-    )
-    document = convert_qdrant_point_to_haystack_document(point, use_sparse_embeddings=False)
-    assert "my-id" == document.id
-    assert "Lorem ipsum" == document.content
-    assert "text" == document.content_type
-    assert {"test_field": 1} == document.meta
-    assert 0.0 == np.sum(np.array([1.0, 0.0, 0.0, 0.0]) - document.embedding)
-    assert not hasattr(document, "dataframe") or document.dataframe is None
-
-
-def test_documents_to_points_skips_dataframe():
-    doc = Document(
-        id="my-id",
-        content="Lorem ipsum",
-        embedding=[1.0, 0.0, 0.0, 0.0],
-    )
-
-    doc.dataframe = DataFrame([[1, 2], [3, 4]])
-
-    points = convert_haystack_documents_to_qdrant_points([doc], use_sparse_embeddings=False)
-    assert len(points) == 1
-
-    assert points[0].payload["content"] == "Lorem ipsum"
-    assert points[0].vector == [1.0, 0.0, 0.0, 0.0]
-    assert "dataframe" not in points[0].payload

{qdrant_haystack-8.0.0 → qdrant_haystack-9.0.0}/tests/test_document_store.py

@@ -1,5 +1,5 @@
 from typing import List
-from unittest.mock import patch
+from unittest.mock import MagicMock, patch
 
 import pytest
 from haystack import Document
@@ -15,6 +15,7 @@ from haystack.testing.document_store import (
 from qdrant_client.http import models as rest
 
 from haystack_integrations.document_stores.qdrant.document_store import (
+    DENSE_VECTORS_NAME,
     SPARSE_VECTORS_NAME,
     QdrantDocumentStore,
     QdrantStoreError,
@@ -151,3 +152,79 @@ class TestQdrantDocumentStore(CountDocumentsTest, WriteDocumentsTest, DeleteDocu
 
             with pytest.raises(QdrantStoreError):
                 document_store._query_hybrid(query_sparse_embedding=sparse_embedding, query_embedding=embedding)
+
+    def test_set_up_collection_with_existing_incompatible_collection(self):
+        document_store = QdrantDocumentStore(location=":memory:", use_sparse_embeddings=True)
+
+        # Mock collection info with named vectors but missing DENSE_VECTORS_NAME
+        mock_collection_info = MagicMock()
+        mock_collection_info.config.params.vectors = {"some_other_vector": MagicMock()}
+
+        with patch.object(document_store.client, "collection_exists", return_value=True), patch.object(
+            document_store.client, "get_collection", return_value=mock_collection_info
+        ):
+
+            with pytest.raises(QdrantStoreError, match="created outside of Haystack"):
+                document_store._set_up_collection("test_collection", 768, False, "cosine", True, False)
+
+    def test_set_up_collection_use_sparse_embeddings_true_without_named_vectors(self):
+        """Test that an error is raised when use_sparse_embeddings is True but collection doesn't have named vectors"""
+        document_store = QdrantDocumentStore(location=":memory:", use_sparse_embeddings=True)
+
+        # Mock collection info without named vectors
+        mock_collection_info = MagicMock()
+        mock_collection_info.config.params.vectors = MagicMock(spec=rest.VectorsConfig)
+
+        with patch.object(document_store.client, "collection_exists", return_value=True), patch.object(
+            document_store.client, "get_collection", return_value=mock_collection_info
+        ):
+
+            with pytest.raises(QdrantStoreError, match="without sparse embedding vectors"):
+                document_store._set_up_collection("test_collection", 768, False, "cosine", True, False)
+
+    def test_set_up_collection_use_sparse_embeddings_false_with_named_vectors(self):
+        """Test that an error is raised when use_sparse_embeddings is False but collection has named vectors"""
+        document_store = QdrantDocumentStore(location=":memory:", use_sparse_embeddings=False)
+
+        # Mock collection info with named vectors
+        mock_collection_info = MagicMock()
+        mock_collection_info.config.params.vectors = {DENSE_VECTORS_NAME: MagicMock()}
+
+        with patch.object(document_store.client, "collection_exists", return_value=True), patch.object(
+            document_store.client, "get_collection", return_value=mock_collection_info
+        ):
+
+            with pytest.raises(QdrantStoreError, match="with sparse embedding vectors"):
+                document_store._set_up_collection("test_collection", 768, False, "cosine", False, False)
+
+    def test_set_up_collection_with_distance_mismatch(self):
+        document_store = QdrantDocumentStore(location=":memory:", use_sparse_embeddings=False, similarity="cosine")
+
+        # Mock collection info with different distance
+        mock_collection_info = MagicMock()
+        mock_collection_info.config.params.vectors = MagicMock()
+        mock_collection_info.config.params.vectors.distance = rest.Distance.DOT
+        mock_collection_info.config.params.vectors.size = 768
+
+        with patch.object(document_store.client, "collection_exists", return_value=True), patch.object(
+            document_store.client, "get_collection", return_value=mock_collection_info
+        ):
+
+            with pytest.raises(ValueError, match="different similarity"):
+                document_store._set_up_collection("test_collection", 768, False, "cosine", False, False)
+
+    def test_set_up_collection_with_dimension_mismatch(self):
+        document_store = QdrantDocumentStore(location=":memory:", use_sparse_embeddings=False, similarity="cosine")
+
+        # Mock collection info with different vector size
+        mock_collection_info = MagicMock()
+        mock_collection_info.config.params.vectors = MagicMock()
+        mock_collection_info.config.params.vectors.distance = rest.Distance.COSINE
+        mock_collection_info.config.params.vectors.size = 512
+
+        with patch.object(document_store.client, "collection_exists", return_value=True), patch.object(
+            document_store.client, "get_collection", return_value=mock_collection_info
+        ):
+
+            with pytest.raises(ValueError, match="different vector size"):
+                document_store._set_up_collection("test_collection", 768, False, "cosine", False, False)