microsoft-sharepoint-haystack 1.1.0__tar.gz → 1.2.0__tar.gz

{microsoft_sharepoint_haystack-1.1.0 → microsoft_sharepoint_haystack-1.2.0}/CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## [integrations/microsoft_sharepoint-v1.1.0] - 2026-06-25
+
+### 🚀 Features
+
+- *(microsoft_sharepoint)* Add MSSharePointFetcher (#3485)
+
+
 ## [integrations/microsoft_sharepoint-v1.0.0] - 2026-06-22
 
 ### 🚀 Features

{microsoft_sharepoint_haystack-1.1.0 → microsoft_sharepoint_haystack-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: microsoft-sharepoint-haystack
-Version: 1.1.0
+Version: 1.2.0
 Summary: Haystack integration for microsoft_sharepoint
 Project-URL: Documentation, https://github.com/deepset-ai/haystack-core-integrations/tree/main/integrations/microsoft_sharepoint#readme
 Project-URL: Issues, https://github.com/deepset-ai/haystack-core-integrations/issues
@@ -30,15 +30,20 @@ Description-Content-Type: text/markdown
 [![PyPI - Version](https://img.shields.io/pypi/v/microsoft-sharepoint-haystack.svg)](https://pypi.org/project/microsoft-sharepoint-haystack)
 [![PyPI - Python Version](https://img.shields.io/pypi/pyversions/microsoft-sharepoint-haystack.svg)](https://pypi.org/project/microsoft-sharepoint-haystack)
 
+- [Integration page](https://haystack.deepset.ai/integrations/microsoft-sharepoint)
 - [Changelog](https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/microsoft_sharepoint/CHANGELOG.md)
 
-## Running the tests
+---
 
-```bash
-hatch run test:unit          # offline, no credentials needed
-hatch run test:integration   # hits live Microsoft Graph (see below)
-```
+## Contributing
 
-The integration tests (the `TestLive` classes in `tests/`) exercise the retriever and the fetcher
-against live Microsoft Graph. They **skip themselves** when their inputs are not set, so the default
-local and CI runs stay offline. To run them you need a delegated Microsoft Graph access token.
+Refer to the general [Contribution Guidelines](https://github.com/deepset-ai/haystack-core-integrations/blob/main/CONTRIBUTING.md).
+
+The integration tests run against live Microsoft Graph and are skipped unless their
+environment variables are set:
+
+- `MS_SHAREPOINT_ACCESS_TOKEN` — a delegated Microsoft Graph access token; required by the
+  retriever and fetcher tests.
+- `MS_SHAREPOINT_TEST_FILE_URL` — a SharePoint file URL; also required by the fetcher tests.
+- `MS_GRAPH_TENANT_ID`, `MS_GRAPH_CLIENT_ID` and `MS_GRAPH_CLIENT_SECRET` — required by the
+  app-only retriever tests.

microsoft_sharepoint_haystack-1.2.0/README.md

@@ -0,0 +1,22 @@
+# microsoft-sharepoint-haystack
+
+[![PyPI - Version](https://img.shields.io/pypi/v/microsoft-sharepoint-haystack.svg)](https://pypi.org/project/microsoft-sharepoint-haystack)
+[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/microsoft-sharepoint-haystack.svg)](https://pypi.org/project/microsoft-sharepoint-haystack)
+
+- [Integration page](https://haystack.deepset.ai/integrations/microsoft-sharepoint)
+- [Changelog](https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/microsoft_sharepoint/CHANGELOG.md)
+
+---
+
+## Contributing
+
+Refer to the general [Contribution Guidelines](https://github.com/deepset-ai/haystack-core-integrations/blob/main/CONTRIBUTING.md).
+
+The integration tests run against live Microsoft Graph and are skipped unless their
+environment variables are set:
+
+- `MS_SHAREPOINT_ACCESS_TOKEN` — a delegated Microsoft Graph access token; required by the
+  retriever and fetcher tests.
+- `MS_SHAREPOINT_TEST_FILE_URL` — a SharePoint file URL; also required by the fetcher tests.
+- `MS_GRAPH_TENANT_ID`, `MS_GRAPH_CLIENT_ID` and `MS_GRAPH_CLIENT_SECRET` — required by the
+  app-only retriever tests.

{microsoft_sharepoint_haystack-1.1.0 → microsoft_sharepoint_haystack-1.2.0}/pyproject.toml

@@ -133,6 +133,7 @@ ignore = [
     "PLR0912",
     "PLR0913",
     "PLR0915",
+    "PLR0917",
     # Allow `Any` type - used legitimately for dynamic types and SDK boundaries
     "ANN401",
 ]

{microsoft_sharepoint_haystack-1.1.0 → microsoft_sharepoint_haystack-1.2.0}/src/haystack_integrations/common/microsoft_sharepoint/utils.py

@@ -2,6 +2,9 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
+import asyncio
+from typing import TypeVar
+
 import httpx
 from haystack import logging
 from haystack.utils import Secret
@@ -18,12 +21,31 @@ from .errors import SharePointConfigError, SharePointRequestError
 
 logger = logging.getLogger(__name__)
 
+T = TypeVar("T")
+
 DEFAULT_GRAPH_URL = "https://graph.microsoft.com/v1.0"
 _RETRYABLE_STATUS = frozenset({429, 500, 502, 503, 504})
 # Fallback backoff when a throttled response carries no `Retry-After` header: 1s, 2s, 4s, ...
 _EXPONENTIAL_BACKOFF = wait_exponential()
 
 
+async def _gather_tasks_with_cancel(tasks: list[asyncio.Task[T]]) -> list[T]:
+    """
+    Wait for all tasks, cancelling and draining unfinished siblings if one fails.
+
+    :param tasks: Tasks to wait for.
+    :returns:
+        The task results in input order.
+    """
+    try:
+        return await asyncio.gather(*tasks)
+    except BaseException:
+        for task in tasks:
+            task.cancel()
+        await asyncio.gather(*tasks, return_exceptions=True)
+        raise
+
+
 def resolve_access_token(access_token: str | Secret) -> str:
     """Return the bearer token string, resolving it if a `Secret` was passed."""
     if not isinstance(access_token, Secret):

{microsoft_sharepoint_haystack-1.1.0 → microsoft_sharepoint_haystack-1.2.0}/src/haystack_integrations/components/fetchers/microsoft_sharepoint/fetcher.py

@@ -18,6 +18,7 @@ from haystack.utils import Secret
 from haystack_integrations.common.microsoft_sharepoint.errors import SharePointConfigError, SharePointRequestError
 from haystack_integrations.common.microsoft_sharepoint.utils import (
     DEFAULT_GRAPH_URL,
+    _gather_tasks_with_cancel,
     build_async_retrying,
     build_retrying,
     resolve_access_token,
@@ -193,7 +194,8 @@ class MSSharePointFetcher:
                 return await self._process_async(client, token, url, hints)
 
         async with httpx.AsyncClient(timeout=self.timeout, follow_redirects=True) as client:
-            results = await asyncio.gather(*(fetch_one(client, url, hints) for url, hints in resolved))
+            tasks = [asyncio.create_task(fetch_one(client, url, hints)) for url, hints in resolved]
+            results = await _gather_tasks_with_cancel(tasks)
 
         return {"streams": [stream for stream in results if stream is not None]}
 

{microsoft_sharepoint_haystack-1.1.0 → microsoft_sharepoint_haystack-1.2.0}/src/haystack_integrations/components/retrievers/microsoft_sharepoint/retriever.py

@@ -44,10 +44,10 @@ class MSSharePointRetriever:
     download or convert the underlying files. Compose a downstream fetcher/converter (such as
     `MSSharePointFetcher`) when full content is needed.
 
-    The retriever takes a per-user `access_token` as a run input, typically wired
-    from an upstream `OAuthResolver`. The token must carry delegated Microsoft Graph permissions
-    (for example `Files.Read.All` and, for site/list scoping, `Sites.Read.All`). The Search API supports
-    delegated permissions only.
+    The retriever takes an `access_token` as a run input. For delegated (on-behalf-of) auth, pass a
+    per-user token wired from an upstream `OAuthResolver`; the token must carry delegated Microsoft Graph
+    permissions (for example `Files.Read.All` and, for site/list scoping, `Sites.Read.All`). For app-only
+    (client-credentials) auth, pass an application token and set the `region` parameter at init time.
 
     ### Usage example
     ```python
@@ -76,6 +76,7 @@ class MSSharePointRetriever:
         top_k: int = 10,
         fields: list[str] | None = None,
         query_template: str | None = None,
+        region: str | None = None,
         graph_url: str = DEFAULT_GRAPH_URL,
         timeout: float = 30.0,
         max_retries: int = 3,
@@ -96,6 +97,10 @@ class MSSharePointRetriever:
             `'{searchTerms} path:"https://contoso.sharepoint.com/sites/Team"'`. The literal `{searchTerms}`
             placeholder is replaced by the run-time query. The template uses
             [Keyword Query Language (KQL)](https://learn.microsoft.com/en-us/sharepoint/dev/general-development/keyword-query-language-kql-syntax-reference).
+        :param region: The region code for the Microsoft Search index, for example `"NAM"`, `"EUR"`, or `"APC"`.
+            Required when using application permissions (app-only / client-credentials auth); omit for delegated
+            (on-behalf-of) tokens. See the `region` property of the
+            [searchRequest resource](https://learn.microsoft.com/en-us/graph/api/resources/searchrequest).
         :param graph_url: The Microsoft Graph base URL. Defaults to `https://graph.microsoft.com/v1.0`.
             Override for sovereign clouds.
         :param timeout: The HTTP timeout in seconds for each request to Microsoft Graph.
@@ -121,6 +126,7 @@ class MSSharePointRetriever:
         self.top_k = top_k
         self.fields = fields
         self.query_template = query_template
+        self.region = region
         self.graph_url = graph_url.rstrip("/")
         self.timeout = timeout
         self.max_retries = max_retries
@@ -134,9 +140,9 @@ class MSSharePointRetriever:
             operators directly in the query, for example `filetype:docx`, `author:"Jane Doe"`, or
             `path:"https://contoso.sharepoint.com/sites/Team"`. See the
             [KQL syntax reference](https://learn.microsoft.com/en-us/sharepoint/dev/general-development/keyword-query-language-kql-syntax-reference).
-        :param access_token: A delegated Microsoft Graph bearer token for the user whose content is searched,
-            typically wired from an upstream `OAuthResolver` (which emits a plain `str`). A `Secret` is also
-            accepted and resolved internally.
+        :param access_token: A Microsoft Graph bearer token. For delegated auth, pass a per-user token wired from
+            an upstream `OAuthResolver`. For app-only (client-credentials) auth, pass an application token and set
+            `region` at init time. A `Secret` is also accepted and resolved internally.
         :param top_k: Overrides the `top_k` configured at initialization for this run.
         :returns: A dictionary with a `documents` key holding the list of retrieved `Document` objects.
         :raises SharePointConfigError: If `access_token` is a `Secret` that does not resolve to a string.
@@ -171,9 +177,9 @@ class MSSharePointRetriever:
             operators directly in the query, for example `filetype:docx`, `author:"Jane Doe"`, or
             `path:"https://contoso.sharepoint.com/sites/Team"`. See the
             [KQL syntax reference](https://learn.microsoft.com/en-us/sharepoint/dev/general-development/keyword-query-language-kql-syntax-reference).
-        :param access_token: A delegated Microsoft Graph bearer token for the user whose content is searched,
-            typically wired from an upstream `OAuthResolver` (which emits a plain `str`). A `Secret` is also
-            accepted and resolved internally.
+        :param access_token: A Microsoft Graph bearer token. For delegated auth, pass a per-user token wired from
+            an upstream `OAuthResolver`. For app-only (client-credentials) auth, pass an application token and set
+            `region` at init time. A `Secret` is also accepted and resolved internally.
         :param top_k: Overrides the `top_k` configured at initialization for this run.
         :returns: A dictionary with a `documents` key holding the list of retrieved `Document` objects.
         :raises SharePointConfigError: If `access_token` is a `Secret` that does not resolve to a string.
@@ -215,6 +221,8 @@ class MSSharePointRetriever:
         }
         if self.fields:
             request["fields"] = self.fields
+        if self.region:
+            request["region"] = self.region
 
         return {"requests": [request]}
 
@@ -328,6 +336,7 @@ class MSSharePointRetriever:
             top_k=self.top_k,
             fields=self.fields,
             query_template=self.query_template,
+            region=self.region,
             graph_url=self.graph_url,
             timeout=self.timeout,
             max_retries=self.max_retries,

{microsoft_sharepoint_haystack-1.1.0 → microsoft_sharepoint_haystack-1.2.0}/tests/test_fetcher.py

@@ -1,6 +1,7 @@
 # SPDX-FileCopyrightText: 2022-present deepset GmbH <info@deepset.ai>
 #
 # SPDX-License-Identifier: Apache-2.0
+import asyncio
 import base64
 import json
 import os
@@ -458,6 +459,60 @@ class TestPipeline:
 
 @pytest.mark.asyncio
 class TestRunAsync:
+    async def test_failure_cancels_and_drains_siblings(self):
+        fetcher = MSSharePointFetcher(max_retries=0)
+        slow_started = asyncio.Event()
+        slow_cancelled = False
+        current_task = asyncio.current_task()
+        tasks_before = asyncio.all_tasks() - {current_task}
+        slow_url = "https://host/slow.docx"
+        slow_marker = _encode_share_url(slow_url)
+
+        async def get(url, *_args, **_kwargs):
+            nonlocal slow_cancelled
+            if slow_marker in url:
+                slow_started.set()
+                try:
+                    await asyncio.Event().wait()
+                except asyncio.CancelledError:
+                    slow_cancelled = True
+                    raise
+            await slow_started.wait()
+            msg = "failed"
+            raise httpx.ReadError(msg)
+
+        targets = [
+            _drive_item_document(url=slow_url),
+            _drive_item_document(url="https://host/failing.docx"),
+        ]
+        with patch.object(httpx.AsyncClient, "get", AsyncMock(side_effect=get)):
+            with pytest.raises(httpx.ReadError):
+                await fetcher.run_async(access_token="tok", targets=targets)
+
+        assert slow_cancelled is True
+        assert asyncio.all_tasks() - {current_task} == tasks_before
+
+    async def test_raise_on_failure_false_skips_failed_items(self):
+        fetcher = MSSharePointFetcher(max_retries=0, raise_on_failure=False)
+        missing_url = "https://host/missing.docx"
+        missing_marker = _encode_share_url(missing_url)
+
+        async def get(url, *_args, **_kwargs):
+            if missing_marker in url:
+                msg = "failed"
+                raise httpx.ReadError(msg)
+            return _binary_response(content=b"ok")
+
+        targets = [
+            _drive_item_document(url=missing_url),
+            _drive_item_document(url="https://host/healthy.docx"),
+        ]
+        with patch.object(httpx.AsyncClient, "get", AsyncMock(side_effect=get)):
+            streams = (await fetcher.run_async(access_token="tok", targets=targets))["streams"]
+
+        assert len(streams) == 1
+        assert streams[0].data == b"ok"
+
     async def test_downloads_drive_item(self):
         fetcher = MSSharePointFetcher()
         get = AsyncMock(return_value=_binary_response(content=b"async-bytes"))

{microsoft_sharepoint_haystack-1.1.0 → microsoft_sharepoint_haystack-1.2.0}/tests/test_retriever.py

@@ -7,6 +7,7 @@ from unittest.mock import AsyncMock, patch
 
 import httpx
 import pytest
+import requests
 from haystack import Document, Pipeline
 from haystack.utils import Secret
 
@@ -100,6 +101,7 @@ class TestInit:
         assert retriever.top_k == 10
         assert retriever.fields is None
         assert retriever.query_template is None
+        assert retriever.region is None
         assert retriever.graph_url == "https://graph.microsoft.com/v1.0"
         assert retriever.timeout == 30.0
         assert retriever.max_retries == 3
@@ -140,12 +142,18 @@ class TestSerialization:
                 "top_k": 5,
                 "fields": ["title"],
                 "query_template": '{searchTerms} path:"https://x"',
+                "region": None,
                 "graph_url": "https://graph.microsoft.us/v1.0",
                 "timeout": 10.0,
                 "max_retries": 1,
             },
         }
 
+    def test_to_dict_with_region(self):
+        retriever = MSSharePointRetriever(region="NAM")
+        data = retriever.to_dict()
+        assert data["init_parameters"]["region"] == "NAM"
+
     def test_from_dict_round_trip(self):
         retriever = MSSharePointRetriever(entity_types=["site"], top_k=7, query_template="{searchTerms}")
         restored = MSSharePointRetriever.from_dict(retriever.to_dict())
@@ -241,6 +249,19 @@ class TestRun:
         assert "queryTemplate" not in mock_post.call_args.kwargs["json"]["requests"][0]["query"]
         assert "fields" not in mock_post.call_args.kwargs["json"]["requests"][0]
 
+    def test_region_included_in_request_body_when_set(self):
+        retriever = MSSharePointRetriever(region="NAM")
+        with patch.object(httpx.Client, "post", return_value=_make_response(json_body=EMPTY_RESPONSE)) as mock_post:
+            retriever.run(query="q", access_token="tok")
+        request = mock_post.call_args.kwargs["json"]["requests"][0]
+        assert request["region"] == "NAM"
+
+    def test_region_omitted_from_request_body_when_none(self):
+        retriever = MSSharePointRetriever()
+        with patch.object(httpx.Client, "post", return_value=_make_response(json_body=EMPTY_RESPONSE)) as mock_post:
+            retriever.run(query="q", access_token="tok")
+        assert "region" not in mock_post.call_args.kwargs["json"]["requests"][0]
+
     def test_empty_results(self):
         retriever = MSSharePointRetriever()
         with patch.object(httpx.Client, "post", return_value=_make_response(json_body=EMPTY_RESPONSE)):
@@ -410,3 +431,53 @@ class TestLive:
         for doc in documents:
             assert isinstance(doc, Document)
             assert "web_url" in doc.meta
+
+
+@pytest.mark.integration
+@pytest.mark.skipif(
+    not all(os.environ.get(v) for v in ("MS_GRAPH_TENANT_ID", "MS_GRAPH_CLIENT_ID", "MS_GRAPH_CLIENT_SECRET")),
+    reason="MS_GRAPH_TENANT_ID / MS_GRAPH_CLIENT_ID / MS_GRAPH_CLIENT_SECRET not set",
+)
+class TestLiveAppOnly:
+    """
+    End-to-end tests using app-only (client credentials) authentication.
+
+    Set these env vars before running:
+        MS_GRAPH_TENANT_ID     — Azure AD tenant ID
+        MS_GRAPH_CLIENT_ID     — App registration client ID
+        MS_GRAPH_CLIENT_SECRET — App registration client secret
+        MS_GRAPH_REGION        — Search region, e.g. "NAM" (default: "DEU")
+
+    Run with:
+        hatch run test:integration
+    """
+
+    @staticmethod
+    def _get_app_token() -> str:
+        tenant_id = os.environ["MS_GRAPH_TENANT_ID"]
+        client_id = os.environ["MS_GRAPH_CLIENT_ID"]
+        client_secret = os.environ["MS_GRAPH_CLIENT_SECRET"]
+        response = requests.post(
+            f"https://login.microsoftonline.com/{tenant_id}/oauth2/v2.0/token",
+            data={
+                "client_id": client_id,
+                "client_secret": client_secret,
+                "scope": "https://graph.microsoft.com/.default",
+                "grant_type": "client_credentials",
+            },
+            timeout=30,
+        )
+        response.raise_for_status()
+        return response.json()["access_token"]
+
+    def test_app_only_search_with_region_succeeds(self):
+        token = self._get_app_token()
+        region = os.environ.get("MS_GRAPH_REGION", "DEU")
+
+        # With region, the request should succeed.
+        retriever = MSSharePointRetriever(top_k=3, region=region, max_retries=0)
+        documents = retriever.run(query="test", access_token=token)["documents"]
+        assert isinstance(documents, list)
+        for doc in documents:
+            assert isinstance(doc, Document)
+            assert "web_url" in doc.meta

microsoft_sharepoint_haystack-1.1.0/README.md

@@ -1,17 +0,0 @@
-# microsoft-sharepoint-haystack
-
-[![PyPI - Version](https://img.shields.io/pypi/v/microsoft-sharepoint-haystack.svg)](https://pypi.org/project/microsoft-sharepoint-haystack)
-[![PyPI - Python Version](https://img.shields.io/pypi/pyversions/microsoft-sharepoint-haystack.svg)](https://pypi.org/project/microsoft-sharepoint-haystack)
-
-- [Changelog](https://github.com/deepset-ai/haystack-core-integrations/blob/main/integrations/microsoft_sharepoint/CHANGELOG.md)
-
-## Running the tests
-
-```bash
-hatch run test:unit          # offline, no credentials needed
-hatch run test:integration   # hits live Microsoft Graph (see below)
-```
-
-The integration tests (the `TestLive` classes in `tests/`) exercise the retriever and the fetcher
-against live Microsoft Graph. They **skip themselves** when their inputs are not set, so the default
-local and CI runs stay offline. To run them you need a delegated Microsoft Graph access token.