opensolr-haystack 0.7.0__tar.gz → 0.8.0__tar.gz

{opensolr_haystack-0.7.0 → opensolr_haystack-0.8.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: opensolr-haystack
-Version: 0.7.0
+Version: 0.8.0
 Summary: Haystack integration for Opensolr — managed Apache Solr DocumentStore with server-side embeddings and hybrid BM25+kNN retrieval
 Author-email: Opensolr <support@opensolr.com>
 License: MIT
@@ -61,8 +61,8 @@ the query. The store embeds server-side at both index and query time.
 
 ## Try it without an account
 
-There is a public demo account. Point the package at it and search, ingestion and
-answers work immediately, with no signup:
+There is a public demo account. Point the package at it and everything in this
+README works immediately, with no signup:
 
 ```bash
 export OPENSOLR_EMAIL=mcp@opensolr.com
@@ -70,17 +70,18 @@ export OPENSOLR_API_KEY=420b8b23e7b12dc8ab838932145a5065
 ```
 
 `mcp_demo_d1__dense` is already loaded with 300 news articles, so search, filtering
-and grounded answers work the moment you connect. You also get the full write path:
-create your own index on the account, ingest into it, query it, delete it.
-Reading files and pictures (the file loader and file indexing below, search by image)
-needs your own key: the shared demo key does not include them.
+and grounded answers work the moment you connect. You also get the write path:
+create your own index on the account, ingest into it and query it. Deletion is not
+available on this shared key: no index here can be deleted or reconfigured by hand.
+Whatever you create is removed automatically after 3 days.
 
 Know what you are working with:
 
 - **Anything you create there is deleted after 3 days.** Automatically, without warning
   or export. That includes indexes you created and every document in them.
-- **The account is shared with everyone reading this.** Your index is visible to them,
-  they can change or delete it, and you can do the same to theirs. Never put anything
+- **The account is shared with everyone reading this.** Your index is visible to them
+  and they can add documents to it, as you can to theirs. Nobody can delete or
+  reconfigure an index here (that is switched off for this key), but never put anything
   real, private or client-owned in it.
 - **The limits are per index, and deliberately small.** 200 MB of bandwidth and 50 MB
   of disk per index. Bandwidth is the one you will hit first: it covers a demo, a
@@ -172,8 +173,9 @@ text from PDF/DOCX/XLSX for you.
 ## Lexical-only mode
 
 Don't need vectors? Pure keyword search skips the embedding call entirely —
-zero AI quota, and it works on **any** Opensolr index, including non-vector
-ones and older Solr versions.
+zero AI quota. It runs on any Opensolr index that has `title`, `description` and
+`text` fields, including indexes outside the vector regions and older Solr
+versions (see [Index requirements](#index-requirements)).
 
 ## Search by image
 
@@ -241,7 +243,7 @@ Data Ingestion with `rtf`, PDFs found by the Web Crawler), a search returns each
 **once**, as its best page. Its metadata carries `page_number` (the page number),
 `page_url` (opens the PDF at that page, `...pdf#page=14`), `document_id`,
 `more_pages` (its other matching pages, each with `page` and `page_url`) and
-`matching_pages`; every hit carries its `uri`. `group_pages=False` returns the pages
+`matching_pages`, and its `uri`; any other hit keeps exactly the metadata it was sent with. `group_pages=False` returns the pages
 as separate results.
 
 ## Searching a period
@@ -257,6 +259,74 @@ OpensolrHybridRetriever(document_store=store, date_from="2026-01-01")
 
 They combine with every other option — filters, `fresh_bias`, hybrid or lexical.
 
+## Index requirements
+
+Everything this package does runs on an Opensolr index that sits in a **vector
+region** and carries the **Opensolr vector schema** — the configuration every
+Opensolr vector index runs. An index created through the package (`create_index`,
+`create_if_missing`) has both. An index made elsewhere may not: a server outside
+the vector regions, an older Solr version, or a schema of its own.
+
+**The check is automatic.** The first time the package touches an index, it asks
+the platform once whether that index can do what is asked: one request per index
+per client, kept for the client's life, never one per search. If the check itself
+cannot run (a network error, the platform busy), nothing is blocked: the
+operation runs exactly as it always did, and the check is tried again after 5
+minutes. When an index differs from the Opensolr vector schema, a warning is logged
+once, on the `opensolr` logger, with the reasons and the fix.
+
+What the check gates — only what cannot work otherwise:
+
+| Capability | Needed for | When it is missing |
+|---|---|---|
+| `ingestion` | Data Ingestion: adding documents and indexing files | The index is outside the vector regions: refused before a document or a byte is sent |
+| `vector_search` | Vector and hybrid search | The index has no vector field of the right size: refused before any embedding is paid for |
+| `pdf_pages` | Grouping the pages of a PDF in search results | Results come back ungrouped, as before |
+| `keyword_search` | Keyword (lexical) search | Never blocked |
+
+**The errors say exactly what is going on.** A blocked operation raises
+`OpensolrSchemaError` (a subclass of `OpensolrError`) with `.index`, `.capability`
+and `.report` (the full check), and a message with every reason and the fix:
+
+```
+Index 'shop_it' cannot take documents or files through Data Ingestion. Index shop_it
+is on IT-SOLR-9-6 (Milano, Italy), which is not a vector region: Data Ingestion,
+server-side embeddings and hybrid search run only on the servers of a vector region
+(DE-SOLR-9 (Germany, Solr 9.00), FINLAND9 (Finland, Solr 9.60), ...). This index
+cannot be made ready: create a new index in one of the vector regions (...), for
+example with create_index, and send your documents there.
+```
+
+When Solr itself refuses a request, `OpensolrSolrError` carries Solr's own reason
+and the index name; it is still an `httpx.HTTPStatusError`, so code that caught
+that before keeps working.
+
+```python
+from haystack_integrations.document_stores.opensolr import OpensolrDocumentStore, OpensolrSchemaError, OpensolrSolrError
+
+try:
+    store.write_files("./contracts")
+except OpensolrSchemaError as e:
+    print(e.capability, e.report["fix"]["action"])   # ingestion new_index
+```
+
+**See the details, and fix what can be fixed:**
+
+```python
+store.check_schema()   # ready, capabilities, server, Solr version, missing / different fields, problems, fix
+store.apply_schema(force=False)   # installs the Opensolr vector schema; force=True when the index holds documents
+```
+
+`apply_schema` exists so an index of a vector region whose configuration was
+changed, or never set, can take everything this package does without being
+recreated: it installs the same configuration every Opensolr vector index runs,
+and **replaces the index configuration**. An index that already holds documents
+needs `force=True`; those documents keep the fields they were written with and
+must be sent again. An index outside the vector regions cannot take it — create
+a new index in a region from `vector_regions` and send your documents there. The
+package never installs the schema on its own.
+
+
 ## Your index schema
 
 Documents follow the Opensolr document model (`title`, `description`, `text`,

{opensolr_haystack-0.7.0 → opensolr_haystack-0.8.0}/README.md

@@ -43,8 +43,8 @@ the query. The store embeds server-side at both index and query time.
 
 ## Try it without an account
 
-There is a public demo account. Point the package at it and search, ingestion and
-answers work immediately, with no signup:
+There is a public demo account. Point the package at it and everything in this
+README works immediately, with no signup:
 
 ```bash
 export OPENSOLR_EMAIL=mcp@opensolr.com
@@ -52,17 +52,18 @@ export OPENSOLR_API_KEY=420b8b23e7b12dc8ab838932145a5065
 ```
 
 `mcp_demo_d1__dense` is already loaded with 300 news articles, so search, filtering
-and grounded answers work the moment you connect. You also get the full write path:
-create your own index on the account, ingest into it, query it, delete it.
-Reading files and pictures (the file loader and file indexing below, search by image)
-needs your own key: the shared demo key does not include them.
+and grounded answers work the moment you connect. You also get the write path:
+create your own index on the account, ingest into it and query it. Deletion is not
+available on this shared key: no index here can be deleted or reconfigured by hand.
+Whatever you create is removed automatically after 3 days.
 
 Know what you are working with:
 
 - **Anything you create there is deleted after 3 days.** Automatically, without warning
   or export. That includes indexes you created and every document in them.
-- **The account is shared with everyone reading this.** Your index is visible to them,
-  they can change or delete it, and you can do the same to theirs. Never put anything
+- **The account is shared with everyone reading this.** Your index is visible to them
+  and they can add documents to it, as you can to theirs. Nobody can delete or
+  reconfigure an index here (that is switched off for this key), but never put anything
   real, private or client-owned in it.
 - **The limits are per index, and deliberately small.** 200 MB of bandwidth and 50 MB
   of disk per index. Bandwidth is the one you will hit first: it covers a demo, a
@@ -154,8 +155,9 @@ text from PDF/DOCX/XLSX for you.
 ## Lexical-only mode
 
 Don't need vectors? Pure keyword search skips the embedding call entirely —
-zero AI quota, and it works on **any** Opensolr index, including non-vector
-ones and older Solr versions.
+zero AI quota. It runs on any Opensolr index that has `title`, `description` and
+`text` fields, including indexes outside the vector regions and older Solr
+versions (see [Index requirements](#index-requirements)).
 
 ## Search by image
 
@@ -223,7 +225,7 @@ Data Ingestion with `rtf`, PDFs found by the Web Crawler), a search returns each
 **once**, as its best page. Its metadata carries `page_number` (the page number),
 `page_url` (opens the PDF at that page, `...pdf#page=14`), `document_id`,
 `more_pages` (its other matching pages, each with `page` and `page_url`) and
-`matching_pages`; every hit carries its `uri`. `group_pages=False` returns the pages
+`matching_pages`, and its `uri`; any other hit keeps exactly the metadata it was sent with. `group_pages=False` returns the pages
 as separate results.
 
 ## Searching a period
@@ -239,6 +241,74 @@ OpensolrHybridRetriever(document_store=store, date_from="2026-01-01")
 
 They combine with every other option — filters, `fresh_bias`, hybrid or lexical.
 
+## Index requirements
+
+Everything this package does runs on an Opensolr index that sits in a **vector
+region** and carries the **Opensolr vector schema** — the configuration every
+Opensolr vector index runs. An index created through the package (`create_index`,
+`create_if_missing`) has both. An index made elsewhere may not: a server outside
+the vector regions, an older Solr version, or a schema of its own.
+
+**The check is automatic.** The first time the package touches an index, it asks
+the platform once whether that index can do what is asked: one request per index
+per client, kept for the client's life, never one per search. If the check itself
+cannot run (a network error, the platform busy), nothing is blocked: the
+operation runs exactly as it always did, and the check is tried again after 5
+minutes. When an index differs from the Opensolr vector schema, a warning is logged
+once, on the `opensolr` logger, with the reasons and the fix.
+
+What the check gates — only what cannot work otherwise:
+
+| Capability | Needed for | When it is missing |
+|---|---|---|
+| `ingestion` | Data Ingestion: adding documents and indexing files | The index is outside the vector regions: refused before a document or a byte is sent |
+| `vector_search` | Vector and hybrid search | The index has no vector field of the right size: refused before any embedding is paid for |
+| `pdf_pages` | Grouping the pages of a PDF in search results | Results come back ungrouped, as before |
+| `keyword_search` | Keyword (lexical) search | Never blocked |
+
+**The errors say exactly what is going on.** A blocked operation raises
+`OpensolrSchemaError` (a subclass of `OpensolrError`) with `.index`, `.capability`
+and `.report` (the full check), and a message with every reason and the fix:
+
+```
+Index 'shop_it' cannot take documents or files through Data Ingestion. Index shop_it
+is on IT-SOLR-9-6 (Milano, Italy), which is not a vector region: Data Ingestion,
+server-side embeddings and hybrid search run only on the servers of a vector region
+(DE-SOLR-9 (Germany, Solr 9.00), FINLAND9 (Finland, Solr 9.60), ...). This index
+cannot be made ready: create a new index in one of the vector regions (...), for
+example with create_index, and send your documents there.
+```
+
+When Solr itself refuses a request, `OpensolrSolrError` carries Solr's own reason
+and the index name; it is still an `httpx.HTTPStatusError`, so code that caught
+that before keeps working.
+
+```python
+from haystack_integrations.document_stores.opensolr import OpensolrDocumentStore, OpensolrSchemaError, OpensolrSolrError
+
+try:
+    store.write_files("./contracts")
+except OpensolrSchemaError as e:
+    print(e.capability, e.report["fix"]["action"])   # ingestion new_index
+```
+
+**See the details, and fix what can be fixed:**
+
+```python
+store.check_schema()   # ready, capabilities, server, Solr version, missing / different fields, problems, fix
+store.apply_schema(force=False)   # installs the Opensolr vector schema; force=True when the index holds documents
+```
+
+`apply_schema` exists so an index of a vector region whose configuration was
+changed, or never set, can take everything this package does without being
+recreated: it installs the same configuration every Opensolr vector index runs,
+and **replaces the index configuration**. An index that already holds documents
+needs `force=True`; those documents keep the fields they were written with and
+must be sent again. An index outside the vector regions cannot take it — create
+a new index in a region from `vector_regions` and send your documents there. The
+package never installs the schema on its own.
+
+
 ## Your index schema
 
 Documents follow the Opensolr document model (`title`, `description`, `text`,

{opensolr_haystack-0.7.0 → opensolr_haystack-0.8.0}/haystack_integrations/document_stores/opensolr/__init__.py

@@ -1,6 +1,7 @@
 """Opensolr DocumentStore for Haystack — managed Apache Solr with server-side
 embeddings and native hybrid (BM25 + kNN) search."""
 
+from haystack_integrations.document_stores.opensolr.client import OpensolrError, OpensolrSchemaError, OpensolrSolrError
 from haystack_integrations.document_stores.opensolr.store import OpensolrDocumentStore
 
-__all__ = ["OpensolrDocumentStore"]
+__all__ = ["OpensolrDocumentStore", "OpensolrError", "OpensolrSchemaError", "OpensolrSolrError"]

{opensolr_haystack-0.7.0 → opensolr_haystack-0.8.0}/haystack_integrations/document_stores/opensolr/client.py

@@ -14,12 +14,15 @@ import re
 import json
 import time
 import hashlib
+import logging
 from pathlib import Path
 from typing import Any, Dict, List, Optional, Sequence, Tuple, Union
 from urllib.parse import quote
 
 import httpx
 
+_log = logging.getLogger("opensolr")
+
 MGMT_BASE = "https://opensolr.com/solr_manager/api"
 AI_BASE = "https://api.opensolr.com/solr_manager/api"
 
@@ -743,6 +746,29 @@ class OpensolrError(RuntimeError):
     """Raised when an Opensolr API call fails."""
 
 
+class OpensolrSolrError(OpensolrError, httpx.HTTPStatusError):
+    """Solr refused a request on an index: the message carries Solr's own reason. Also an
+    ``httpx.HTTPStatusError``, so code that caught that before still catches it."""
+
+    def __init__(self, message: str, response: "httpx.Response") -> None:
+        httpx.HTTPStatusError.__init__(self, message, request=response.request, response=response)
+
+
+class OpensolrSchemaError(OpensolrError):
+    """The index cannot do what was asked: it is outside the vector regions, or its schema
+    lacks what the operation needs. ``report`` is the full check (see ``check_schema``)."""
+
+    def __init__(self, index: str, capability: str, report: Dict[str, Any]) -> None:
+        self.index = index
+        self.capability = capability
+        self.report = report
+        what = OpensolrClient.CAPABILITY_TEXT.get(capability, capability)
+        problems = " ".join(report.get("problems") or []) or "The index check says it cannot."
+        fix = (report.get("fix") or {}).get("text") or ""
+        super().__init__(f"Index {index!r} cannot {what}. {problems} {fix} "
+                         f"(client.check_schema({index!r}) shows the details.)".replace("  ", " "))
+
+
 class OpensolrClient:
     """Authenticated client for Opensolr management + AI endpoints.
 
@@ -758,8 +784,9 @@ class OpensolrClient:
         self.api_key = api_key
         self._http = httpx.Client(timeout=timeout, follow_redirects=True)
         self._core_info_cache: Dict[str, Dict[str, Any]] = {}
-        # indexes whose schema has no PDF page fields: searched and read without them
-        self._flat_indexes: set = set()
+        # the schema check of each index, made once (check_schema); when it could not run
+        self._schema_reports: Dict[str, Dict[str, Any]] = {}
+        self._schema_failed: Dict[str, float] = {}
 
     # ------------------------------------------------------------------ #
     # low level                                                          #
@@ -792,9 +819,21 @@ class OpensolrClient:
         except json.JSONDecodeError as exc:
             raise OpensolrError(f"{label}: non-JSON response: {resp.text[:200]}") from exc
         if isinstance(body, dict) and body.get("status") is False:
-            raise OpensolrError(f"{label}: {body.get('msg', body)}")
+            detail = body.get("detail")
+            raise OpensolrError(f"{label}: {body.get('msg', body)}" + (f": {detail}" if detail else ""))
         return body
 
+    def _solr_answer(self, index: str, what: str, resp: "httpx.Response") -> Dict[str, Any]:
+        """Solr's JSON, or :class:`OpensolrSolrError` with Solr's own reason."""
+        if resp.status_code >= 400:
+            try:
+                why = (resp.json().get("error") or {}).get("msg") or ""
+            except (ValueError, AttributeError):
+                why = re.sub(r"<[^>]+>", " ", resp.text)[:300].strip()
+            raise OpensolrSolrError(
+                f"Solr refused the {what} on index {index} (HTTP {resp.status_code}): {why or resp.reason_phrase}", resp)
+        return resp.json()
+
     def _post_with_retry(self, *, url: str, label: str, **kwargs: Any) -> "httpx.Response":
         """POST, retrying only when the connection failed BEFORE a response was produced.
 
@@ -940,6 +979,7 @@ class OpensolrClient:
         ``wait=True``, polls ``ingest_status`` until the job completes
         (the queue is processed every minute).
         """
+        self.require_capability(index, "ingestion")
         resp = self._post_with_retry(label="ingest", url=
             f"{AI_BASE}/ingest",
             json={**self._auth_params(), "core_name": index, "documents": documents},
@@ -1036,8 +1076,7 @@ class OpensolrClient:
     def solr_select(self, index: str, params: Dict[str, Any]) -> Dict[str, Any]:
         base, auth = self.solr_endpoint(index)
         resp = self._post_with_retry(label="solr_select", url=f"{base}/select", data={"wt": "json", **params}, auth=auth)
-        resp.raise_for_status()
-        return resp.json()
+        return self._solr_answer(index, "search", resp)
 
     def hybrid_search(
         self,
@@ -1073,6 +1112,7 @@ class OpensolrClient:
         if not 0.0 <= alpha_f <= 1.0:
             raise ValueError(f"alpha must be between 0 and 1, got {alpha_f}")
 
+        self.require_capability(index, "vector_search")
         clean = query.replace("{", " ").replace("}", " ").replace('"', " ")
         vector = self.embed(index, query, is_query=True)
         compact = json.dumps(vector, separators=(",", ":"))
@@ -1232,8 +1272,91 @@ class OpensolrClient:
             json=payload,
             auth=auth,
         )
-        resp.raise_for_status()
-        return resp.json()
+        return self._solr_answer(index, "update", resp)
+
+    # ------------------------------------------------------------------ #
+    # the Opensolr vector schema: checked once per index, installed on request #
+    # ------------------------------------------------------------------ #
+
+    #: Seconds before a schema check that could not run is tried again (never on every call).
+    SCHEMA_RETRY_SECONDS = 300.0
+
+    #: What each capability lets a client do, for the error messages.
+    CAPABILITY_TEXT = {
+        "ingestion": "take documents or files through Data Ingestion",
+        "vector_search": "be searched by meaning (vector or hybrid search)",
+        "pdf_pages": "return PDF pages",
+        "keyword_search": "be searched by keywords",
+    }
+
+    def check_schema(self, index: str) -> Dict[str, Any]:
+        """Whether an index is ready for the Opensolr packages, and every reason when it is not.
+
+        One request to the platform, which reads the index's schema and compares it with the
+        Opensolr vector schema. The answer carries ``ready``, ``capabilities``
+        (``ingestion``, ``vector_search``, ``pdf_pages``, ``keyword_search``), the server,
+        the Solr version, whether the server is in a vector region, the fields and types
+        that are missing or different, ``problems`` (sentences) and ``fix`` (``none``,
+        ``apply_schema`` or ``new_index``, with what to do).
+        """
+        return self.mgmt("check_schema", core_name=index)
+
+    def apply_schema(self, index: str, force: bool = False) -> Dict[str, Any]:
+        """Install the Opensolr vector schema on an index of a vector region.
+
+        It replaces the index configuration with the one every Opensolr vector index runs,
+        so Data Ingestion, vector and hybrid search and PDF pages all work on it. An index
+        that already holds documents needs ``force=True``: those documents keep the fields
+        they were written with and must be sent again. An index outside the vector regions
+        cannot take it: the error says where to create one. Returns the new check.
+        """
+        self._schema_reports.pop(index, None)
+        self._schema_failed.pop(index, None)
+        body = self.mgmt("apply_schema", core_name=index, force="1" if force else "0")
+        check = body.get("check") if isinstance(body, dict) else None
+        if isinstance(check, dict) and check.get("status") is True:
+            self._schema_reports[index] = check
+        return body
+
+    def schema_report(self, index: str) -> Optional[Dict[str, Any]]:
+        """The check of an index, made once for this client (a check that could not run is
+        tried again after :data:`SCHEMA_RETRY_SECONDS`, and nothing is concluded from it)."""
+        if index in self._schema_reports:
+            return self._schema_reports[index]
+        failed_at = self._schema_failed.get(index)
+        if failed_at is not None and time.time() - failed_at < self.SCHEMA_RETRY_SECONDS:
+            return None
+        try:
+            report = self.check_schema(index)
+        except OpensolrError as exc:
+            self._schema_failed[index] = time.time()
+            _log.warning("Opensolr: the schema of index %s could not be checked (%s); working as before, without the check", index, exc)
+            return None
+        self._schema_failed.pop(index, None)
+        self._schema_reports[index] = report
+        if not report.get("ready"):
+            fix = (report.get("fix") or {}).get("text") or ""
+            _log.warning("Opensolr: index %s is not on the Opensolr vector schema: %s %s", index,
+                         " ".join(report.get("problems") or []), fix)
+        return report
+
+    def require_capability(self, index: str, capability: str) -> None:
+        """Raise :class:`OpensolrSchemaError` when the index's check says it cannot do this.
+
+        Uses the check made once for this client; when the check could not run, nothing is
+        blocked and the operation runs as it always did.
+        """
+        report = self.schema_report(index)
+        if report is None:
+            return
+        caps = report.get("capabilities") or {}
+        if caps.get(capability) is False:
+            raise OpensolrSchemaError(index, capability, report)
+
+    def has_capability(self, index: str, capability: str) -> bool:
+        """True only when the index's check says it can (an unknown answer is False)."""
+        report = self.schema_report(index)
+        return bool(report and (report.get("capabilities") or {}).get(capability))
 
     # ------------------------------------------------------------------ #
     # PDF pages, documents, pictures                                     #
@@ -1242,22 +1365,16 @@ class OpensolrClient:
     def select_grouped(self, index: str, params: Dict[str, Any], group_pages: bool = True) -> Dict[str, Any]:
         """``solr_select`` with the pages of each PDF grouped under its best page.
 
-        See :func:`apply_page_grouping`. An index whose schema has no page fields is searched
-        as it is, and remembered, so it is asked once.
+        See :func:`apply_page_grouping`. Grouped only when the index's check (made once)
+        says it has the PDF page fields; otherwise searched as it is.
         """
-        if not group_pages or index in self._flat_indexes:
+        if not group_pages or not self.has_capability(index, "pdf_pages"):
             return self.solr_select(index, params)
         grouped = dict(params)
         if isinstance(grouped.get("fq"), list):
             grouped["fq"] = list(grouped["fq"])
         apply_page_grouping(grouped)
-        try:
-            return self.solr_select(index, grouped)
-        except httpx.HTTPStatusError as exc:
-            if exc.response.status_code == 400 and PARENT_FIELD in exc.response.text:
-                self._flat_indexes.add(index)
-                return self.solr_select(index, params)
-            raise
+        return self.solr_select(index, grouped)
 
     def read_document(
         self,
@@ -1285,24 +1402,17 @@ class OpensolrClient:
         last = min(last, first + READ_MAX_PAGES - 1)
         esc = key.replace("\\", "\\\\").replace('"', '\\"')
         fl = "id,uri,title,description,text,content_type,creation_date,meta_ext_id"
-        docs: List[Dict[str, Any]] = []
-        if index not in self._flat_indexes:
-            try:
-                body = self.solr_select(index, {
-                    "q": f'id:"{esc}" OR {PARENT_FIELD}:"{esc}" OR meta_ext_id:"{esc}"',
-                    "fq": f"(*:* -{PAGE_FIELD}:[* TO *]) OR {PAGE_FIELD}:[{first} TO {last}]",
-                    "sort": f"{PAGE_FIELD} asc",
-                    "rows": last - first + 1,
-                    "fl": f"{fl},{PAGE_FIELD},{PARENT_FIELD}",
-                })
-                docs = body.get("response", {}).get("docs", [])
-            except httpx.HTTPStatusError as exc:
-                if exc.response.status_code != 400 or (PAGE_FIELD not in exc.response.text and PARENT_FIELD not in exc.response.text):
-                    raise
-                self._flat_indexes.add(index)
-        if index in self._flat_indexes:
+        if self.has_capability(index, "pdf_pages"):
+            body = self.solr_select(index, {
+                "q": f'id:"{esc}" OR {PARENT_FIELD}:"{esc}" OR meta_ext_id:"{esc}"',
+                "fq": f"(*:* -{PAGE_FIELD}:[* TO *]) OR {PAGE_FIELD}:[{first} TO {last}]",
+                "sort": f"{PAGE_FIELD} asc",
+                "rows": last - first + 1,
+                "fl": f"{fl},{PAGE_FIELD},{PARENT_FIELD}",
+            })
+        else:
             body = self.solr_select(index, {"q": f'id:"{esc}" OR meta_ext_id:"{esc}"', "rows": 1, "fl": fl})
-            docs = body.get("response", {}).get("docs", [])
+        docs: List[Dict[str, Any]] = body.get("response", {}).get("docs", [])
         if not docs:
             raise OpensolrError(f"read_document: nothing in {index} for {document!r}")
 
@@ -1519,6 +1629,8 @@ class OpensolrClient:
         Returns ``{"documents": [...], "pending": [...], "failed": [...], "jobs": [...]}``;
         ``jobs`` are the Data Ingestion job ids (``wait=True`` blocks until they are done).
         """
+        # before a byte is sent: an index that cannot take Data Ingestion says so now, not after the uploads
+        self.require_capability(index, "ingestion")
         meta = {str(k): v for k, v in (metadata or {}).items() if isinstance(v, (str, int, float, bool))}
         root = Path(path).expanduser()
         files = list_files(root, pattern, recursive)

{opensolr_haystack-0.7.0 → opensolr_haystack-0.8.0}/haystack_integrations/document_stores/opensolr/store.py

@@ -217,15 +217,16 @@ class OpensolrDocumentStore:
         score = solr_doc.get("score")
         ext = _flat(solr_doc.get("meta_ext_id"))
         doc_id = str(ext) if ext else str(_flat(solr_doc.get("id", "")))
-        # Where the hit lives: uri; a page of a PDF adds page_number, page_url, document_id, more_pages
+        # A page of a PDF also says where it lives (uri, page_number, page_url, document_id, more_pages);
+        # any other document keeps exactly the meta it was sent with
         where = page_info(solr_doc, expanded)
         if where.get("page"):
             where["page_number"] = where.pop("page")
             # every page of a PDF carries the PDF's meta_ext_id: the page number keeps ids apart
             doc_id = f"{doc_id}#page={where['page_number']}"
-        for key, value in where.items():
-            if value not in ("", None):
-                meta.setdefault(key, value)
+            for key, value in where.items():
+                if value not in ("", None):
+                    meta.setdefault(key, value)
         return Document(
             id=doc_id,
             content=str(content),
@@ -290,6 +291,26 @@ class OpensolrDocumentStore:
             self.client.ingest(self.index, docs[i : i + 50], wait=self.ingest_wait)
         return len(docs)
 
+    def check_schema(self) -> Dict[str, Any]:
+        """Whether this index is ready for every feature of the store, and exactly why not.
+
+        Returns the platform's check: ``ready``, ``capabilities`` (``ingestion``,
+        ``vector_search``, ``pdf_pages``, ``keyword_search``), the server, its Solr
+        version, the fields and types that differ from the Opensolr vector schema,
+        ``problems`` and ``fix``. The store runs this check once on its own.
+        """
+        self._ensure_index()
+        return self.client.check_schema(self.index)
+
+    def apply_schema(self, force: bool = False) -> Dict[str, Any]:
+        """Install the Opensolr vector schema on this index (an index of a vector region).
+
+        It replaces the index configuration; an index that already holds documents needs
+        ``force=True``, and those documents must be sent again. Returns the new check.
+        """
+        self._ensure_index()
+        return self.client.apply_schema(self.index, force=force)
+
     def write_files(
         self,
         path: Union[str, Path],
@@ -402,7 +423,7 @@ class OpensolrDocumentStore:
         dated in that period (``creation_date``). ``group_pages`` (on by default) returns
         a PDF indexed page by page once, as its best page: ``meta`` carries
         ``page_number``, ``page_url`` (opens the PDF at that page), ``document_id`` and
-        ``more_pages`` (its other matching pages). Every document carries ``uri``.
+        ``more_pages`` (its other matching pages) and its ``uri``.
         """
         self._ensure_index()
         params: Dict[str, Any] = {"rows": top_k, "fl": "*,score"}
@@ -434,6 +455,8 @@ class OpensolrDocumentStore:
             expanded = body.get("expanded") or {}
             return [self._doc_from_solr(d, expanded) for d in body["response"]["docs"]]
 
+        # An index without the vector fields says so before an embedding is paid for
+        self.client.require_capability(self.index, "vector_search")
         # The embedder must never see an operator — it has no concept of negation, so a '-'
         # term reads as one more word and pulls results towards what the caller asked to drop.
         vector = self.client.embed(

{opensolr_haystack-0.7.0 → opensolr_haystack-0.8.0}/opensolr_haystack.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: opensolr-haystack
-Version: 0.7.0
+Version: 0.8.0
 Summary: Haystack integration for Opensolr — managed Apache Solr DocumentStore with server-side embeddings and hybrid BM25+kNN retrieval
 Author-email: Opensolr <support@opensolr.com>
 License: MIT
@@ -61,8 +61,8 @@ the query. The store embeds server-side at both index and query time.
 
 ## Try it without an account
 
-There is a public demo account. Point the package at it and search, ingestion and
-answers work immediately, with no signup:
+There is a public demo account. Point the package at it and everything in this
+README works immediately, with no signup:
 
 ```bash
 export OPENSOLR_EMAIL=mcp@opensolr.com
@@ -70,17 +70,18 @@ export OPENSOLR_API_KEY=420b8b23e7b12dc8ab838932145a5065
 ```
 
 `mcp_demo_d1__dense` is already loaded with 300 news articles, so search, filtering
-and grounded answers work the moment you connect. You also get the full write path:
-create your own index on the account, ingest into it, query it, delete it.
-Reading files and pictures (the file loader and file indexing below, search by image)
-needs your own key: the shared demo key does not include them.
+and grounded answers work the moment you connect. You also get the write path:
+create your own index on the account, ingest into it and query it. Deletion is not
+available on this shared key: no index here can be deleted or reconfigured by hand.
+Whatever you create is removed automatically after 3 days.
 
 Know what you are working with:
 
 - **Anything you create there is deleted after 3 days.** Automatically, without warning
   or export. That includes indexes you created and every document in them.
-- **The account is shared with everyone reading this.** Your index is visible to them,
-  they can change or delete it, and you can do the same to theirs. Never put anything
+- **The account is shared with everyone reading this.** Your index is visible to them
+  and they can add documents to it, as you can to theirs. Nobody can delete or
+  reconfigure an index here (that is switched off for this key), but never put anything
   real, private or client-owned in it.
 - **The limits are per index, and deliberately small.** 200 MB of bandwidth and 50 MB
   of disk per index. Bandwidth is the one you will hit first: it covers a demo, a
@@ -172,8 +173,9 @@ text from PDF/DOCX/XLSX for you.
 ## Lexical-only mode
 
 Don't need vectors? Pure keyword search skips the embedding call entirely —
-zero AI quota, and it works on **any** Opensolr index, including non-vector
-ones and older Solr versions.
+zero AI quota. It runs on any Opensolr index that has `title`, `description` and
+`text` fields, including indexes outside the vector regions and older Solr
+versions (see [Index requirements](#index-requirements)).
 
 ## Search by image
 
@@ -241,7 +243,7 @@ Data Ingestion with `rtf`, PDFs found by the Web Crawler), a search returns each
 **once**, as its best page. Its metadata carries `page_number` (the page number),
 `page_url` (opens the PDF at that page, `...pdf#page=14`), `document_id`,
 `more_pages` (its other matching pages, each with `page` and `page_url`) and
-`matching_pages`; every hit carries its `uri`. `group_pages=False` returns the pages
+`matching_pages`, and its `uri`; any other hit keeps exactly the metadata it was sent with. `group_pages=False` returns the pages
 as separate results.
 
 ## Searching a period
@@ -257,6 +259,74 @@ OpensolrHybridRetriever(document_store=store, date_from="2026-01-01")
 
 They combine with every other option — filters, `fresh_bias`, hybrid or lexical.
 
+## Index requirements
+
+Everything this package does runs on an Opensolr index that sits in a **vector
+region** and carries the **Opensolr vector schema** — the configuration every
+Opensolr vector index runs. An index created through the package (`create_index`,
+`create_if_missing`) has both. An index made elsewhere may not: a server outside
+the vector regions, an older Solr version, or a schema of its own.
+
+**The check is automatic.** The first time the package touches an index, it asks
+the platform once whether that index can do what is asked: one request per index
+per client, kept for the client's life, never one per search. If the check itself
+cannot run (a network error, the platform busy), nothing is blocked: the
+operation runs exactly as it always did, and the check is tried again after 5
+minutes. When an index differs from the Opensolr vector schema, a warning is logged
+once, on the `opensolr` logger, with the reasons and the fix.
+
+What the check gates — only what cannot work otherwise:
+
+| Capability | Needed for | When it is missing |
+|---|---|---|
+| `ingestion` | Data Ingestion: adding documents and indexing files | The index is outside the vector regions: refused before a document or a byte is sent |
+| `vector_search` | Vector and hybrid search | The index has no vector field of the right size: refused before any embedding is paid for |
+| `pdf_pages` | Grouping the pages of a PDF in search results | Results come back ungrouped, as before |
+| `keyword_search` | Keyword (lexical) search | Never blocked |
+
+**The errors say exactly what is going on.** A blocked operation raises
+`OpensolrSchemaError` (a subclass of `OpensolrError`) with `.index`, `.capability`
+and `.report` (the full check), and a message with every reason and the fix:
+
+```
+Index 'shop_it' cannot take documents or files through Data Ingestion. Index shop_it
+is on IT-SOLR-9-6 (Milano, Italy), which is not a vector region: Data Ingestion,
+server-side embeddings and hybrid search run only on the servers of a vector region
+(DE-SOLR-9 (Germany, Solr 9.00), FINLAND9 (Finland, Solr 9.60), ...). This index
+cannot be made ready: create a new index in one of the vector regions (...), for
+example with create_index, and send your documents there.
+```
+
+When Solr itself refuses a request, `OpensolrSolrError` carries Solr's own reason
+and the index name; it is still an `httpx.HTTPStatusError`, so code that caught
+that before keeps working.
+
+```python
+from haystack_integrations.document_stores.opensolr import OpensolrDocumentStore, OpensolrSchemaError, OpensolrSolrError
+
+try:
+    store.write_files("./contracts")
+except OpensolrSchemaError as e:
+    print(e.capability, e.report["fix"]["action"])   # ingestion new_index
+```
+
+**See the details, and fix what can be fixed:**
+
+```python
+store.check_schema()   # ready, capabilities, server, Solr version, missing / different fields, problems, fix
+store.apply_schema(force=False)   # installs the Opensolr vector schema; force=True when the index holds documents
+```
+
+`apply_schema` exists so an index of a vector region whose configuration was
+changed, or never set, can take everything this package does without being
+recreated: it installs the same configuration every Opensolr vector index runs,
+and **replaces the index configuration**. An index that already holds documents
+needs `force=True`; those documents keep the fields they were written with and
+must be sent again. An index outside the vector regions cannot take it — create
+a new index in a region from `vector_regions` and send your documents there. The
+package never installs the schema on its own.
+
+
 ## Your index schema
 
 Documents follow the Opensolr document model (`title`, `description`, `text`,

{opensolr_haystack-0.7.0 → opensolr_haystack-0.8.0}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "opensolr-haystack"
-version = "0.7.0"
+version = "0.8.0"
 description = "Haystack integration for Opensolr — managed Apache Solr DocumentStore with server-side embeddings and hybrid BM25+kNN retrieval"
 readme = "README.md"
 license = { text = "MIT" }