vector-graph-rag 0.1.5__tar.gz → 0.2.1__tar.gz

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: vector-graph-rag
-Version: 0.1.5
+Version: 0.2.1
 Summary: A Graph RAG implementation using pure vector search with Milvus
 Project-URL: Homepage, https://github.com/zilliztech/vector-graph-rag
 Project-URL: Documentation, https://zilliztech.github.io/vector-graph-rag/
@@ -195,18 +195,23 @@ rag.rebuild_documents_with_triplets([
 <details>
 <summary>🔄 <b>Incremental document updates</b> — click to expand</summary>
 
-Use `upsert_documents()` when a source file, message, or page is created or modified.
-The method replaces only that source document's chunks and graph references.
+Use `upsert_documents_by_source()` when a source file, message, or page is
+created or modified. In Vector Graph RAG, a `Document` is a passage/chunk; the
+source object is identified by `metadata["source"]` or the explicit `source`
+argument. The method replaces only that source's chunks and graph references.
+Source-level writes are not transactionally atomic, but the same upsert/delete
+operation can be retried after an interruption to converge the source back to a
+consistent state.
 
 ```python
 from langchain_core.documents import Document
 
-rag.upsert_documents(
-    document_id="sharepoint:file-123",
+rag.upsert_documents_by_source(
     documents=[
         Document(
             page_content="Einstein developed relativity at Princeton.",
             metadata={
+                "source": "sharepoint:file-123",
                 "triplets": [
                     ["Einstein", "developed", "relativity"],
                     ["Einstein", "worked at", "Princeton"],
@@ -214,13 +219,17 @@ rag.upsert_documents(
             },
         ),
     ],
-    metadata={"source": "sharepoint"},
     extract_triplets=False,
 )
 
-rag.delete_documents("sharepoint:file-123")
+rag.delete_documents_by_source("sharepoint:file-123")
 ```
 
+> **Migration note:** v0.1.5 exposed `upsert_documents(document_id=...)` and
+> `delete_documents(document_id)`. These names were removed in v0.2.0 because
+> `Document` means passage/chunk in this project. Use the `*_by_source()` APIs
+> shown above.
+
 The legacy `add_*` ingestion helpers rebuild the full knowledge base and are planned
 for removal in v1.0.0. For explicit full refreshes, use `rebuild_texts()`,
 `rebuild_documents()`, or `rebuild_documents_with_triplets()`.

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/README.md

@@ -105,18 +105,23 @@ rag.rebuild_documents_with_triplets([
 <details>
 <summary>🔄 <b>Incremental document updates</b> — click to expand</summary>
 
-Use `upsert_documents()` when a source file, message, or page is created or modified.
-The method replaces only that source document's chunks and graph references.
+Use `upsert_documents_by_source()` when a source file, message, or page is
+created or modified. In Vector Graph RAG, a `Document` is a passage/chunk; the
+source object is identified by `metadata["source"]` or the explicit `source`
+argument. The method replaces only that source's chunks and graph references.
+Source-level writes are not transactionally atomic, but the same upsert/delete
+operation can be retried after an interruption to converge the source back to a
+consistent state.
 
 ```python
 from langchain_core.documents import Document
 
-rag.upsert_documents(
-    document_id="sharepoint:file-123",
+rag.upsert_documents_by_source(
     documents=[
         Document(
             page_content="Einstein developed relativity at Princeton.",
             metadata={
+                "source": "sharepoint:file-123",
                 "triplets": [
                     ["Einstein", "developed", "relativity"],
                     ["Einstein", "worked at", "Princeton"],
@@ -124,13 +129,17 @@ rag.upsert_documents(
             },
         ),
     ],
-    metadata={"source": "sharepoint"},
     extract_triplets=False,
 )
 
-rag.delete_documents("sharepoint:file-123")
+rag.delete_documents_by_source("sharepoint:file-123")
 ```
 
+> **Migration note:** v0.1.5 exposed `upsert_documents(document_id=...)` and
+> `delete_documents(document_id)`. These names were removed in v0.2.0 because
+> `Document` means passage/chunk in this project. Use the `*_by_source()` APIs
+> shown above.
+
 The legacy `add_*` ingestion helpers rebuild the full knowledge base and are planned
 for removal in v1.0.0. For explicit full refreshes, use `rebuild_texts()`,
 `rebuild_documents()`, or `rebuild_documents_with_triplets()`.

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/docs/faq.md

@@ -43,27 +43,25 @@ Frequently asked questions about Vector Graph RAG — covering when to use it, c
     For very large corpora, consider using a remote Milvus instance rather than Milvus Lite for better performance and scalability.
 
 ??? note "How do I update or delete one source document?"
-    Use `upsert_documents()` for source-document create/update and `delete_documents()` for delete. A source document can be a file, web page, message, or another business-level object. Its parsed chunks are passed as LangChain `Document` objects.
+    Use `upsert_documents_by_source()` for source create/update and `delete_documents_by_source()` for delete. A source can be a file, web page, message, or another business-level object. Its parsed chunks are passed as LangChain `Document` objects, and each chunk should carry the same stable `metadata["source"]` value.
 
     ```python
     from langchain_core.documents import Document
 
-    rag.upsert_documents(
-        document_id="file-123",
+    rag.upsert_documents_by_source(
         documents=[
             Document(
                 page_content="Updated document chunk text.",
-                metadata={"page": 1},
+                metadata={"source": "file-123", "page": 1},
             )
         ],
-        metadata={"source": "sharepoint"},
         extract_triplets=True,
     )
 
-    rag.delete_documents("file-123")
+    rag.delete_documents_by_source("file-123")
     ```
 
-    Incremental updates replace only the chunks and graph references for that `document_id`. They are not transactionally atomic, so avoid interrupting or concurrently mutating the same collection prefix during an update.
+    Incremental updates replace only the chunks and graph references for that source value. They are not transactionally atomic, so avoid interrupting or concurrently mutating the same collection prefix during an update.
 
 ??? note "Can I use local/open-source LLMs?"
     Yes. Vector Graph RAG uses the OpenAI-compatible API format, so any LLM that exposes an OpenAI-compatible endpoint will work. This includes local models served via [Ollama](https://ollama.com/), [vLLM](https://github.com/vllm-project/vllm), [LM Studio](https://lmstudio.ai/), or any other OpenAI-compatible server. You can configure the base URL and model name when initializing the RAG instance. Keep in mind that triplet extraction and reranking quality depend heavily on the LLM's capability — weaker models may produce incomplete or inaccurate triplets, which directly affects retrieval quality. For best results, use a model with strong instruction-following and reasoning abilities.

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/docs/getting-started.md

@@ -134,7 +134,7 @@ finance_rag = VectorGraphRAG(milvus_uri="./data.db", collection_prefix="finance"
 ## Adding Documents
 
 !!! warning "Full rebuild vs incremental updates"
-    `add_texts()`, `add_documents()`, and `add_documents_with_triplets()` rebuild the full knowledge base for the current collection prefix. These legacy convenience APIs are planned for removal in v1.0.0. Use `rebuild_texts()`, `rebuild_documents()`, or `rebuild_documents_with_triplets()` for full refreshes. For source-document create/update/delete flows, use `upsert_documents()` and `delete_documents()`.
+    `add_texts()`, `add_documents()`, and `add_documents_with_triplets()` rebuild the full knowledge base for the current collection prefix. These legacy convenience APIs are planned for removal in v1.0.0. Use `rebuild_texts()`, `rebuild_documents()`, or `rebuild_documents_with_triplets()` for full refreshes. For source create/update/delete flows, use `upsert_documents_by_source()` and `delete_documents_by_source()`.
 
 ### From Text Strings
 
@@ -184,17 +184,17 @@ rag.rebuild_documents(result.documents, extract_triplets=True)
 
 ### Incremental Updates
 
-Use `upsert_documents()` when a source file, page, or message is created or modified. The method replaces only that source document's chunks and graph references.
+Use `upsert_documents_by_source()` when a source file, page, or message is created or modified. In Vector Graph RAG, a `Document` is a passage/chunk, and `metadata["source"]` identifies the source object whose chunks should be replaced.
 
 ```python
 from langchain_core.documents import Document
 
-rag.upsert_documents(
-    document_id="sharepoint:file-123",
+rag.upsert_documents_by_source(
     documents=[
         Document(
             page_content="Einstein developed relativity at Princeton.",
             metadata={
+                "source": "sharepoint:file-123",
                 "triplets": [
                     ["Einstein", "developed", "relativity"],
                     ["Einstein", "worked at", "Princeton"],
@@ -202,15 +202,17 @@ rag.upsert_documents(
             },
         )
     ],
-    metadata={"source": "sharepoint"},
     extract_triplets=False,
 )
 
-rag.delete_documents("sharepoint:file-123")
+rag.delete_documents_by_source("sharepoint:file-123")
 ```
 
+!!! warning "v0.2.0 migration"
+    `upsert_documents(document_id=...)` and `delete_documents(document_id)` were removed because `Document` means passage/chunk in this project. Use `upsert_documents_by_source()` and `delete_documents_by_source()` with a stable `metadata["source"]` value instead.
+
 !!! note "Consistency model"
-    Incremental updates perform a document-level cascade across passages, entities, and relations. They are not transactionally atomic, so avoid interrupting or concurrently mutating the same collection prefix during an update.
+    Incremental updates perform a source-level cascade across passages, entities, and relations. They are not transactionally atomic, and queries may observe intermediate state if a write is interrupted. If an upsert or delete fails, rerun the same operation for the same source to converge the source back to a consistent state. Avoid concurrently mutating the same collection prefix during an update.
 
 ## Querying
 

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/docs/how-it-works.md

@@ -223,4 +223,4 @@ flowchart TD
     Triplet extraction and reranking quality depends heavily on the underlying LLM's capability. Weaker models may produce incomplete or inaccurate triplets, which directly affects retrieval quality.
 
 !!! warning "Graph Consistency"
-    The knowledge graph topology is maintained as a logical layer on top of Milvus via cross-referenced ID fields. Since Milvus does not support transactions, multi-step mutations (e.g., document-level cascade updates) are not atomic and may leave inconsistent state if interrupted. Use `rebuild_documents()` for initial bulk indexing or full refreshes, and use `upsert_documents()` / `delete_documents()` for source-document updates when the write operation is expected to run to completion.
+    The knowledge graph topology is maintained as a logical layer on top of Milvus via cross-referenced ID fields. Since Milvus does not support transactions, multi-step mutations (e.g., source-level cascade updates) are not atomic, and queries may observe intermediate state if a write is interrupted. Source-level `upsert_documents_by_source()` and `delete_documents_by_source()` operations are designed to be retryable: rerun the same operation for the same source after a failure to converge the source back to a consistent state. Use `rebuild_documents()` for initial bulk indexing or full refreshes.

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/docs/index.md

@@ -114,7 +114,7 @@ print(result.answer)
 ```
 
 !!! note "Ingestion semantics"
-    The legacy `add_*` ingestion helpers rebuild the full knowledge base and are planned for removal in v1.0.0. Use `rebuild_texts()`, `rebuild_documents()`, or `rebuild_documents_with_triplets()` for full refreshes, and use `upsert_documents()` / `delete_documents()` when a source file, page, or message changes later.
+    The legacy `add_*` ingestion helpers rebuild the full knowledge base and are planned for removal in v1.0.0. Use `rebuild_texts()`, `rebuild_documents()`, or `rebuild_documents_with_triplets()` for full refreshes, and use `upsert_documents_by_source()` / `delete_documents_by_source()` when a source file, page, or message changes later.
 
 !!! tip "Getting Started"
     See the [Getting Started](getting-started.md) guide for installation and configuration options.

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/docs/python-api.md

@@ -177,7 +177,7 @@ Legacy compatibility wrapper for rebuilding the graph from plain text strings.
 
 !!! warning "Full rebuild"
     This method delegates to `rebuild_texts()` and rebuilds the full knowledge base.
-    This legacy convenience API is planned for removal in v1.0.0. Use [`rebuild_texts`](#rebuild_texts) for full refreshes, or [`upsert_documents`](#upsert_documents) for document-level incremental updates.
+    This legacy convenience API is planned for removal in v1.0.0. Use [`rebuild_texts`](#rebuild_texts) for full refreshes, or [`upsert_documents_by_source`](#upsert_documents_by_source) for source-level incremental updates.
 
 ```python
 def add_texts(
@@ -251,7 +251,7 @@ result = rag.rebuild_texts([
 Legacy compatibility wrapper for rebuilding the graph from [LangChain `Document`](https://python.langchain.com/docs/modules/data_connection/document_loaders/) objects.
 
 !!! warning "Full rebuild"
-    `add_documents()` keeps its original behavior for backward compatibility: it drops and recreates the Milvus collections for this graph before indexing the provided documents. This legacy API is planned for removal in v1.0.0. Use [`rebuild_documents`](#rebuild_documents) when you want this behavior explicitly, or [`upsert_documents`](#upsert_documents) for incremental create/update.
+    `add_documents()` keeps its original behavior for backward compatibility: it drops and recreates the Milvus collections for this graph before indexing the provided documents. This legacy API is planned for removal in v1.0.0. Use [`rebuild_documents`](#rebuild_documents) when you want this behavior explicitly, or [`upsert_documents_by_source`](#upsert_documents_by_source) for source-level incremental create/update.
 
 ```python
 def add_documents(
@@ -311,7 +311,7 @@ Use this for initial bulk indexing, benchmark rebuilds, or when you intentionall
 Legacy compatibility wrapper for rebuilding the graph from documents where triplets have already been extracted externally.
 
 !!! warning "Full rebuild"
-    This method delegates to `rebuild_documents_with_triplets()` and rebuilds the full knowledge base. This legacy convenience API is planned for removal in v1.0.0. For pre-extracted triplets in an incremental update, put the triplets in each chunk's `metadata["triplets"]` and call [`upsert_documents`](#upsert_documents) with `extract_triplets=False`.
+    This method delegates to `rebuild_documents_with_triplets()` and rebuilds the full knowledge base. This legacy convenience API is planned for removal in v1.0.0. For pre-extracted triplets in an incremental update, put the triplets in each chunk's `metadata["triplets"]` and call [`upsert_documents_by_source`](#upsert_documents_by_source) with `extract_triplets=False`.
 
 ```python
 def add_documents_with_triplets(
@@ -375,14 +375,15 @@ Optional `metadata` is stored on the passage and can be used by query filters.
 
 ---
 
-#### `upsert_documents`
+#### `upsert_documents_by_source`
 
-Incrementally create or replace one source document. A source document can be a file, page, message, or any other business-level object. The `documents` argument contains the parsed chunks/passages for that source document.
+Incrementally create or replace all chunks that belong to one source. A source can be a file, page, message, SharePoint item, business record, or any other stable external object. In Vector Graph RAG, a LangChain `Document` is a passage/chunk; source ownership is represented by `metadata["source"]` by default.
 
 ```python
-def upsert_documents(
-    document_id: str,
+def upsert_documents_by_source(
     documents: List[Document],
+    source: Optional[str] = None,
+    source_field: str = "source",
     metadata: Optional[Dict[str, Any]] = None,
     extract_triplets: bool = True,
     show_progress: bool = True,
@@ -391,15 +392,20 @@ def upsert_documents(
 
 | Parameter | Description |
 |---|---|
-| `document_id` | Stable source document ID, such as a file ID or message ID. |
-| `documents` | Parsed chunks/passages for this source document. Missing chunk IDs are generated deterministically from `document_id` and chunk index. |
-| `metadata` | Source-level metadata merged into every chunk. |
+| `documents` | Parsed chunks/passages for one source. If a chunk has `Document.id`, it is used as the passage ID; otherwise a deterministic passage ID is generated from `source` and chunk index. |
+| `source` | Optional stable source value. If omitted, all documents must include the same `Document.metadata[source_field]` value. |
+| `source_field` | Metadata field used to group chunks by source. Defaults to `"source"`. |
+| `metadata` | Source-level metadata merged into every chunk. It must not conflict with `source` or the chunks' `source_field` values. |
 | `extract_triplets` | If `True`, extract triplets from each chunk. If triplets are already in `metadata["triplets"]`, set this to `False`. |
 | `show_progress` | Show a progress bar. |
 
 **Returns:** [`ExtractionResult`](#extractionresult)
 
-If `document_id` does not exist, the method inserts a new document. If it already exists, the method deletes the previous chunks for that document, updates graph references, and inserts the new chunks without rebuilding unrelated documents.
+If the source does not exist, the method inserts a new source. If it already exists, the method deletes the previous chunks for that source, updates graph references, and inserts the new chunks without rebuilding unrelated sources.
+
+Source-level writes are not transactionally atomic. If an upsert fails during the multi-step cascade, rerun the same `upsert_documents_by_source()` call for the same source to converge the source back to the requested state. Queries may observe intermediate state until the retry succeeds.
+
+The method accepts exactly one source per call. If `source` is not provided and the documents contain multiple `metadata[source_field]` values, it raises `ValueError`.
 
 ```python
 from langchain_core.documents import Document
@@ -407,40 +413,51 @@ from langchain_core.documents import Document
 chunks = [
     Document(
         page_content="Alpha owns the blue database.",
-        metadata={"triplets": [["Alpha", "owns", "blue database"]]},
+        metadata={
+            "source": "sharepoint:file-123",
+            "triplets": [["Alpha", "owns", "blue database"]],
+        },
     )
 ]
 
-rag.upsert_documents(
-    document_id="sharepoint:file-123",
+rag.upsert_documents_by_source(
     documents=chunks,
-    metadata={"tenant_id": "team_a", "source": "sharepoint"},
+    metadata={"tenant_id": "team_a"},
     extract_triplets=False,
 )
 ```
 
 ---
 
-#### `delete_documents`
+#### `delete_documents_by_source`
 
-Incrementally delete one source document and remove graph references that only belonged to that document.
+Incrementally delete all chunks that belong to one source and remove graph references that only belonged to that source.
 
 ```python
-def delete_documents(document_id: str) -> bool
+def delete_documents_by_source(
+    source: str,
+    source_field: str = "source",
+) -> bool
 ```
 
 | Parameter | Description |
 |---|---|
-| `document_id` | Stable source document ID previously used with `upsert_documents()`. |
+| `source` | Stable source value previously used with `upsert_documents_by_source()`. |
+| `source_field` | Metadata field used to group chunks by source. Defaults to `"source"`. |
 
 **Returns:** `True` if at least one passage was deleted; otherwise `False`.
 
-Shared entities and relations are preserved when other documents still reference them. Orphaned relations and entities are removed.
+Shared entities and relations are preserved when other sources still reference them. Orphaned relations and entities are removed.
+
+Source-level deletes are retryable. If a delete fails after partially cleaning graph records, rerun `delete_documents_by_source()` with the same `source` and `source_field` to finish the cascade.
 
 ```python
-deleted = rag.delete_documents("sharepoint:file-123")
+deleted = rag.delete_documents_by_source("sharepoint:file-123")
 ```
 
+!!! warning "v0.2.0 migration"
+    `upsert_documents(document_id=...)` and `delete_documents(document_id)` were removed because `Document` means passage/chunk in this project. These methods now raise `RuntimeError` with migration guidance. Use `upsert_documents_by_source()` and `delete_documents_by_source()` with a stable `metadata["source"]` value.
+
 ---
 
 #### `query`
@@ -654,7 +671,7 @@ print(f"Final passages: {len(result.passages)}")
 
 ## ExtractionResult
 
-A data class returned by document ingestion methods, including legacy `add_*`, `rebuild_*`, and `upsert_documents()`. It summarises what was ingested and extracted.
+A data class returned by document ingestion methods, including legacy `add_*`, `rebuild_*`, and `upsert_documents_by_source()`. It summarises what was ingested and extracted.
 
 ```python
 from vector_graph_rag import ExtractionResult

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/docs/rest-api.md

@@ -161,7 +161,7 @@ curl -X DELETE http://localhost:8000/graph/my_graph
 Indexes documents by rebuilding the selected graph. Optionally extracts knowledge graph triplets from the text using the configured LLM.
 
 !!! warning "Full rebuild"
-    This endpoint uses the Python full-rebuild ingestion path. Calling it replaces the current graph contents for the selected `graph_name`. Document-level incremental upsert/delete is currently available through the Python API via `upsert_documents()` and `delete_documents()`.
+    This endpoint uses the Python full-rebuild ingestion path. Calling it replaces the current graph contents for the selected `graph_name`. Source-level incremental upsert/delete is currently available through the Python API via `upsert_documents_by_source()` and `delete_documents_by_source()`.
 
 **Query Parameters**
 
@@ -230,7 +230,7 @@ curl -X POST "http://localhost:8000/add_documents?graph_name=my_graph" \
 Imports documents from URLs or local file paths. Supports chunking for large documents.
 
 !!! warning "Full rebuild"
-    Imported documents are indexed through the full-rebuild ingestion path for the target graph. Use the Python API for document-level incremental updates after parsing files externally.
+    Imported documents are indexed through the full-rebuild ingestion path for the target graph. Use the Python API for source-level incremental updates after parsing files externally.
 
 **Request Body**
 
@@ -402,7 +402,7 @@ Retrieves a single document by its ID.
 
 | Parameter | Type | Description |
 |---|---|---|
-| `document_id` | `string` | The document ID. |
+| `document_id` | `string` | The passage/document ID used by the REST document CRUD endpoints. This is not the source key used by `upsert_documents_by_source()`. |
 
 **Query Parameters**
 

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "vector-graph-rag"
-version = "0.1.5"
+version = "0.2.1"
 description = "A Graph RAG implementation using pure vector search with Milvus"
 readme = "README.md"
 license = { text = "MIT" }

{vector_graph_rag-0.1.5 → vector_graph_rag-0.2.1}/src/vector_graph_rag/__init__.py

@@ -18,7 +18,7 @@ from vector_graph_rag.rag import VectorGraphRAG, create_rag
 from vector_graph_rag.storage.embeddings import EmbeddingModel
 from vector_graph_rag.storage.milvus import MilvusStore
 
-__version__ = "0.1.5"
+__version__ = "0.2.1"
 
 __all__ = [
     "Settings",