vector-graph-rag 0.4.0__tar.gz → 0.4.1__tar.gz

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/CHANGELOG.md

@@ -1,5 +1,24 @@
 # Changelog
 
+## Unreleased
+
+## 0.4.1
+
+### Fixes
+
+- Preserve entity/passage links for every chunk that shares a relation, including full rebuilds, incremental source updates and passage creation. Source deletion can now clean up shared entities after every source has been removed.
+- Reject invalid chunk sizes and overlap values before importing documents. Mixed short/long file imports can no longer report success while silently omitting text because of an invalid character-splitting step.
+- Read knowledge-base statistics from strongly consistent Milvus counts across all sources. Counts reflect incremental updates, deletions and reconnects; storage failures raise an error and return HTTP 500 through the REST statistics endpoint.
+
+### Documentation
+
+- Add the Jev 1.14 evaluation report, paired latency calibration and updated quality–latency comparison.
+- Document current knowledge-base statistics, storage failures and concurrent-update behavior.
+
+### Upgrading
+
+The shared-relation fix applies to new writes. Existing missing adjacency links are not repaired automatically. Rebuild affected knowledge bases with `rebuild_documents()` using the complete corpus and its source metadata. A rebuild replaces all data under the selected collection prefix, so include every source that should remain available.
+
 ## 0.4.0
 
 ### What's changed

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: vector-graph-rag
-Version: 0.4.0
+Version: 0.4.1
 Summary: A Graph RAG implementation using pure vector search with Milvus
 Project-URL: Homepage, https://github.com/zilliztech/vector-graph-rag
 Project-URL: Documentation, https://zilliztech.github.io/vector-graph-rag/
@@ -398,7 +398,7 @@ Before introducing Jev, we evaluated Vector Graph RAG on three multi-hop QA benc
 
 **Two-stage Jev reaches the quality–latency Pareto frontier.**
 
-**86.07% average Recall@5**, the highest among the compared methods, with **3.13 seconds of additional model-call latency**. On MuSiQue and 2Wiki (1,000 questions each), two-stage Jev reaches the Pareto frontier using the displayed timings. This two-dataset result is separate from the historical three-dataset average above.
+**Jev 1.14 achieves 86.61% average Recall@5**, the highest among the compared methods, with **approximately 4.9 seconds of estimated direct API latency**. On MuSiQue and 2Wiki (1,000 questions each), two-stage Jev reaches the Pareto frontier using the displayed timings. This two-dataset result is separate from the historical three-dataset average above.
 
 ![Quality and additional model latency](docs/assets/evaluation/quality-latency.png)
 

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/README.md

@@ -290,7 +290,7 @@ Before introducing Jev, we evaluated Vector Graph RAG on three multi-hop QA benc
 
 **Two-stage Jev reaches the quality–latency Pareto frontier.**
 
-**86.07% average Recall@5**, the highest among the compared methods, with **3.13 seconds of additional model-call latency**. On MuSiQue and 2Wiki (1,000 questions each), two-stage Jev reaches the Pareto frontier using the displayed timings. This two-dataset result is separate from the historical three-dataset average above.
+**Jev 1.14 achieves 86.61% average Recall@5**, the highest among the compared methods, with **approximately 4.9 seconds of estimated direct API latency**. On MuSiQue and 2Wiki (1,000 questions each), two-stage Jev reaches the Pareto frontier using the displayed timings. This two-dataset result is separate from the historical three-dataset average above.
 
 ![Quality and additional model latency](docs/assets/evaluation/quality-latency.png)
 

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/docs/architecture/design-philosophy.md

@@ -238,7 +238,7 @@ flowchart TB
 | **Local-first** | Yes | Yes | Partial | No | No | **Yes (Milvus Lite)** |
 | **Setup complexity** | Minimal | Moderate | Moderate | High | High | **Minimal** |
 
-### Performance on Multi-Hop QA Benchmarks
+### Historical Performance on Multi-Hop QA Benchmarks
 
 | Method | MuSiQue | HotpotQA | 2WikiMultiHopQA | Average |
 |--------|---------|----------|-----------------|---------|
@@ -248,7 +248,7 @@ flowchart TB
 | **Vector Graph RAG** | 73.0% | **96.3%** | **94.1%** | **87.8%** |
 
 > [!INFO]
-> Recall@5 on standard multi-hop QA benchmarks. Vector Graph RAG achieves the highest average score while using fewer LLM calls and no graph database. See [Evaluation](../evaluation.md) for methodology details.
+> Historical three-dataset Recall@5 comparison. The October 16, 2026 two-stage Jev 1.14 evaluation reaches 77.73% on MuSiQue and 95.50% on 2Wiki (86.61% average), the highest-quality point on the observed quality–latency Pareto frontier. Its two-dataset scope is separate from this table. See [Evaluation](../evaluation.md#jev-reranker-evaluation) for methodology and paired comparison with Jev 1.13.
 
 ### Key Differentiators
 

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/docs/assets/evaluation/quality-latency.svg

@@ -255,6 +255,7 @@ L 1047.6 110.16
     <path d="M 123.198784 299.028522
 L 255.174996 221.497435
 L 304.195667 154.117565
+L 404.684758 148.017522
 " clip-path="url(#pfb3f67a1a0)" style="fill: none; stroke-dasharray: 4.81,2.08; stroke-dashoffset: 0; stroke: #168568; stroke-opacity: 0.7; stroke-width: 1.3"/>
    </g>
    <g id="patch_3">
@@ -272,12 +273,12 @@ L 1047.6 495.72
     <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #777f8b" transform="translate(609.843161 232.522437)">82.55% · ≈8.25 s</text>
    </g>
    <g id="text_20">
-    <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #168568" transform="translate(265.174996 244.232747)">Jev · relations only</text>
+    <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #168568" transform="translate(265.174996 244.232747)">Jev 1.13 · relations only</text>
     <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #168568" transform="translate(265.174996 260.601263)">80.04% · 2.28 s</text>
    </g>
    <g id="text_21">
-    <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #168568" transform="translate(314.195667 123.883581)">Jev · two-stage</text>
-    <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #168568" transform="translate(314.195667 140.421393)">86.07% · 3.13 s</text>
+    <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #168568" transform="translate(219.195667 115.955768)">Jev 1.13 · two-stage</text>
+    <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #168568" transform="translate(219.195667 132.421393)">86.07% · 3.13 s</text>
    </g>
    <g id="text_22">
     <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #a16826" transform="translate(442.363178 215.399524)">GPT-6 Luna</text>
@@ -288,12 +289,16 @@ L 1047.6 495.72
     <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #2674bd" transform="translate(766.993357 170.921611)">83.34% · 10.97 s</text>
    </g>
    <g id="text_24">
-    <text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #777f8b" x="135.198784" y="303.028522" transform="rotate(-0 135.198784 303.028522)">Naive RAG · NV-Embed-v2  (73.10%)</text>
+    <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #087049" transform="translate(414.684758 117.783537)">Jev 1.14 · two-stage</text>
+    <text style="font-size: 11px; font-family: 'DejaVu Sans'; fill: #087049" transform="translate(414.684758 134.32135)">86.61% · ≈4.87 s</text>
    </g>
    <g id="text_25">
-    <text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #777f8b" x="135.198784" y="389.31387" transform="rotate(-0 135.198784 389.31387)">Naive RAG · BGE-large-en-v1.5  (65.38%)</text>
+    <text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #777f8b" x="135.198784" y="303.028522" transform="rotate(-0 135.198784 303.028522)">Naive RAG · NV-Embed-v2  (73.10%)</text>
    </g>
    <g id="text_26">
+    <text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #777f8b" x="135.198784" y="389.31387" transform="rotate(-0 135.198784 389.31387)">Naive RAG · BGE-large-en-v1.5  (65.38%)</text>
+   </g>
+   <g id="text_27">
     <text style="font-size: 11px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #777f8b" x="135.198784" y="463.957913" transform="rotate(-0 135.198784 463.957913)">Naive RAG · ColBERTv2  (58.70%)</text>
    </g>
    <g id="PathCollection_1">
@@ -329,21 +334,15 @@ z
    </g>
    <g id="PathCollection_3">
     <defs>
-     <path id="mfa49a455e4" d="M 0 -6.892024
-L -1.547356 -2.129753
-L -6.554705 -2.129753
-L -2.503674 0.813493
-L -4.05103 5.575765
-L -0 2.632519
-L 4.05103 5.575765
-L 2.503674 0.813493
-L 6.554705 -2.129753
-L 1.547356 -2.129753
+     <path id="m36a28b9472" d="M -0 6.324555
+L 6.324555 0
+L 0 -6.324555
+L -6.324555 -0
 z
 " style="stroke: #ffffff; stroke-width: 0.9"/>
     </defs>
     <g clip-path="url(#pfb3f67a1a0)">
-     <use xlink:href="#mfa49a455e4" x="304.195667" y="154.117565" style="fill: #168568; stroke: #ffffff; stroke-width: 0.9"/>
+     <use xlink:href="#m36a28b9472" x="304.195667" y="154.117565" style="fill: #168568; stroke: #ffffff; stroke-width: 0.9"/>
     </g>
    </g>
    <g id="PathCollection_4">
@@ -383,6 +382,25 @@ z
     </g>
    </g>
    <g id="PathCollection_6">
+    <defs>
+     <path id="mb28cfb273e" d="M 0 -6.892024
+L -1.547356 -2.129753
+L -6.554705 -2.129753
+L -2.503674 0.813493
+L -4.05103 5.575765
+L -0 2.632519
+L 4.05103 5.575765
+L 2.503674 0.813493
+L 6.554705 -2.129753
+L 1.547356 -2.129753
+z
+" style="stroke: #ffffff; stroke-width: 0.9"/>
+    </defs>
+    <g clip-path="url(#pfb3f67a1a0)">
+     <use xlink:href="#mb28cfb273e" x="404.684758" y="148.017522" style="fill: #087049; stroke: #ffffff; stroke-width: 0.9"/>
+    </g>
+   </g>
+   <g id="PathCollection_7">
     <defs>
      <path id="m077a5d3511" d="M -3.708099 3.708099
 L 3.708099 3.708099
@@ -395,38 +413,38 @@ z
      <use xlink:href="#m077a5d3511" x="123.198784" y="299.028522" style="fill: #ffffff; stroke: #777f8b"/>
     </g>
    </g>
-   <g id="PathCollection_7">
+   <g id="PathCollection_8">
     <g clip-path="url(#pfb3f67a1a0)">
      <use xlink:href="#m077a5d3511" x="123.198784" y="385.31387" style="fill: #ffffff; stroke: #777f8b"/>
     </g>
    </g>
-   <g id="PathCollection_8">
+   <g id="PathCollection_9">
     <g clip-path="url(#pfb3f67a1a0)">
      <use xlink:href="#m077a5d3511" x="123.198784" y="459.957913" style="fill: #ffffff; stroke: #777f8b"/>
     </g>
    </g>
    <g id="legend_1">
     <g id="line2d_25">
-     <path d="M 855.489375 477.837187
-L 867.489375 477.837187
-L 879.489375 477.837187
+     <path d="M 797.900625 477.837187
+L 809.900625 477.837187
+L 821.900625 477.837187
 " style="fill: none; stroke-dasharray: 4.81,2.08; stroke-dashoffset: 0; stroke: #168568; stroke-opacity: 0.7; stroke-width: 1.3"/>
     </g>
-    <g id="text_27">
-     <text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: start" x="889.089375" y="482.037187" transform="rotate(-0 889.089375 482.037187)">Observed Pareto frontier</text>
+    <g id="text_28">
+     <text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: start" x="831.500625" y="482.037187" transform="rotate(-0 831.500625 482.037187)">Pareto frontier (displayed timings)</text>
     </g>
    </g>
   </g>
-  <g id="text_28">
-   <text style="font-weight: 700; font-size: 22px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #253143" x="97.2" y="36.72" transform="rotate(-0 97.2 36.72)">Retrieval quality–latency trade-off</text>
-  </g>
   <g id="text_29">
-   <text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #657181" x="97.2" y="70.38" transform="rotate(-0 97.2 70.38)">MuSiQue + 2Wiki · Equal-weight Recall@5 · 1,000 queries per dataset</text>
+   <text style="font-weight: 700; font-size: 22px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #253143" x="97.2" y="36.72" transform="rotate(-0 97.2 36.72)">Retrieval quality–latency trade-off</text>
   </g>
   <g id="text_30">
-   <text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #657181" x="97.2" y="556.92" transform="rotate(-0 97.2 556.92)">Jev, Luna and Flash: measured API means. HippoRAG 2: reference estimate (≈); separate evaluation setup.</text>
+   <text style="font-size: 12px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #657181" x="97.2" y="70.38" transform="rotate(-0 97.2 70.38)">MuSiQue + 2Wiki · Equal-weight Recall@5 · 1,000 queries per dataset</text>
   </g>
   <g id="text_31">
+   <text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #657181" x="97.2" y="556.92" transform="rotate(-0 97.2 556.92)">Jev 1.14: calibrated direct API estimate (≈). HippoRAG 2: reference estimate (≈). Other points: measured API means.</text>
+  </g>
+  <g id="text_32">
    <text style="font-size: 10px; font-family: 'DejaVu Sans'; text-anchor: start; fill: #657181" x="97.2" y="578.34" transform="rotate(-0 97.2 578.34)">0 s excludes embedding, retrieval and answer generation. API means include network and provider queueing; runs differ in time/concurrency.</text>
   </g>
  </g>

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/docs/evaluation.md

@@ -45,7 +45,7 @@ Vector Graph RAG improves over Naive RAG by **19.6% in relative average Recall@5
 [^1]: [HippoRAG: Neurobiologically Inspired Long-Term Memory for LLMs (NeurIPS 2024)](https://arxiv.org/abs/2405.14831)
 [^2]: [From RAG to Memory: Non-Parametric Continual Learning for LLMs (2025)](https://arxiv.org/abs/2502.14802)
 
-The historical evaluation reached **87.8% average Recall@5**, leading on 2Wiki while trailing HippoRAG 2 on MuSiQue. **Two-stage Jev now improves both: 76.78% on MuSiQue and 95.35% on 2Wiki**, ahead of the compared baselines on each dataset. See the [Jev results below](#jev-reranker-evaluation) for the full comparison.
+The historical evaluation reached **87.8% average Recall@5**, leading on 2Wiki while trailing HippoRAG 2 on MuSiQue. **Two-stage Jev now improves both: 77.73% on MuSiQue and 95.50% on 2Wiki**, ahead of the compared baselines on each dataset. See the [Jev results below](#jev-reranker-evaluation) for the full comparison.
 
 ---
 
@@ -91,7 +91,7 @@ We began evaluating Jev in September 2026, following its availability, and expan
 
 ### Reaching the quality–latency Pareto frontier
 
-**Two-stage Jev achieves the highest Recall@5 among the compared methods on both datasets**, averaging **86.07%** across **MuSiQue and 2Wiki, 1,000 questions each**. Using the displayed timings, it reaches the quality–latency Pareto frontier with **3.13 seconds of additional model-call time**.
+**Two-stage Jev 1.14 achieves the highest Recall@5 among the compared methods on both datasets**, averaging **86.61%** across **MuSiQue and 2Wiki, 1,000 questions each**. Using the displayed timings, it reaches the quality–latency Pareto frontier with **approximately 4.9 seconds of estimated direct API time**.
 
 The pipeline first selects relations, then reranks their source passages together with direct vector-search candidates. The second stage judges full passages, so the final ordering can use evidence beyond the relation text. It is a separate recipe and dataset scope from the historical three-dataset average above.
 
@@ -103,16 +103,17 @@ The pipeline first selects relations, then reranks their source passages togethe
 | HippoRAG 2 | 74.70% | 90.40% | 82.55% | ≈8.25 s (reference) |
 | Vector Graph RAG + GPT-6 Luna | 72.18% | 93.23% | 82.70% | 5.35 s |
 | Vector Graph RAG + DeepSeek V4.1 Flash | 73.20% | 93.48% | 83.34% | 10.97 s |
-| Vector Graph RAG + Jev · relations only | 69.30% | 90.78% | 80.04% | 2.28 s |
-| **Vector Graph RAG + Jev · two-stage** | **76.78%** | **95.35%** | **86.07%** | **3.13 s** |
+| Vector Graph RAG + Jev 1.13 · relations only | 69.30% | 90.78% | 80.04% | 2.28 s |
+| Vector Graph RAG + Jev 1.13 · two-stage | 76.78% | 95.35% | 86.07% | 3.13 s |
+| **Vector Graph RAG + Jev 1.14 · two-stage** | **77.73%** | **95.50%** | **86.61%** | **≈4.9 s** |
 
 ![Quality and additional model latency](assets/evaluation/quality-latency.png)
 
-Compared with relation-only Jev, the second stage adds **6.03 percentage points of average Recall@5** for about **0.85 seconds** more model-call time. It also improves average recall by **2.73 points over Vector Graph RAG + DeepSeek V4.1 Flash** and **3.52 points over HippoRAG 2**.
+Jev 1.14 improves average Recall@5 over 1.13 by **0.55 percentage points** (paired 95% bootstrap interval **+0.21 to +0.90**). The additional 1,800 queries beyond the 200-query pilot reproduce the gain: **+0.54 points**, interval **+0.20 to +0.88**. MuSiQue contributes most of it; 2Wiki alone has no clear gain, and average Recall@10 remains essentially unchanged (88.80% to 88.75%).
 
-On the plotted Pareto frontier, no alternative offers both at least the same recall and no more additional model-call time, with a strict improvement in either. Two-stage Jev is the frontier's highest-quality option; direct retrieval and relation-only Jev offer lower-latency choices at lower recall.
+On the plotted Pareto frontier, no alternative offers both at least the same recall and no more additional model-call time, with a strict improvement in either. Jev 1.14 is the highest-quality option among the evaluated configurations; Jev 1.13 remains a lower-latency frontier point at 86.07% / 3.13 s. This is a scoped best result, not a claim against every published system.
 
-The horizontal axis excludes embedding, retrieval and answer generation. Jev, Luna and Flash use measured API means; HippoRAG 2 retains a reference estimate. Test dates, concurrency and provider conditions differ, so the frontier is descriptive rather than a controlled end-to-end speed claim. Naive RAG adds no judgment call, hence zero additional time.
+The horizontal axis excludes embedding, retrieval and answer generation. Jev 1.14 uses a calibrated direct API estimate; the other Jev, Luna and Flash points use measured API means, and HippoRAG 2 retains a reference estimate. Test dates, concurrency and provider conditions differ, so the frontier is descriptive rather than a controlled end-to-end speed claim. Naive RAG adds no judgment call, hence zero additional time.
 
 See the [full evaluation report](../evaluation/jev/two-stage/README.md) for the Flash audit and chart reproduction, and [Jev score reproduction](../evaluation/jev/two-stage/README.md#recompute-without-api-calls). Enable the implementation through the [reranking guide](guides/reranking.md).
 

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/docs/guides/incremental-updates.md

@@ -176,6 +176,19 @@ A production ingestion job should track which source keys succeeded or failed.
 If a retry sees no matching passages during delete, the source may already have
 been removed by the previous attempt.
 
+## Repairing Existing Entity/Passage Links
+
+Every passage sharing a relation must appear in the relation's entities'
+`passage_ids`, and each passage must contain those entities in `entity_ids`.
+Missing links can leave orphan entities when sources are deleted.
+
+Upgrading to the shared-relation fix preserves these links for new writes.
+To repair a knowledge base indexed with missing links, call
+`rebuild_documents()` with the complete corpus and retain its source metadata.
+The rebuild replaces all data under the selected collection prefix, so include
+every source that should remain available. Upgrading the package alone does
+not repair stored adjacency metadata.
+
 ## ID Guidance
 
 Use `source` for the external object identity. Use `Document.id` only when you

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/docs/guides/reranking.md

@@ -36,7 +36,7 @@ rag = VectorGraphRAG(
 | `reranker_model` | Relation reranking, plus passage reranking for Jev | Inherits `llm_model` |
 | `answer_model` | Final answer generation | Inherits `llm_model` |
 
-A non-Jev `reranker_model` selects generative relation reranking. `jev` and `jev-*` names select the Jev two-stage pipeline; there is no strategy switch. The `jev` alias currently resolves to `jev-1.13.0` (a project recommendation, not an official latest-model alias). Pin the full name for reproducible runs. To follow the official stable release, use `reranker_model="jev-latest"`; `jev-preview` is also accepted. These are the [official aliases](https://docs.typesafe.ai/models), not the bare string `latest`. Moving aliases bypass the local response cache so a previous release cannot silently supply the result. Jev extraction and answer generation are not supported and are rejected during configuration.
+A non-Jev `reranker_model` selects generative relation reranking. `jev` and `jev-*` names select the Jev two-stage pipeline; there is no strategy switch. The `jev` alias currently resolves to `jev-1.13.0` (a project recommendation, not an official latest-model alias). Pin the full name for reproducible runs. The [Jev 1.14 evaluation](../../evaluation/jev/two-stage/README.md#jev-114-paired-comparison) adds a full benchmark result via OpenRouter; it does not change this runtime alias or the direct-service configuration. To follow the official stable release, use `reranker_model="jev-latest"`; `jev-preview` is also accepted. These are the [official aliases](https://docs.typesafe.ai/models), not the bare string `latest`. Moving aliases bypass the local response cache so a previous release cannot silently supply the result. Jev extraction and answer generation are not supported and are rejected during configuration.
 
 `retrieve()` skips answer generation; `query()` generates an answer from the same selected passages. Both still perform query entity extraction. The facade constructs all task clients, so their credentials must be configured even when only retrieval is requested.
 

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/docs/index.md

@@ -136,9 +136,9 @@ These benchmarks use Recall@5 and HippoRAG’s pre-extracted triplets. See [Eval
 
 ## Two-stage Jev reaches the quality–latency Pareto frontier
 
-**86.07% average Recall@5** on MuSiQue and 2Wiki (1,000 questions each)—the highest among the compared methods. Adding passage reranking to relation selection improves recall by **6.03 percentage points** over relation-only Jev, with recorded mean additional model-call time increasing from **2.28 to 3.13 seconds**.
+**Jev 1.14 achieves 86.61% average Recall@5** on MuSiQue and 2Wiki (1,000 questions each), the highest among the compared methods. Its estimated direct API mean is **approximately 4.9 seconds**. The same-query improvement over Jev 1.13 is **0.55 percentage points**, reproduced on the additional 1,800 queries beyond the pilot.
 
-With the displayed timings, this puts two-stage Jev on the **quality–latency Pareto frontier**. Jev, GPT-6 Luna and DeepSeek V4.1 Flash use measured API means; HippoRAG 2 retains a reference estimate. Measurements exclude the rest of the search pipeline and differ in test conditions. This two-dataset evaluation is separate from the historical three-dataset results.
+With the displayed timings, this puts two-stage Jev on the **quality–latency Pareto frontier**. Jev 1.14 uses a calibrated direct API estimate; the other Jev, Luna and Flash points use measured API means, and HippoRAG 2 retains a reference estimate. Measurements exclude the rest of the search pipeline and differ in test conditions. This two-dataset evaluation is separate from the historical three-dataset results.
 
 ![Quality and latency](assets/evaluation/quality-latency.png)
 

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/docs/reference/python-api.md

@@ -662,7 +662,10 @@ for passage in result.passages:
 
 #### `get_stats`
 
-Return collection statistics.
+Return the current entity, relation, and passage counts stored in Milvus for
+this instance's collection prefix. Counts include all sources and reflect
+completed imports, replacements and deletions, including writes from another
+instance connected to the same knowledge base.
 
 ```python
 def get_stats() -> dict
@@ -670,6 +673,14 @@ def get_stats() -> dict
 
 **Returns:** A dictionary with entity, relation, and passage counts.
 
+Each call performs a strongly consistent `count(*)` query on each existing
+collection. Only counts are returned; documents and vectors are not fetched.
+Existing collections must be loaded. Missing collections contribute zero;
+connection, permission and query failures raise an exception.
+
+The three counts are read separately. If another process is updating the graph
+at the same time, the counts can reflect different stages of that update.
+
 ```python
 stats = rag.get_stats()
 print(stats)

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/docs/reference/rest-api.md

@@ -587,6 +587,11 @@ curl -X POST "http://localhost:8000/query?graph_name=my_graph" \
 
 Returns aggregate statistics for a graph.
 
+Counts come from strongly consistent `count(*)` queries against the graph's
+three collections, which must be loaded. Missing collections contribute zero.
+Database query failures return HTTP 500. The collections are counted separately,
+so concurrent updates can be observed at different stages.
+
 **Path Parameters**
 
 | Parameter | Type | Description |

vector_graph_rag-0.4.1/evaluation/jev/README.md

@@ -0,0 +1,15 @@
+# Jev retrieval evaluation
+
+Jev evaluation began in September 2026; the full two-stage results below were added in October 2026. Jev 1.14 was added on October 16, 2026. The comparison includes GPT-6 Luna and DeepSeek V4.1 Flash.
+
+## Two-stage Jev reaches the quality–latency Pareto frontier
+
+Across **MuSiQue and 2Wiki, 1,000 questions each**, two-stage Jev 1.14 achieves **86.61% average Recall@5**, compared with **82.70% for GPT-6 Luna** and **83.34% for DeepSeek V4.1 Flash**. Their additional model-call means are **≈4.9 s** (estimated direct), **5.35 s** and **10.97 s**, respectively. Different test conditions and external reference points limit direct speed comparisons.
+
+![Quality and latency](../../docs/assets/evaluation/quality-latency.png)
+
+The two-stage pipeline adds passage reranking after relation selection. Jev 1.14 improves average Recall@5 by **0.55 percentage points** over 1.13 on the same 2,000 questions; the additional 1,800 queries beyond the pilot confirm a **0.54-point** improvement. Most of the gain comes from MuSiQue; Recall@10 is essentially unchanged. Both versions remain on the observed Pareto frontier: 1.14 is the highest-quality point, while 1.13 retains its recorded 3.13-second timing. These are separate service runs, not a controlled speed comparison.
+
+See the [full evaluation report](two-stage/README.md) for dataset scores, measured timing scope, Flash audit and chart reproduction. Enable the pipeline with `reranker_model="jev"`; the [reranking guide](../../docs/guides/reranking.md) covers configuration.
+
+The [frozen Jev recipe and score reproduction](two-stage/README.md) support these results. This two-dataset comparison is separate from the project's historical three-dataset average.

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/evaluation/jev/two-stage/README.md

@@ -1,6 +1,6 @@
 # Two-stage Jev reaches the quality–latency Pareto frontier
 
-Jev evaluation began in September 2026; the full two-stage results below were added in October 2026. The Luna/Flash comparison was updated on October 14, 2026. MuSiQue and 2Wiki each contain 1,000 questions; averages weight the two datasets equally. The generative rerankers are GPT-6 Luna and official DeepSeek V4.1 Flash.
+Jev evaluation began in September 2026; the full two-stage results below were added in October 2026. The Luna/Flash comparison was updated on October 14, 2026; Jev 1.14 was added on October 16, 2026. MuSiQue and 2Wiki each contain 1,000 questions; averages weight the two datasets equally. The generative rerankers are GPT-6 Luna and official DeepSeek V4.1 Flash.
 
 ![Quality and additional model-call latency](../../../docs/assets/evaluation/quality-latency.png)
 
@@ -12,16 +12,42 @@ Jev evaluation began in September 2026; the full two-stage results below were ad
 | HippoRAG 2 | 74.70% | 90.40% | 82.55% | ≈8.25 s (reference) |
 | Vector Graph RAG + GPT-6 Luna | 72.18% | 93.23% | 82.70% | 5.35 s |
 | Vector Graph RAG + DeepSeek V4.1 Flash | 73.20% | 93.48% | 83.34% | 10.97 s |
-| Vector Graph RAG + Jev · relations only | 69.30% | 90.78% | 80.04% | 2.28 s |
-| **Vector Graph RAG + Jev · two-stage** | **76.78%** | **95.35%** | **86.07%** | **3.13 s** |
+| Vector Graph RAG + Jev 1.13 · relations only | 69.30% | 90.78% | 80.04% | 2.28 s |
+| Vector Graph RAG + Jev 1.13 · two-stage | 76.78% | 95.35% | 86.07% | 3.13 s |
+| **Vector Graph RAG + Jev 1.14 · two-stage** | **77.73%** | **95.50%** | **86.61%** | **≈4.9 s** |
 
 Luna uses **medium** reasoning: xhigh had similar recall (82.53% versus 82.70%) but higher mean latency (8.86 s versus 5.35 s); Flash uses **low** because high produced very long reasoning and exhausted the output limit on some questions.
 
-Two-stage Jev has the highest recall among these methods: 6.03 percentage points above relation-only Jev, 3.36 above Luna, and 2.73 above Flash. It is the highest-quality non-dominated point using the displayed timings. The chart computes that frontier from the data; this is not a controlled end-to-end speed comparison.
+Two-stage Jev 1.14 has the highest recall among the evaluated configurations: 86.61% average Recall@5. Both Jev versions are non-dominated using the displayed timings: 1.14 extends the frontier to higher quality; 1.13 remains at lower recorded latency. The chart computes the frontier from the data; this is not a controlled end-to-end speed comparison.
+
+## Jev 1.14 paired comparison
+
+All 2,000 questions completed (1,000 per dataset), retaining the frozen graph, candidates, prompts, ordering and scoring rules. Two hundred exact-input pilot results were reused after hash and score checks; the other 1,800 were requested once successfully. Only `typesafe/jev-1.14` was called through OpenRouter; every response identified `typesafe/jev-1.14-20261014`.
+
+| Scope | R@5 improvement over 1.13 | Paired 95% bootstrap interval |
+|---|---:|---:|
+| All 2,000 queries | +0.546 pp | +0.208 to +0.896 pp |
+| Additional 1,800 queries | +0.537 pp | +0.199 to +0.884 pp |
+| MuSiQue, all 1,000 | +0.942 pp | +0.317 to +1.617 pp |
+| 2Wiki, all 1,000 | +0.150 pp | −0.050 to +0.400 pp |
+
+The small R@5 gain reproduces beyond the pilot and is concentrated in MuSiQue. Across all queries, 64 improve, 1,901 tie and 35 regress. R@10 is essentially unchanged: 88.796% to 88.746% (difference −0.050 pp, interval −0.267 to +0.179). The intervals use 5,000 paired, dataset-stratified bootstrap replicates with fixed seed 20261016. The additional queries were not part of the pilot comparison, but the historical benchmark is not an untouched holdout.
+
+The new phase took 54.46 minutes and 10,735 network attempts, including three recorded failures. Across pilot and full run, 11,883 successful payload hashes were verified with no successful resends. New reported charges were $6.8382, or $7.0253 including unknown failed-request reserves; reused pilot charges were $0.7261. Charges are API-reported values, not an account invoice. No other decision models were included in the full run.
+
+| Timing cohort | Mean | P50 | P90 |
+|---|---:|---:|---:|
+| Reused 200-query pilot | 5.81 s | 3.17 s | 20.71 s |
+| Additional 1,800 queries | 5.76 s | 3.22 s | 21.41 s |
+| Full 2,000 queries (OpenRouter) | 5.77 s | 3.21 s | 21.40 s |
+
+These are sums of successful HTTP durations per query, not cache-read timings; failed-attempt waits are excluded and retained separately in the experiment ledger. Jev 1.14 used four concurrent queries, serial batches within each query, and at most 64 judgments per request. Jev 1.13 used the earlier direct service and different batching. The paired results compare the fixed application recipe across version/service runs; they do not isolate model weights from all service effects.
+
+The chart estimates official-direct Jev 1.14 latency at **≈4.9 s**, scaling the recorded 5.77 s by paired direct/OpenRouter timings for 12 size-stratified queries (67 identical payload pairs); see the [calibration record](jev114-latency-calibration.json) and [offline calculation](calibrate_latency.py).
 
 ## Measurement scope
 
-The horizontal axis excludes embedding, retrieval and answer generation. Zero means no extra model call. Jev timings use the recorded timing subsets: equal-weight means of per-query request-duration sums from 500 MuSiQue and 900 2Wiki queries, with three concurrent queries. Luna and Flash use measured full-evaluation API means, not the older generative-model timing estimates. Network, provider queueing, concurrency and measurement dates differ between runs. HippoRAG 2 retains an approximate reference time; its scores and the ColBERTv2/NV-Embed-v2 points are published aggregates rather than new runs. See the [comparison provenance](#comparison-basis).
+The horizontal axis excludes embedding, retrieval and answer generation. Zero means no extra model call. Jev 1.13 timings use the recorded timing subsets: equal-weight means of per-query request-duration sums from 500 MuSiQue and 900 2Wiki queries, with three concurrent queries. Luna and Flash use measured full-evaluation API means, not the older generative-model timing estimates. Network, provider queueing, concurrency and measurement dates differ between runs. HippoRAG 2 retains an approximate reference time; its scores and the ColBERTv2/NV-Embed-v2 points are published aggregates rather than new runs. See the [comparison provenance](#comparison-basis).
 
 The frozen candidates, input prompts, relation order and historical scoring are retained for the new generative runs. Their selected relations expand to source passages in order, followed by naive retrieval to fill ten passages. Only message output text and the `useful_relations` JSON field are scored; reasoning text and field aliases are not used. MuSiQue matches distinct full passages and 2Wiki distinct titles. Jev's two-stage recipe is described in the [frozen recipe](#frozen-recipe).
 
@@ -43,7 +69,7 @@ Reported usage was 8,092,904 input tokens (769,664 cached) and 4,181,868 output
 
 ## Saved data and chart reproduction
 
-`comparison.csv` contains per-dataset Recall@5/10 aggregates; `flash-summary.json` contains the Flash audit, usage and timing statistics; `chart-data.json` freezes the plotted coordinates. These aggregates reproduce the figure, not the underlying model calls or independent per-query scoring. Raw responses and credentials are not included in the repository. `results.json.gz` and `summarize.py` provide frozen evidence fingerprints and score replay for the naive and both Jev methods; `summary.json` contains their recomputed scores.
+`comparison.csv` contains per-dataset Recall@5/10 aggregates; `flash-summary.json` contains the Flash audit, usage and timing statistics; `chart-data.json` freezes the plotted coordinates. These aggregates reproduce the figure, not the underlying model calls or independent per-query scoring. Raw responses and credentials are not included in the repository. `results.json.gz` and `summarize.py` provide frozen evidence fingerprints and score replay for the naive and both Jev methods; `summary.json` contains their recomputed scores. `jev114-audit.json` records the version comparison, timing, usage and validation; `compare_versions.py` replays its paired differences and bootstrap intervals from the fingerprint records.
 
 ```bash
 uv run --offline --no-project --python 3.12 --with matplotlib evaluation/jev/two-stage/plot.py
@@ -53,7 +79,7 @@ The offline command requires matplotlib to be cached locally. The figure omits r
 
 ## Frozen recipe
 
-- Model: `jev-1.13.0`; exact prompts are in [`llm/jev.py`](../../../src/vector_graph_rag/llm/jev.py).
+- Models: historical `jev-1.13.0` and OpenRouter `typesafe/jev-1.14`; exact prompts are in [`llm/jev.py`](../../../src/vector_graph_rag/llm/jev.py).
 - Score the full candidate relation table with shared context; retain the top 64, breaking ties by original order.
 - Expand relations to their source passages round-robin, preserving each relation's passage order.
 - Alternate graph passages with the dense retrieval top 10; deduplicate full text and cap the union at 16.
@@ -71,9 +97,10 @@ External aggregates in the evaluation report come from [HippoRAG Tables 2–3](h
 
 ```bash
 uv run --no-project python evaluation/jev/two-stage/summarize.py
+uv run --offline --no-project --python 3.12 --with numpy evaluation/jev/two-stage/compare_versions.py
 ```
 
-`results.json.gz` holds 6,000 method/query records for naive retrieval, relation-only Jev and two-stage Jev, including ordered SHA-256 evidence fingerprints and metrics. `manifest.json` identifies the 2,000 queries and dataset file hashes. The script independently checks saved Recall@5/10 against evidence fingerprints and verifies the expected query set for each method. Fingerprints do not reconstruct candidate generation or model calls; raw model caches and local databases are not included.
+`results.json.gz` holds 8,000 method/query records for naive retrieval, relation-only Jev 1.13 and two-stage Jev 1.13/1.14, including ordered SHA-256 evidence fingerprints and metrics. `manifest.json` identifies the 2,000 queries and dataset file hashes. The script independently checks saved Recall@5/10 against evidence fingerprints and verifies the expected query set for each method. Fingerprints do not reconstruct candidate generation or model calls; raw model caches and local databases are not included.
 
 ## Run against your index
 
@@ -86,6 +113,6 @@ uv run python evaluation/evaluate.py --dataset musique --data-dir evaluation/dat
   --reranker-model jev-1.13.0 --method graph --top-k 10 --max-samples 10
 ```
 
-Repeat with `--dataset 2wikimultihopqa`. Remove `--max-samples` for all 1,000 questions. This exercises the shipped two-stage implementation on the configured index; it does not reconstruct the frozen hybrid candidate pools. Indexing, embeddings and model calls can incur charges. Errors must be inspected in evaluation logs.
+Repeat with `--dataset 2wikimultihopqa`. Remove `--max-samples` for all 1,000 questions. This example uses the shipped direct-service 1.13 configuration; the 1.14 result above used the OpenRouter Decisions endpoint. It exercises the shipped two-stage implementation on the configured index; it does not reconstruct the frozen hybrid candidate pools. Indexing, embeddings and model calls can incur charges. Errors must be inspected in evaluation logs.
 
 Earlier model comparisons and their source artifacts are preserved in the [evaluation snapshot](https://github.com/zilliztech/vector-graph-rag/tree/e32c077ccde8c577f9335b7966ebf22d46e18334/evaluation/jev).

vector_graph_rag-0.4.1/evaluation/jev/two-stage/calibrate_latency.py

@@ -0,0 +1,19 @@
+"""Recompute the direct-service estimate from archived paired timings."""
+import json
+import math
+from pathlib import Path
+
+root = Path(__file__).parent
+data = json.loads((root / 'jev114-latency-calibration.json').read_text())
+weighted = 0.0
+for stratum in data['strata']:
+    pairs = [p for p in data['paired_timings']
+             if p['dataset'] == stratum['dataset']
+             and p['index'] in stratum['sample_indices']]
+    ratio = sum(p['direct_seconds'] for p in pairs) / sum(p['proxy_seconds'] for p in pairs)
+    assert math.isclose(ratio, stratum['ratio'], abs_tol=1e-12)
+    weighted += stratum['population_queries'] / 2000 * stratum['population_proxy_mean'] * ratio
+assert math.isclose(weighted, data['estimated_full_direct_seconds'], abs_tol=1e-12)
+chart = json.loads((root / 'chart-data.json').read_text())
+assert math.isclose(weighted, chart['points']['jev_two_114']['latency'], abs_tol=1e-12)
+print(f'Official-direct estimate: {weighted:.6f} s; {len(data["paired_timings"])} verified timing pairs')

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/evaluation/jev/two-stage/chart-data.json

@@ -31,6 +31,13 @@
     "flash": {
       "quality": 83.33749999999999,
       "latency": 10.970034438348375
+    },
+    "jev_two_114": {
+      "quality": 86.6125,
+      "latency": 4.87210045028051,
+      "latency_kind": "estimated_official_direct",
+      "measured_openrouter_latency": 5.769053221371956,
+      "latency_source": "jev114-latency-calibration.json"
     }
   },
   "naive_points": [
@@ -65,6 +72,11 @@
       "jev_two",
       3.1327848465049435,
       86.06666666666666
+    ],
+    [
+      "jev_two_114",
+      4.87210045028051,
+      86.6125
     ]
   ]
 }

vector_graph_rag-0.4.1/evaluation/jev/two-stage/compare_versions.py

@@ -0,0 +1,52 @@
+"""Verify the saved paired bootstrap intervals using fingerprint-only records."""
+
+import gzip
+import json
+from pathlib import Path
+
+import numpy as np
+
+ROOT = Path(__file__).parent
+rows = json.loads(gzip.decompress((ROOT / "results.json.gz").read_bytes()))
+old = {(r["dataset"], r["id"]): r for r in rows if r["method"] == "jev_two"}
+new = [r for r in rows if r["method"] == "jev_two_114"]
+audit = json.loads((ROOT / "jev114-audit.json").read_text())
+assert len(new) == len(old) == 2000
+for r in new:
+    assert set(r["gold"]) == set(old[r["dataset"], r["id"]]["gold"])
+for expected in audit["results"]:
+    selected = [
+        r
+        for r in new
+        if (expected["cohort"] == "all2000" or r["cohort"] == expected["cohort"])
+        and (expected["dataset"] == "combined" or r["dataset"] == expected["dataset"])
+    ]
+    assert len(selected) == expected["n"]
+    for k in (5, 10):
+        groups = [
+            np.array(
+                [
+                    r[f"r{k}"] - old[r["dataset"], r["id"]][f"r{k}"]
+                    for r in selected
+                    if r["dataset"] == ds
+                ]
+            )
+            for ds in ("musique", "2wikimultihopqa")
+        ]
+        groups = [g for g in groups if len(g)]
+        # Preserve the archived per-query order for exact bootstrap replay.
+        rng = np.random.default_rng(20261016)
+        bootstrap = sum(
+            rng.choice(g, size=(5000, len(g)), replace=True).mean(axis=1) for g in groups
+        ) / len(groups)
+        delta = float(np.mean([g.mean() for g in groups]) * 100)
+        interval = np.quantile(bootstrap, [0.025, 0.975]) * 100
+        assert abs(delta - expected[f"delta_r{k}_pp"]) < 1e-10
+        assert np.max(np.abs(interval - expected[f"ci95_r{k}_pp"])) < 1e-10
+        print(
+            expected["cohort"],
+            expected["dataset"],
+            f"R@{k}",
+            f"{delta:+.3f}pp",
+            interval.round(3).tolist(),
+        )

{vector_graph_rag-0.4.0 → vector_graph_rag-0.4.1}/evaluation/jev/two-stage/comparison.csv

@@ -14,3 +14,6 @@ jev_two,average,2000,0.8606666666666667,0.8879583333333333
 Luna medium,musique,1000,0.7218333333333333,0.78675
 Luna medium,2wikimultihopqa,1000,0.93225,0.94525
 Luna medium,average,2000,0.8270416666666667,0.866
+jev_two_114,musique,1000,0.77725,0.8196666666666667
+jev_two_114,2wikimultihopqa,1000,0.955,0.95525
+jev_two_114,average,2000,0.8661249999999999,0.8874583333333333