raglite-toolkit 1.2.1__tar.gz → 1.2.2__tar.gz

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/.github/workflows/pr-verify.yml

@@ -34,3 +34,12 @@ jobs:
 
       - name: Run tests
         run: python -m pytest tests/ -q
+
+      - name: Check out TypeScript SDK for shared fixtures
+        uses: actions/checkout@v4
+        with:
+          repository: creatorpiyush/raglite
+          path: .ts-sdk
+
+      - name: Check shared fixtures match the TypeScript SDK
+        run: python scripts/sync_shared_fixtures.py --check --source .ts-sdk

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/.gitignore

@@ -42,3 +42,4 @@ coverage.xml
 
 # Logs
 *.log
+.ts-sdk/

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/ARCHITECTURE.md

@@ -126,9 +126,9 @@ graph TD
 ### 4.1 Ingestion & Indexing
 1. `doc.build()` invokes the appropriate `BaseLoader` based on file extension (`.pdf`, `.txt`, `.json`, `.md`, `.docx`).
 2. Calculates SHA-256 hash of raw document content.
-3. Checks existing `IndexMetadata` in `VectorStore`. If hash matches, skips re-indexing.
+3. Checks existing `IndexMetadata` in `VectorStore`. The cached index is reused when the index format version, content hash, chunk size, overlap and embedding provider/model all match (URL sources are hashed by their fetched text).
 4. If hash differs or force rebuild requested:
-   - `RecursiveCharacterTextSplitter` chunks text (default size 1000, overlap 200).
+   - `RecursiveChunker` splits text into word-based chunks (default 500 words, 50-word overlap).
    - `EmbeddingFactory` generates normalized vectors for each chunk.
    - `VectorStore.add()` saves chunks and `VectorStore.save_index_metadata()` persists index metadata.
 
@@ -162,27 +162,32 @@ Built using **FastAPI** framework for async capabilities, automatic OpenAPI docs
 ```python
 from abc import ABC, abstractmethod
 from typing import List, Optional
-from raglite.types import IndexMetadata, SearchResult, StoredChunk
+from raglite.types import IndexMetadata, StoredChunk
+from raglite.vectordb.base import VectorSearchHit
 
 class VectorStore(ABC):
+    @property
     @abstractmethod
-    def load(self, doc_id: str) -> None: ...
+    def namespace(self) -> str: ...
 
     @abstractmethod
-    def reset(self, doc_id: str) -> None: ...
+    def load(self) -> None: ...
 
     @abstractmethod
-    def add(self, doc_id: str, chunks: List[StoredChunk]) -> None: ...
+    def reset(self) -> None: ...
 
     @abstractmethod
-    def search(self, doc_id: str, query_vector: List[float], top_k: int, min_score: Optional[float] = None) -> List[SearchResult]: ...
+    def add(self, chunks: List[StoredChunk]) -> None: ...
 
     @abstractmethod
-    def count(self, doc_id: str) -> int: ...
+    def search(self, embedding: List[float], top_k: int) -> List[VectorSearchHit]: ...
 
     @abstractmethod
-    def save_index_metadata(self, doc_id: str, meta: IndexMetadata) -> None: ...
+    def count(self) -> int: ...
 
     @abstractmethod
-    def read_index_metadata(self, doc_id: str) -> Optional[IndexMetadata]: ...
+    def save_index_metadata(self, metadata: IndexMetadata) -> None: ...
+
+    @abstractmethod
+    def read_index_metadata(self) -> Optional[IndexMetadata]: ...
 ```

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/CHANGELOG.md

@@ -2,6 +2,18 @@
 
 All notable changes to this project will be documented in this file.
 
+## [1.2.2] - Unreleased
+
+### Changed
+- **Upgrades keep cached indexes:** The build cache is now keyed on an index format version (`formatVersion` in `IndexMetadata`) instead of the package version, so upgrading RAGLite no longer re-embeds every index. Indexes built by 1.2.1 are reused as-is; indexes from older releases are rebuilt once.
+
+### Fixed
+- **Chunker parity with the TypeScript SDK:** Words are now split on exactly the whitespace characters JavaScript treats as whitespace. Previously text containing a byte order mark (U+FEFF), U+0085 or U+001C–U+001F was chunked differently from the TypeScript SDK. Existing indexes are not rebuilt for this; pass `rebuild=True` if your sources contain those characters.
+- **Docs:** ARCHITECTURE.md now describes the word-based `RecursiveChunker` (500 words, 50 overlap) and the actual `VectorStore` interface. The README no longer lists LanceDB, which is TypeScript-only.
+
+### Internal
+- Added cross-SDK fixtures (`tests/fixtures/shared/`, copied from the TypeScript SDK with `scripts/sync_shared_fixtures.py`) that pin chunking and hashing output. CI fails if the copy is out of date.
+
 ## [1.2.1] - 2026-10-02
 
 ### Fixed

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: raglite-toolkit
-Version: 1.2.1
+Version: 1.2.2
 Summary: Build semantic search, multi-provider question answering, and REST APIs over your documents in a few lines of Python.
 Project-URL: Homepage, https://github.com/creatorpiyush/raglite-py
 Project-URL: Repository, https://github.com/creatorpiyush/raglite-py
@@ -172,7 +172,7 @@ print()
 
 ## Pluggable Vector Databases
 
-`raglite` supports pluggable vector stores (Memory, Qdrant, Pinecone, LanceDB, or custom subclasses):
+`raglite` supports pluggable vector stores (Memory, Qdrant, Pinecone, or custom subclasses; LanceDB is currently TypeScript-only):
 
 ### Memory Store (Default)
 ```python
@@ -336,7 +336,7 @@ Document("./policy.pdf", {
 
 ## How Caching Works
 
-Every `build()` call fingerprints the source file with a **SHA-256 content hash** and persists it alongside the vectors. The cached index is reused only if **all** of the following match the stored index:
+Every `build()` call fingerprints the source (file bytes, or the fetched text for URLs) with a **SHA-256 content hash** and persists it alongside the vectors. The cached index is reused only if **all** of the following match the stored index:
 
 | Factor | Triggers rebuild if changed |
 |--------|-----------------------------|
@@ -344,7 +344,7 @@ Every `build()` call fingerprints the source file with a **SHA-256 content hash*
 | Chunk size | `chunkSize` changed |
 | Overlap | `overlap` changed |
 | Embedding provider/model | Provider or model string changed |
-| Library version | Package version bumped |
+| Index format | Stored index layout changed by a release (rare; ordinary upgrades reuse the index) |
 
 Pass `rebuild=True` to `build()` to force a fresh index regardless.
 
@@ -360,7 +360,7 @@ Each document is stored under `.raglite/<sha256-prefix>/`, so multiple documents
 from raglite.vectordb.base import VectorStore
 
 class MyVectorStore(VectorStore):
-    # Implement: load, reset, add, search, count,
+    # Implement: namespace (property), load, reset, add, search, count,
     #            save_index_metadata, read_index_metadata
     ...
 ```

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/README.md

@@ -126,7 +126,7 @@ print()
 
 ## Pluggable Vector Databases
 
-`raglite` supports pluggable vector stores (Memory, Qdrant, Pinecone, LanceDB, or custom subclasses):
+`raglite` supports pluggable vector stores (Memory, Qdrant, Pinecone, or custom subclasses; LanceDB is currently TypeScript-only):
 
 ### Memory Store (Default)
 ```python
@@ -290,7 +290,7 @@ Document("./policy.pdf", {
 
 ## How Caching Works
 
-Every `build()` call fingerprints the source file with a **SHA-256 content hash** and persists it alongside the vectors. The cached index is reused only if **all** of the following match the stored index:
+Every `build()` call fingerprints the source (file bytes, or the fetched text for URLs) with a **SHA-256 content hash** and persists it alongside the vectors. The cached index is reused only if **all** of the following match the stored index:
 
 | Factor | Triggers rebuild if changed |
 |--------|-----------------------------|
@@ -298,7 +298,7 @@ Every `build()` call fingerprints the source file with a **SHA-256 content hash*
 | Chunk size | `chunkSize` changed |
 | Overlap | `overlap` changed |
 | Embedding provider/model | Provider or model string changed |
-| Library version | Package version bumped |
+| Index format | Stored index layout changed by a release (rare; ordinary upgrades reuse the index) |
 
 Pass `rebuild=True` to `build()` to force a fresh index regardless.
 
@@ -314,7 +314,7 @@ Each document is stored under `.raglite/<sha256-prefix>/`, so multiple documents
 from raglite.vectordb.base import VectorStore
 
 class MyVectorStore(VectorStore):
-    # Implement: load, reset, add, search, count,
+    # Implement: namespace (property), load, reset, add, search, count,
     #            save_index_metadata, read_index_metadata
     ...
 ```

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/pyproject.toml

@@ -8,7 +8,7 @@ packages = ["src/raglite"]
 
 [project]
 name = "raglite-toolkit"
-version = "1.2.1"
+version = "1.2.2"
 description = "Build semantic search, multi-provider question answering, and REST APIs over your documents in a few lines of Python."
 readme = "README.md"
 requires-python = ">=3.10"

raglite_toolkit-1.2.2/scripts/sync_shared_fixtures.py

@@ -0,0 +1,54 @@
+"""Copy or check the cross-SDK fixtures shared with the TypeScript SDK.
+
+The TypeScript repo (raglite) is the source of truth: it generates
+tests/fixtures/shared/*.json with scripts/generate-shared-fixtures.ts. This
+repo keeps an identical copy so both SDKs are tested against the same
+expected outputs.
+
+    python scripts/sync_shared_fixtures.py            # copy from ../raglite
+    python scripts/sync_shared_fixtures.py --check    # exit 1 if out of date
+    python scripts/sync_shared_fixtures.py --source /path/to/raglite
+"""
+import argparse
+import filecmp
+import shutil
+import sys
+from pathlib import Path
+
+REPO_ROOT = Path(__file__).resolve().parent.parent
+TARGET = REPO_ROOT / "tests" / "fixtures" / "shared"
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser(description=__doc__.splitlines()[0])
+    parser.add_argument("--source", type=Path, default=REPO_ROOT.parent / "raglite")
+    parser.add_argument("--check", action="store_true")
+    args = parser.parse_args()
+
+    source = args.source / "tests" / "fixtures" / "shared"
+    files = sorted(source.glob("*.json"))
+    if not files:
+        print(f"No shared fixtures found in {source}", file=sys.stderr)
+        return 1
+
+    stale = [f.name for f in files if not (TARGET / f.name).exists() or not filecmp.cmp(f, TARGET / f.name, shallow=False)]
+    extra = sorted({p.name for p in TARGET.glob("*.json")} - {f.name for f in files})
+
+    if args.check:
+        for name in stale:
+            print(f"out of date: {name}", file=sys.stderr)
+        for name in extra:
+            print(f"not in source: {name}", file=sys.stderr)
+        return 1 if stale or extra else 0
+
+    TARGET.mkdir(parents=True, exist_ok=True)
+    for f in files:
+        shutil.copyfile(f, TARGET / f.name)
+    for name in extra:
+        (TARGET / name).unlink()
+    print(f"Synced {len(files)} fixture(s) from {source}")
+    return 0
+
+
+if __name__ == "__main__":
+    sys.exit(main())

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/src/raglite/chunking/recursive.py

@@ -4,17 +4,24 @@ from typing import List
 from ..errors import ChunkingError
 from .base import BaseChunker
 
+# The exact set JavaScript's \s matches. Python's \s differs (it includes
+# \x1c-\x1f and \x85 but not \ufeff), which would make the two SDKs chunk the
+# same text differently.
+_JS_WHITESPACE = re.compile(
+    "[\t\n\v\f\r \u00a0\u1680\u2000-\u200a\u2028\u2029\u202f\u205f\u3000\ufeff]+"
+)
+
 
 class RecursiveChunker(BaseChunker):
     def split(self, text: str) -> List[str]:
-        if not text.strip():
+        if not _JS_WHITESPACE.sub("", text):
             return []
         if self.overlap >= self.chunk_size:
             raise ChunkingError(
                 f"overlap ({self.overlap}) must be smaller than chunkSize ({self.chunk_size})"
             )
 
-        words = [w for w in re.split(r"\s+", text) if w]
+        words = [w for w in _JS_WHITESPACE.split(text) if w]
         if len(words) <= self.chunk_size:
             return [" ".join(words)]
 

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/src/raglite/constants.py

@@ -5,7 +5,16 @@ try:
     PACKAGE_VERSION = importlib.metadata.version("raglite-toolkit")
 except importlib.metadata.PackageNotFoundError:
     PACKAGE_NAME = "raglite-toolkit"
-    PACKAGE_VERSION = "1.2.1"  # local development fallback
+    PACKAGE_VERSION = "1.2.2"  # local development fallback
+
+# Version of the stored index layout (chunking, ids, payloads). The build cache
+# is keyed on this instead of PACKAGE_VERSION so upgrading the package does not
+# re-embed every index. Bump it only when a change makes existing indexes
+# incompatible.
+INDEX_FORMAT_VERSION = 1
+
+# Releases that wrote format-1 indexes before ``formatVersion`` was recorded.
+LEGACY_FORMAT_1_VERSIONS = frozenset({"1.2.1"})
 
 
 SUPPORTED_EXTENSIONS = {".pdf", ".txt", ".json", ".md", ".markdown", ".docx"}

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/src/raglite/core/document.py

@@ -4,7 +4,7 @@ from typing import Any, Dict, Generator, Optional, Union
 
 from ..chunking import RecursiveChunker
 from ..config import DocumentOptions, ResolvedConfig, resolve_config
-from ..constants import PACKAGE_VERSION
+from ..constants import INDEX_FORMAT_VERSION, LEGACY_FORMAT_1_VERSIONS, PACKAGE_VERSION
 from ..embeddings import Embedder, create_embedder
 from ..errors import FileNotIndexedError, LoaderError, RagLiteError
 from ..llm import generate_answer, stream_answer
@@ -168,6 +168,7 @@ class Document:
         from datetime import timezone
         metadata = IndexMetadata(
             version=PACKAGE_VERSION,
+            formatVersion=INDEX_FORMAT_VERSION,
             source=self.file_path,
             sourceHash=source_hash,
             chunkSize=c_size,
@@ -361,7 +362,7 @@ class Document:
         overlap: int,
         embeddings_config: Any,
     ) -> bool:
-        if existing.version != PACKAGE_VERSION:
+        if index_format_version(existing) != INDEX_FORMAT_VERSION:
             return False
         if existing.sourceHash != source_hash:
             return False
@@ -378,6 +379,13 @@ class Document:
         return True
 
 
+def index_format_version(metadata: IndexMetadata) -> Optional[int]:
+    """Indexes written before ``formatVersion`` existed are identified by package version."""
+    if metadata.formatVersion is not None:
+        return metadata.formatVersion
+    return 1 if metadata.version in LEGACY_FORMAT_1_VERSIONS else None
+
+
 def _query_embeddings_config(
     existing: IndexMetadata, configured: EmbeddingProviderConfig
 ) -> EmbeddingProviderConfig:

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/src/raglite/types.py

@@ -96,7 +96,9 @@ class AnswerResult(BaseModel):
 class IndexMetadata(BaseModel):
     model_config = ConfigDict(populate_by_name=True, extra="allow")
 
-    version: str
+    version: str  # package version that built the index (informational)
+    # Index layout version; see INDEX_FORMAT_VERSION. None on indexes built before 1.2.2.
+    formatVersion: Optional[int] = Field(default=None, alias="formatVersion")
     source: str
     sourceHash: str = Field(..., alias="sourceHash")
     chunkSize: int = Field(..., alias="chunkSize")

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/src/raglite/vectordb/pinecone.py

@@ -126,7 +126,8 @@ class PineconeVectorStore(VectorStore):
     def save_index_metadata(self, metadata: IndexMetadata) -> None:
         dim = metadata.embeddingDimensions
         zero_vec = [0.0] * dim
-        payload = metadata.model_dump(by_alias=True)
+        # Pinecone rejects null metadata values.
+        payload = metadata.model_dump(by_alias=True, exclude_none=True)
         payload["isMetadata"] = True
         self._request(
             "POST",
@@ -154,6 +155,9 @@ class PineconeVectorStore(VectorStore):
         try:
             return IndexMetadata(
                 version=m.get("version", ""),
+                formatVersion=(
+                    int(m["formatVersion"]) if m.get("formatVersion") is not None else None
+                ),
                 source=m.get("source", ""),
                 sourceHash=m.get("sourceHash", ""),
                 chunkSize=int(m.get("chunkSize", 0)),

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/src/raglite/vectordb/qdrant.py

@@ -197,6 +197,7 @@ class QdrantVectorStore(VectorStore):
         try:
             return IndexMetadata(
                 version=payload.get("version", ""),
+                formatVersion=payload.get("formatVersion"),
                 source=payload.get("source", ""),
                 sourceHash=payload.get("sourceHash", ""),
                 chunkSize=payload.get("chunkSize", 0),

raglite_toolkit-1.2.2/tests/fixtures/shared/chunker.json

@@ -0,0 +1,100 @@
+[
+  {
+    "name": "empty",
+    "text": "",
+    "chunkSize": 500,
+    "overlap": 50,
+    "chunks": []
+  },
+  {
+    "name": "whitespace only",
+    "text": " \n\t ",
+    "chunkSize": 500,
+    "overlap": 50,
+    "chunks": []
+  },
+  {
+    "name": "short text",
+    "text": "Refunds are issued within 30 days.",
+    "chunkSize": 500,
+    "overlap": 50,
+    "chunks": ["Refunds are issued within 30 days."]
+  },
+  {
+    "name": "sliding window",
+    "text": "w1 w2 w3 w4 w5 w6 w7 w8 w9 w10 w11 w12",
+    "chunkSize": 5,
+    "overlap": 2,
+    "chunks": ["w1 w2 w3 w4 w5", "w4 w5 w6 w7 w8", "w7 w8 w9 w10 w11", "w10 w11 w12"]
+  },
+  {
+    "name": "last window ends exactly",
+    "text": "w1 w2 w3 w4 w5 w6 w7 w8",
+    "chunkSize": 5,
+    "overlap": 2,
+    "chunks": ["w1 w2 w3 w4 w5", "w4 w5 w6 w7 w8"]
+  },
+  {
+    "name": "no overlap",
+    "text": "w1 w2 w3 w4 w5 w6 w7 w8 w9 w10",
+    "chunkSize": 4,
+    "overlap": 0,
+    "chunks": ["w1 w2 w3 w4", "w5 w6 w7 w8", "w9 w10"]
+  },
+  {
+    "name": "mixed whitespace collapses",
+    "text": "a\n\nb\t c   d\r\ne",
+    "chunkSize": 500,
+    "overlap": 50,
+    "chunks": ["a b c d e"]
+  },
+  {
+    "name": "no-break and ideographic spaces",
+    "text": "a b　c d",
+    "chunkSize": 2,
+    "overlap": 0,
+    "chunks": ["a b", "c d"]
+  },
+  {
+    "name": "byte order mark",
+    "text": "alpha betagamma",
+    "chunkSize": 500,
+    "overlap": 50,
+    "chunks": ["alpha beta gamma"]
+  },
+  {
+    "name": "next line control",
+    "text": "alpha…beta",
+    "chunkSize": 500,
+    "overlap": 50,
+    "chunks": ["alpha…beta"]
+  },
+  {
+    "name": "file separator control",
+    "text": "alpha\u001cbeta",
+    "chunkSize": 500,
+    "overlap": 50,
+    "chunks": ["alpha\u001cbeta"]
+  },
+  {
+    "name": "accents and emoji",
+    "text": "café naïve 返金 🚀 ok",
+    "chunkSize": 2,
+    "overlap": 1,
+    "chunks": ["café naïve", "naïve 返金", "返金 🚀", "🚀 ok"]
+  },
+  {
+    "name": "unspaced CJK stays one word",
+    "text": "返金は三十日以内に行われます。配送には二営業日かかります。",
+    "chunkSize": 5,
+    "overlap": 1,
+    "chunks": ["返金は三十日以内に行われます。配送には二営業日かかります。"]
+  },
+  {
+    "name": "overlap not smaller than chunk size",
+    "text": "w1 w2 w3",
+    "chunkSize": 2,
+    "overlap": 2,
+    "error": true
+  }
+]

raglite_toolkit-1.2.2/tests/fixtures/shared/hash.json

@@ -0,0 +1,34 @@
+{
+  "hashString": [
+    {
+      "input": "",
+      "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
+    },
+    {
+      "input": "hello",
+      "sha256": "2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824"
+    },
+    {
+      "input": "Refunds 30 días — 返金 🚀",
+      "sha256": "8c0b6fdd3be678888cf2ab1a1f688435999e249e317f809cd444616b147dba9f"
+    },
+    {
+      "input": "line1\r\nline2",
+      "sha256": "d14a91a6d1c6ee83bf0c774ebecbee6d8b393b395dae29eea839c354d6fba9c0"
+    }
+  ],
+  "namespaceFromPath": [
+    {
+      "input": "/tmp/docs/policy.pdf",
+      "namespace": "1b0f23231e019b65"
+    },
+    {
+      "input": "C:\\docs\\policy.pdf",
+      "namespace": "b0ea2bdcc7ddf050"
+    },
+    {
+      "input": "https://example.com/policy",
+      "namespace": "fd4f660e25200691"
+    }
+  ]
+}

{raglite_toolkit-1.2.1 → raglite_toolkit-1.2.2}/tests/unit/test_regressions.py

@@ -1,6 +1,7 @@
 """
 Regression tests for URL indexing and shared vector stores across documents.
 """
+import json
 import shutil
 import tempfile
 from pathlib import Path
@@ -8,6 +9,7 @@ from unittest.mock import patch
 
 import pytest
 
+from raglite.constants import INDEX_FORMAT_VERSION
 from raglite.core.collection import DocumentCollection
 from raglite.core.document import Document
 from raglite.embeddings.local import LocalEmbedder
@@ -208,3 +210,45 @@ def test_collection_search_raises_when_every_document_fails(tmp_dir, embedder_co
     ):
         with pytest.raises(RuntimeError, match="bad api key"):
             collection.search("refund")
+
+
+def _build_then_edit(tmp_dir, edit):
+    path = Path(tmp_dir) / "policy.txt"
+    path.write_text("Refunds are issued within 30 days.", encoding="utf-8")
+    opts = {"storeDir": str(Path(tmp_dir) / ".raglite"), "logLevel": "silent"}
+    doc = Document(str(path), opts)
+    doc.build()
+    meta_path = Path(tmp_dir) / ".raglite" / doc.store_namespace / "metadata.json"
+    meta = json.loads(meta_path.read_text(encoding="utf-8"))
+    assert meta["formatVersion"] == INDEX_FORMAT_VERSION
+    edit(meta)
+    meta_path.write_text(json.dumps(meta), encoding="utf-8")
+    return Document(str(path), opts).build()
+
+
+def test_index_reused_across_package_versions(tmp_dir, embedder_configs):
+    result = _build_then_edit(tmp_dir, lambda m: m.update(version="9.9.9"))
+    assert result["cached"] is True
+
+
+def test_index_from_1_2_1_without_format_version_is_reused(tmp_dir, embedder_configs):
+    def edit(m):
+        m["version"] = "1.2.1"
+        m.pop("formatVersion")
+
+    assert _build_then_edit(tmp_dir, edit)["cached"] is True
+
+
+def test_older_index_without_format_version_is_rebuilt(tmp_dir, embedder_configs):
+    def edit(m):
+        m["version"] = "1.2.0"
+        m.pop("formatVersion")
+
+    assert _build_then_edit(tmp_dir, edit)["cached"] is False
+
+
+def test_index_with_other_format_version_is_rebuilt(tmp_dir, embedder_configs):
+    result = _build_then_edit(
+        tmp_dir, lambda m: m.update(formatVersion=INDEX_FORMAT_VERSION + 1)
+    )
+    assert result["cached"] is False

raglite_toolkit-1.2.2/tests/unit/test_shared_fixtures.py

@@ -0,0 +1,42 @@
+"""
+Cross-SDK fixtures: the TypeScript SDK generates these expected outputs and
+both SDKs must reproduce them. Refresh with scripts/sync_shared_fixtures.py.
+"""
+import json
+from pathlib import Path
+
+import pytest
+
+from raglite.chunking import RecursiveChunker
+from raglite.errors import ChunkingError
+from raglite.utils.hash import hash_string, namespace_from_path
+
+FIXTURES = Path(__file__).resolve().parent.parent / "fixtures" / "shared"
+
+
+def _load(name):
+    return json.loads((FIXTURES / name).read_text(encoding="utf-8"))
+
+
+CHUNKER = _load("chunker.json")
+HASH = _load("hash.json")
+
+
+@pytest.mark.parametrize("case", CHUNKER, ids=[c["name"] for c in CHUNKER])
+def test_chunker_matches_typescript(case):
+    chunker = RecursiveChunker(case["chunkSize"], case["overlap"])
+    if case.get("error"):
+        with pytest.raises(ChunkingError):
+            chunker.split(case["text"])
+    else:
+        assert chunker.split(case["text"]) == case["chunks"]
+
+
+@pytest.mark.parametrize("case", HASH["hashString"], ids=lambda c: repr(c["input"]))
+def test_hash_string_matches_typescript(case):
+    assert hash_string(case["input"]) == case["sha256"]
+
+
+@pytest.mark.parametrize("case", HASH["namespaceFromPath"], ids=lambda c: c["input"])
+def test_namespace_matches_typescript(case):
+    assert namespace_from_path(case["input"]) == case["namespace"]