rag-verifier 0.1.0__tar.gz

rag_verifier-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 hlc0000
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

rag_verifier-0.1.0/PKG-INFO

@@ -0,0 +1,273 @@
+Metadata-Version: 2.4
+Name: rag-verifier
+Version: 0.1.0
+Summary: Chunk-level quality verification for RAG ingestion: independence, completeness, traceability, semantic independence, template completeness
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Provides-Extra: parse
+Requires-Dist: pymupdf<2.0,>=1.23; extra == "parse"
+Requires-Dist: marko>=2.0; extra == "parse"
+Requires-Dist: lxml>=4.9; extra == "parse"
+Provides-Extra: langchain
+Requires-Dist: langchain-core>=0.2; extra == "langchain"
+Dynamic: license-file
+
+# rag-verifier
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![CI](https://github.com/hlc0000/rag-verifier/actions/workflows/ci.yml/badge.svg)](https://github.com/hlc0000/rag-verifier/actions/workflows/ci.yml)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](pyproject.toml)
+[![PyPI](https://img.shields.io/badge/PyPI-coming%20soon-orange.svg)](#installation)
+
+🎯 **[Online Demo](https://rag-verifier.streamlit.app)** · 📦 `pip install rag-verifier`
+
+> **PyPI publishing is in progress.** Until then, install from GitHub — see [Installation](#installation).
+
+![demo](docs/demo.png)
+
+Most RAG failures aren't model failures — they're evidence failures. A clause gets split mid-condition: `If the Client breaches any material obligation,` goes into one vector, `the Contractor may terminate this Agreement.` into another. Both chunks look fine to your retriever; neither is answerable on its own. And when the LLM finally answers, it quotes a citation that *sounds* right — with the trigger condition quietly cut away.
+
+rag-verifier is a quality gate for that evidence layer. It checks every chunk **before** it enters your vector store, and every citation **before** it reaches your user.
+
+**Niche:** `chunkvet` audits chunk geometry. `ContextDoctor` does static analysis. `rag-verifier` targets **semantic integrity before ingestion** + **citation faithfulness after generation**. Rule layer is zero-dependency; LLM fallback only.
+
+## One-line CLI
+
+```bash
+rag-verifier check "Therefore the Contractor may terminate this Agreement." \
+  --prev "If the Client breaches any material obligation,"
+```
+
+```
+[FAIL] overall = fail
+  - independence: fail  (Starts with a dependent connective: 'Therefore')
+  - completeness: pass
+  - traceability: uncertain  (No source text provided)
+  - semantic_independence: uncertain  (No LLM callback; semantic layer skipped)
+  - template_completeness: pass
+```
+
+Exit code `0` = pass, `1` = fail or uncertain. Add `--json` for machine-readable output, `--llm` to enable the semantic layer.
+
+## What it checks
+
+Five checks run sequentially on each chunk. The rule layer is **fast** (pure regex) and catches structural defects deterministically — measured **false-positive rate on our evaluation set (1,388 chunks / 20 real-world documents)**. The LLM semantic layer kicks in only when the rule layer says "looks OK" but the chunk might still be semantically broken.
+
+| Check | Rule layer (fast, zero LLM cost) | LLM semantic layer (fallback) |
+|---|---|---|
+| `independence` | dependent connective at start (`However`/`Therefore`/`Accordingly`), incomplete conditional ending (`If`/`When`/`Unless`), fragments, empty chunks, table-continuation markers | trigger condition cut away, elided subject |
+| `completeness` | conditional pairings, concessive pairings, dependent connective with word boundary | — |
+| `traceability` | unresolved anaphora (`this`/`that`/`it` with no in-chunk referent), cross-chunk substantive references (`as shown in Figure 2`) | — |
+| `template_completeness` | bracketed placeholders (`[yyyy]`), underscore blanks, quantity/date/signature slots | — |
+| `semantic_independence` | — | full independent judgment with the complete document as context |
+
+The LLM layer is **fail-open**: if the API is down, the chunk gets `uncertain`, **never** silently reclassified as `supported` or `insufficient`.
+
+## Why this exists: pure-rules vs pure-LLM vs this
+
+| | Pure regex rules | Pure LLM judge | rag-verifier (hybrid) |
+|---|---|---|---|
+| **Cost per chunk** | $0 | $0.001–$0.01 | $0 for rule-pass; LLM only on rule-ambiguous |
+| **Latency** | <1 ms | 500–2000 ms | <1 ms for the 95% rule-pass cases |
+| **Consistency** (same chunk, same verdict every run) | ✅ deterministic | ❌ sampling variance, prompt drift | ✅ deterministic for rule layer; LLM only where needed |
+| **Fail-open on API failure** | N/A | ❌ often silently passes | ✅ returns `uncertain`, never fakes `pass` |
+| **Catches subtle semantic breaks** | ❌ only structural | ✅ but expensive & inconsistent | ✅ LLM layer for the hard cases |
+
+The rule layer does the cheap, deterministic work. The LLM layer exists only for the cases rules can't decide.
+
+## How it fits in the RAG quality stack
+
+This is one layer of a healthy stack, designed to compose with the others:
+
+- **Structure-aware chunkers** (e.g. chunkvet, LangChain text splitters) decide *where to cut*. rag-verifier runs after any chunker — yours, theirs, LangChain's — and gates *what may pass*.
+- **Citation-side checkers** (e.g. spancheck) verify that quoted spans exist. We go one step further into *faithfulness*: a verbatim quote can still be `insufficient` if its trigger condition was cut away.
+- **Static context analyzers** (e.g. ContextDoctor) lint your context before retrieval. We add a fail-open LLM semantic layer on top of deterministic rules, in both Chinese and English.
+
+Parse with anything, chunk with anything, gate with rag-verifier.
+
+## Installation
+
+```bash
+pip install rag-verifier
+```
+
+> **Status:** PyPI upload is being finalized. The command above will work once published. Meanwhile, install from GitHub:
+
+```bash
+pip install git+https://github.com/hlc0000/rag-verifier.git
+```
+
+The core verification layer is **zero-dependency** (pure stdlib — the LLM calls go through `urllib`). Optional extras:
+
+```bash
+pip install "rag-verifier[parse]"        # PDF/DOCX/Markdown/HTML parsing
+pip install "rag-verifier[langchain]"    # VerifierTransformer for LangChain pipelines
+```
+
+<details><summary>Development install (editable)</summary>
+
+```bash
+git clone https://github.com/hlc0000/rag-verifier.git
+cd rag-verifier
+pip install -e ".[parse,langchain]"
+```
+
+</details>
+
+## Usage
+
+### 1. Command line
+
+```bash
+# Single chunk (rule layer only)
+rag-verifier check "Therefore the Contractor may terminate." --prev "If the Client breaches,"
+
+# Read from stdin
+echo "However, the agreement is terminated." | rag-verifier check -
+
+# Citation faithfulness
+rag-verifier citation \
+  --claim "Apache License requires copyright headers." \
+  --citation "Copyright [yyyy] [name of copyright owner]" \
+  --source-text "$(cat apache2.txt)"
+
+# JSON output for piping
+rag-verifier check "..." --json | jq '.overall'
+```
+
+### 2. Chunk quality gate (Python API)
+
+```python
+from rag_verifier import verify_chunk
+
+result = verify_chunk(
+    "Therefore the Contractor has the right to claim damages from the Client.",
+    prev_chunk="If the Client breaches any material obligation under this Agreement,",
+)
+# result["overall"].value == "fail"
+# result["checks"]["independence"]["reason"] == "Starts with a dependent connective: 'Therefore'"
+```
+
+### 3. Citation faithfulness — does the citation actually support the claim?
+
+People search for "RAG citation verification" more than "chunk quality gate" — this is the check that catches AI *appearing* to cite evidence while quietly omitting the premise.
+
+**Three-way verdict** (plus `uncertain` when the LLM is unavailable):
+
+- `supported` — citation both entails the claim **and** stands alone without misleading
+- `unsupported` — citation contradicts the claim → **regenerate the answer**
+- `insufficient` — citation is incomplete or off-topic (e.g. only the result clause of a conditional, trigger condition cut away) → **re-retrieve**
+
+```python
+from rag_verifier import verify_citation
+
+result = verify_citation(
+    claim="Apache License 2.0 requires copyright headers.",
+    citation="Copyright [yyyy] [name of copyright owner]",
+    source_text="... full Apache 2.0 license text ...",
+)
+# result["status"]  -> "insufficient"
+# result["reason"]  -> why the citation doesn't support the claim
+```
+
+A **deterministic gate runs before any LLM call**: if the citation is located verbatim in the source and its preceding context shows a truncated conditional (the citation kept only the result clause), the verdict is `insufficient` immediately — no LLM variance, no API cost. This gate alone lifted truncated-conditional detection from 55% to 95% on our evaluation set.
+
+### 4. With the LLM semantic layer
+
+```bash
+export LLM_API_KEY=sk-xxx
+export LLM_BASE_URL=https://api.deepseek.com/v1   # any OpenAI-compatible endpoint
+export LLM_MODEL=deepseek-chat
+```
+
+```python
+from rag_verifier import verify_chunk, builtin_llm
+
+result = verify_chunk(
+    "Therefore Party A has the right to claim compensation from Party B.",
+    prev_chunk="If Party B breaches the confidentiality obligation,",
+    full_doc="If Party B breaches the confidentiality obligation, therefore Party A ...",
+    llm_callback=builtin_llm,   # inject any LLM callable; without it the semantic check returns uncertain
+)
+```
+
+### 5. LangChain integration (5 lines)
+
+```python
+from langchain_text_splitters import RecursiveCharacterTextSplitter
+from rag_verifier.integrations.langchain import VerifierTransformer
+
+splitter = RecursiveCharacterTextSplitter(chunk_size=500)
+chunks = splitter.split_documents(docs)
+
+gate = VerifierTransformer()              # rule layer, zero LLM calls
+clean_chunks = gate.transform_documents(chunks)   # drops FAIL chunks; marks UNCERTAIN in metadata
+# clean_chunks -> vectorstore
+```
+
+Enable the semantic layer by injecting an LLM callback:
+
+```python
+from rag_verifier import builtin_llm
+gate = VerifierTransformer(llm_callback=builtin_llm)
+```
+
+### 6. Full pipeline: parse → split → verify
+
+```python
+from rag_verifier.parsing import parse_document
+from rag_verifier.chunking import split_document
+from rag_verifier import verify_chunk
+
+doc = parse_document("contract.pdf")      # StructuredDocument
+chunks = split_document(doc)              # list[dict]: {"id", "card", "source_text", ...}
+
+prev = ""
+for item in chunks:
+    content = item["card"]["content"]
+    result = verify_chunk(content, prev_chunk=prev, source_text=item.get("source_text", ""))
+    if result["overall"].value == "fail":
+        failed = [f"{k}: {v['reason']}" for k, v in result["checks"].items() if v["status"] == "fail"]
+        print(f"DROP {item['id']}: {'; '.join(failed)}")
+    prev = content
+```
+
+## Bilingual parity (zh / en) — enforced in CI
+
+Every rule has a Chinese and an English copy, dispatched automatically by `detect_language()` (CJK ratio ≥ 15% → zh). The prompt templates, the `reason` strings, and the three citation verdicts all come in matching pairs. Run an English document → English output. Run a Chinese document → Chinese output. Never mixed.
+
+This is a hard guarantee, not a promise: `scripts/rule_parity_check.py` runs the **positive and negative examples for every rule in both languages** and **fails the build** if any positive case is missed or any negative case is hit by accident. English coverage is not "good enough" — it is byte-for-byte symmetric with Chinese.
+
+## Limitations — not a silver bullet
+
+rag-verifier is a quality gate, not a repair tool. It tells you which chunks are broken; it does not fix them.
+
+- **Does not repair chunks.** A failed chunk needs to be re-split (different chunker, larger overlap) or merged with its neighbor — that's your pipeline's job.
+- **Does not evaluate retrieval quality.** It doesn't measure recall@k or whether the right chunk was retrieved. It only checks whether a chunk, once retrieved, is self-sufficient.
+- **Does not support scanned PDFs.** No OCR; a scanned PDF yields an empty text layer. Use an OCR pipeline first.
+- **Does not support RTF.** RTF headings rarely carry style information, so heading levels are lost during parsing.
+- **Chinese-question-on-English-document is not supported.** The verifier detects the document's language and applies matching rules; cross-language question answering is out of scope.
+- **Rule layer catches structural defects, not factual errors.** A chunk can be perfectly self-contained and still contain wrong information — that's the LLM / citation check's job, not the chunk gate's.
+
+## Supported formats
+
+Text-layer PDF, DOCX, TXT, Markdown, HTML. Not supported: RTF, scanned PDF (see Limitations above).
+
+## Tests
+
+```bash
+python scripts/rule_parity_check.py    # zh/en rule parity — must pass (runs in CI)
+```
+
+## Roadmap & Contributing
+
+- **LlamaIndex integration** ( `VerifierTransformer` for LlamaIndex nodes)
+- **CLI batch mode** (verify a JSONL of chunks in one command)
+- **More document types** for the parsing extra
+
+Found a chunk that *should* be caught but isn't? Or a false positive that broke your pipeline? [Open an issue](https://github.com/hlc0000/rag-verifier/issues) with the chunk text and the expected verdict. Rule-layer fixes are deterministic and permanent — once a pattern is covered, it stays covered.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

rag_verifier-0.1.0/README.md

@@ -0,0 +1,258 @@
+# rag-verifier
+
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+[![CI](https://github.com/hlc0000/rag-verifier/actions/workflows/ci.yml/badge.svg)](https://github.com/hlc0000/rag-verifier/actions/workflows/ci.yml)
+[![Python 3.10+](https://img.shields.io/badge/python-3.10%2B-blue.svg)](pyproject.toml)
+[![PyPI](https://img.shields.io/badge/PyPI-coming%20soon-orange.svg)](#installation)
+
+🎯 **[Online Demo](https://rag-verifier.streamlit.app)** · 📦 `pip install rag-verifier`
+
+> **PyPI publishing is in progress.** Until then, install from GitHub — see [Installation](#installation).
+
+![demo](docs/demo.png)
+
+Most RAG failures aren't model failures — they're evidence failures. A clause gets split mid-condition: `If the Client breaches any material obligation,` goes into one vector, `the Contractor may terminate this Agreement.` into another. Both chunks look fine to your retriever; neither is answerable on its own. And when the LLM finally answers, it quotes a citation that *sounds* right — with the trigger condition quietly cut away.
+
+rag-verifier is a quality gate for that evidence layer. It checks every chunk **before** it enters your vector store, and every citation **before** it reaches your user.
+
+**Niche:** `chunkvet` audits chunk geometry. `ContextDoctor` does static analysis. `rag-verifier` targets **semantic integrity before ingestion** + **citation faithfulness after generation**. Rule layer is zero-dependency; LLM fallback only.
+
+## One-line CLI
+
+```bash
+rag-verifier check "Therefore the Contractor may terminate this Agreement." \
+  --prev "If the Client breaches any material obligation,"
+```
+
+```
+[FAIL] overall = fail
+  - independence: fail  (Starts with a dependent connective: 'Therefore')
+  - completeness: pass
+  - traceability: uncertain  (No source text provided)
+  - semantic_independence: uncertain  (No LLM callback; semantic layer skipped)
+  - template_completeness: pass
+```
+
+Exit code `0` = pass, `1` = fail or uncertain. Add `--json` for machine-readable output, `--llm` to enable the semantic layer.
+
+## What it checks
+
+Five checks run sequentially on each chunk. The rule layer is **fast** (pure regex) and catches structural defects deterministically — measured **false-positive rate on our evaluation set (1,388 chunks / 20 real-world documents)**. The LLM semantic layer kicks in only when the rule layer says "looks OK" but the chunk might still be semantically broken.
+
+| Check | Rule layer (fast, zero LLM cost) | LLM semantic layer (fallback) |
+|---|---|---|
+| `independence` | dependent connective at start (`However`/`Therefore`/`Accordingly`), incomplete conditional ending (`If`/`When`/`Unless`), fragments, empty chunks, table-continuation markers | trigger condition cut away, elided subject |
+| `completeness` | conditional pairings, concessive pairings, dependent connective with word boundary | — |
+| `traceability` | unresolved anaphora (`this`/`that`/`it` with no in-chunk referent), cross-chunk substantive references (`as shown in Figure 2`) | — |
+| `template_completeness` | bracketed placeholders (`[yyyy]`), underscore blanks, quantity/date/signature slots | — |
+| `semantic_independence` | — | full independent judgment with the complete document as context |
+
+The LLM layer is **fail-open**: if the API is down, the chunk gets `uncertain`, **never** silently reclassified as `supported` or `insufficient`.
+
+## Why this exists: pure-rules vs pure-LLM vs this
+
+| | Pure regex rules | Pure LLM judge | rag-verifier (hybrid) |
+|---|---|---|---|
+| **Cost per chunk** | $0 | $0.001–$0.01 | $0 for rule-pass; LLM only on rule-ambiguous |
+| **Latency** | <1 ms | 500–2000 ms | <1 ms for the 95% rule-pass cases |
+| **Consistency** (same chunk, same verdict every run) | ✅ deterministic | ❌ sampling variance, prompt drift | ✅ deterministic for rule layer; LLM only where needed |
+| **Fail-open on API failure** | N/A | ❌ often silently passes | ✅ returns `uncertain`, never fakes `pass` |
+| **Catches subtle semantic breaks** | ❌ only structural | ✅ but expensive & inconsistent | ✅ LLM layer for the hard cases |
+
+The rule layer does the cheap, deterministic work. The LLM layer exists only for the cases rules can't decide.
+
+## How it fits in the RAG quality stack
+
+This is one layer of a healthy stack, designed to compose with the others:
+
+- **Structure-aware chunkers** (e.g. chunkvet, LangChain text splitters) decide *where to cut*. rag-verifier runs after any chunker — yours, theirs, LangChain's — and gates *what may pass*.
+- **Citation-side checkers** (e.g. spancheck) verify that quoted spans exist. We go one step further into *faithfulness*: a verbatim quote can still be `insufficient` if its trigger condition was cut away.
+- **Static context analyzers** (e.g. ContextDoctor) lint your context before retrieval. We add a fail-open LLM semantic layer on top of deterministic rules, in both Chinese and English.
+
+Parse with anything, chunk with anything, gate with rag-verifier.
+
+## Installation
+
+```bash
+pip install rag-verifier
+```
+
+> **Status:** PyPI upload is being finalized. The command above will work once published. Meanwhile, install from GitHub:
+
+```bash
+pip install git+https://github.com/hlc0000/rag-verifier.git
+```
+
+The core verification layer is **zero-dependency** (pure stdlib — the LLM calls go through `urllib`). Optional extras:
+
+```bash
+pip install "rag-verifier[parse]"        # PDF/DOCX/Markdown/HTML parsing
+pip install "rag-verifier[langchain]"    # VerifierTransformer for LangChain pipelines
+```
+
+<details><summary>Development install (editable)</summary>
+
+```bash
+git clone https://github.com/hlc0000/rag-verifier.git
+cd rag-verifier
+pip install -e ".[parse,langchain]"
+```
+
+</details>
+
+## Usage
+
+### 1. Command line
+
+```bash
+# Single chunk (rule layer only)
+rag-verifier check "Therefore the Contractor may terminate." --prev "If the Client breaches,"
+
+# Read from stdin
+echo "However, the agreement is terminated." | rag-verifier check -
+
+# Citation faithfulness
+rag-verifier citation \
+  --claim "Apache License requires copyright headers." \
+  --citation "Copyright [yyyy] [name of copyright owner]" \
+  --source-text "$(cat apache2.txt)"
+
+# JSON output for piping
+rag-verifier check "..." --json | jq '.overall'
+```
+
+### 2. Chunk quality gate (Python API)
+
+```python
+from rag_verifier import verify_chunk
+
+result = verify_chunk(
+    "Therefore the Contractor has the right to claim damages from the Client.",
+    prev_chunk="If the Client breaches any material obligation under this Agreement,",
+)
+# result["overall"].value == "fail"
+# result["checks"]["independence"]["reason"] == "Starts with a dependent connective: 'Therefore'"
+```
+
+### 3. Citation faithfulness — does the citation actually support the claim?
+
+People search for "RAG citation verification" more than "chunk quality gate" — this is the check that catches AI *appearing* to cite evidence while quietly omitting the premise.
+
+**Three-way verdict** (plus `uncertain` when the LLM is unavailable):
+
+- `supported` — citation both entails the claim **and** stands alone without misleading
+- `unsupported` — citation contradicts the claim → **regenerate the answer**
+- `insufficient` — citation is incomplete or off-topic (e.g. only the result clause of a conditional, trigger condition cut away) → **re-retrieve**
+
+```python
+from rag_verifier import verify_citation
+
+result = verify_citation(
+    claim="Apache License 2.0 requires copyright headers.",
+    citation="Copyright [yyyy] [name of copyright owner]",
+    source_text="... full Apache 2.0 license text ...",
+)
+# result["status"]  -> "insufficient"
+# result["reason"]  -> why the citation doesn't support the claim
+```
+
+A **deterministic gate runs before any LLM call**: if the citation is located verbatim in the source and its preceding context shows a truncated conditional (the citation kept only the result clause), the verdict is `insufficient` immediately — no LLM variance, no API cost. This gate alone lifted truncated-conditional detection from 55% to 95% on our evaluation set.
+
+### 4. With the LLM semantic layer
+
+```bash
+export LLM_API_KEY=sk-xxx
+export LLM_BASE_URL=https://api.deepseek.com/v1   # any OpenAI-compatible endpoint
+export LLM_MODEL=deepseek-chat
+```
+
+```python
+from rag_verifier import verify_chunk, builtin_llm
+
+result = verify_chunk(
+    "Therefore Party A has the right to claim compensation from Party B.",
+    prev_chunk="If Party B breaches the confidentiality obligation,",
+    full_doc="If Party B breaches the confidentiality obligation, therefore Party A ...",
+    llm_callback=builtin_llm,   # inject any LLM callable; without it the semantic check returns uncertain
+)
+```
+
+### 5. LangChain integration (5 lines)
+
+```python
+from langchain_text_splitters import RecursiveCharacterTextSplitter
+from rag_verifier.integrations.langchain import VerifierTransformer
+
+splitter = RecursiveCharacterTextSplitter(chunk_size=500)
+chunks = splitter.split_documents(docs)
+
+gate = VerifierTransformer()              # rule layer, zero LLM calls
+clean_chunks = gate.transform_documents(chunks)   # drops FAIL chunks; marks UNCERTAIN in metadata
+# clean_chunks -> vectorstore
+```
+
+Enable the semantic layer by injecting an LLM callback:
+
+```python
+from rag_verifier import builtin_llm
+gate = VerifierTransformer(llm_callback=builtin_llm)
+```
+
+### 6. Full pipeline: parse → split → verify
+
+```python
+from rag_verifier.parsing import parse_document
+from rag_verifier.chunking import split_document
+from rag_verifier import verify_chunk
+
+doc = parse_document("contract.pdf")      # StructuredDocument
+chunks = split_document(doc)              # list[dict]: {"id", "card", "source_text", ...}
+
+prev = ""
+for item in chunks:
+    content = item["card"]["content"]
+    result = verify_chunk(content, prev_chunk=prev, source_text=item.get("source_text", ""))
+    if result["overall"].value == "fail":
+        failed = [f"{k}: {v['reason']}" for k, v in result["checks"].items() if v["status"] == "fail"]
+        print(f"DROP {item['id']}: {'; '.join(failed)}")
+    prev = content
+```
+
+## Bilingual parity (zh / en) — enforced in CI
+
+Every rule has a Chinese and an English copy, dispatched automatically by `detect_language()` (CJK ratio ≥ 15% → zh). The prompt templates, the `reason` strings, and the three citation verdicts all come in matching pairs. Run an English document → English output. Run a Chinese document → Chinese output. Never mixed.
+
+This is a hard guarantee, not a promise: `scripts/rule_parity_check.py` runs the **positive and negative examples for every rule in both languages** and **fails the build** if any positive case is missed or any negative case is hit by accident. English coverage is not "good enough" — it is byte-for-byte symmetric with Chinese.
+
+## Limitations — not a silver bullet
+
+rag-verifier is a quality gate, not a repair tool. It tells you which chunks are broken; it does not fix them.
+
+- **Does not repair chunks.** A failed chunk needs to be re-split (different chunker, larger overlap) or merged with its neighbor — that's your pipeline's job.
+- **Does not evaluate retrieval quality.** It doesn't measure recall@k or whether the right chunk was retrieved. It only checks whether a chunk, once retrieved, is self-sufficient.
+- **Does not support scanned PDFs.** No OCR; a scanned PDF yields an empty text layer. Use an OCR pipeline first.
+- **Does not support RTF.** RTF headings rarely carry style information, so heading levels are lost during parsing.
+- **Chinese-question-on-English-document is not supported.** The verifier detects the document's language and applies matching rules; cross-language question answering is out of scope.
+- **Rule layer catches structural defects, not factual errors.** A chunk can be perfectly self-contained and still contain wrong information — that's the LLM / citation check's job, not the chunk gate's.
+
+## Supported formats
+
+Text-layer PDF, DOCX, TXT, Markdown, HTML. Not supported: RTF, scanned PDF (see Limitations above).
+
+## Tests
+
+```bash
+python scripts/rule_parity_check.py    # zh/en rule parity — must pass (runs in CI)
+```
+
+## Roadmap & Contributing
+
+- **LlamaIndex integration** ( `VerifierTransformer` for LlamaIndex nodes)
+- **CLI batch mode** (verify a JSONL of chunks in one command)
+- **More document types** for the parsing extra
+
+Found a chunk that *should* be caught but isn't? Or a false positive that broke your pipeline? [Open an issue](https://github.com/hlc0000/rag-verifier/issues) with the chunk text and the expected verdict. Rule-layer fixes are deterministic and permanent — once a pattern is covered, it stays covered.
+
+## License
+
+MIT — see [LICENSE](LICENSE).

rag_verifier-0.1.0/pyproject.toml

@@ -0,0 +1,25 @@
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "rag-verifier"
+version = "0.1.0"
+description = "Chunk-level quality verification for RAG ingestion: independence, completeness, traceability, semantic independence, template completeness"
+readme = "README.md"
+requires-python = ">=3.10"
+# 验证层只用标准库；解析/检索的依赖由各自模块按需引入（docling 走独立 venv）。
+dependencies = []
+
+[project.scripts]
+rag-verifier = "rag_verifier.cli:main"
+
+[project.optional-dependencies]
+# pymupdf 2.0 起不再提供 Intel Mac wheel；<2.0 保证全平台可装，
+# 且解析层代码是针对 1.x API 开发验证的
+parse = ["pymupdf>=1.23,<2.0", "marko>=2.0", "lxml>=4.9"]
+# LangChain 集成：提供 VerifierTransformer，可直接接入 LCEL 流水线
+langchain = ["langchain-core>=0.2"]
+
+[tool.setuptools.packages.find]
+include = ["rag_verifier*"]

rag_verifier-0.1.0/rag_verifier/__init__.py

@@ -0,0 +1,27 @@
+"""rag-verifier 对外包入口。
+
+    from rag_verifier import verify_chunks
+
+只依赖验证层（不导入 docling/pymupdf/检索层），可作为独立组件嵌入任意
+Python 流水线：任意解析器产出的块 → 五关质量验证。
+"""
+from rag_verifier.chunking.rules import CHECKS, RuleContext, detect_language, list_rules, register_rule
+from rag_verifier.chunking.semantic_verifier import builtin_llm
+from rag_verifier.citation.verifier import verify_citation
+from rag_verifier.models import EvidenceCard, VerificationResult, VerificationStatus
+from rag_verifier.verify_api import verify_chunk, verify_chunks
+
+__all__ = [
+    "verify_chunk",
+    "verify_chunks",
+    "verify_citation",
+    "EvidenceCard",
+    "VerificationResult",
+    "VerificationStatus",
+    "builtin_llm",
+    "register_rule",
+    "detect_language",
+    "list_rules",
+    "RuleContext",
+    "CHECKS",
+]

rag_verifier-0.1.0/rag_verifier/chunking/__init__.py

@@ -0,0 +1,3 @@
+from rag_verifier.chunking.splitter import split_document
+
+__all__ = ["split_document"]