persian-rag 0.2.0__tar.gz

persian_rag-0.2.0/.gitignore

@@ -0,0 +1,20 @@
+__pycache__/
+*.py[cod]
+*.egg-info/
+.eggs/
+dist/
+build/
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.venv/
+venv/
+.python-version
+.persian-rag/
+*.index.json
+.coverage
+htmlcov/
+.DS_Store
+site/
+*.sqlite
+*.sqlite3

persian_rag-0.2.0/CHANGELOG.md

@@ -0,0 +1,44 @@
+# Changelog
+
+All notable changes to persian-rag are documented here. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and the project uses [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.2.0] - 2026-10-30
+
+### Added
+
+- Rerankers: a `Reranker` protocol, `CrossEncoderReranker` (`rerank` extra, default model `BAAI/bge-reranker-v2-m3`), and a listwise `LLMReranker` that tolerates loosely formatted replies.
+- `RAGPipeline(reranker=..., rerank_candidates=...)` and `RAGPipeline.retrieve()`; reranked results keep the first-stage score in `RetrievalResult.retrieval_score`.
+- Vector stores in the new `persian_rag.vectorstores` package: `QdrantVectorStore` (`qdrant` extra) and `PgVectorStore` for PostgreSQL with pgvector (`pgvector` extra), both with delete support.
+- PDF (`pdf` extra) and Word (`docx` extra) loaders. PDF text is cleaned of presentation forms and bidi controls, and lines stored in visual order are detected and repaired. `load_documents` now reads `.pdf` and `.docx` files and can split PDFs by page.
+- Evaluation module: hit rate, MRR, recall, and nDCG at several cut-offs, document or chunk matching, JSONL datasets, and a faithfulness scorer protocol with token-overlap and LLM-judge implementations.
+- FastAPI server (`server` extra) with `/health`, `/retrieve`, and `/query`, created through `persian_rag.server.create_app`.
+- Async support: `HybridRetriever.aretrieve`, `RAGPipeline.aretrieve`, and `RAGPipeline.aquery`, which accepts async or sync model callables.
+- Caching: `CachedEmbedder` with in-memory and SQLite embedding caches, and an LRU/TTL result cache for `HybridRetriever`.
+- CLI: `persian-rag eval` and `persian-rag serve`, plus `--rerank`, `--rtl-fix`, `--split-pages`, and `--embedding-cache` options.
+- Bilingual documentation site (English and Persian) built with MkDocs Material.
+- Release workflow that publishes to PyPI on version tags through trusted publishing or an API token, and skips publishing when neither is configured.
+- CI jobs for Python 3.10 to 3.13 with all extras, a minimal-dependency run, PostgreSQL with pgvector, the documentation build, and package checks.
+
+### Changed
+
+- `MissingDependencyError` is raised for missing optional packages. It subclasses both `PersianRagError` and `ImportError`.
+- `RAGPipeline.query` raises `TypeError` when given an async model; use `aquery` instead.
+- The `all` extra now installs every optional integration.
+- README rewritten with an extras table, usage guides, and a Persian summary.
+
+## [0.1.0] - 2026-10-30
+
+### Added
+
+- Python package `persian_rag` for Python 3.10 and newer.
+- Persian normalization: Arabic and Persian letters, ZWNJ, digits, optional diacritics, and punctuation.
+- Sentence splitting, tokenization, and fixed, sentence, and recursive chunking.
+- Embedder protocol with hashing, sentence-transformers, and OpenAI-compatible adapters.
+- In-memory cosine store and an optional FAISS adapter.
+- BM25, dense retrieval, and reciprocal rank fusion.
+- `RAGPipeline` with a caller-supplied LLM callable.
+- `persian-rag ingest` and `persian-rag query` commands, plus a JSON index.
+- Unit tests, Ruff, mypy, and a GitHub Actions workflow.
+
+[0.2.0]: https://github.com/sadrmanesh/persian-rag/releases/tag/v0.2.0
+[0.1.0]: https://github.com/sadrmanesh/persian-rag/releases/tag/v0.1.0

persian_rag-0.2.0/CONTRIBUTING.md

@@ -0,0 +1,47 @@
+# Contributing to persian-rag
+
+Thanks for considering a contribution. The project is maintained by Naser Sadrmanesh ([info.sadrmanesh@gmail.com](mailto:info.sadrmanesh@gmail.com)).
+
+Issues and pull requests may be written in Persian or English.
+
+## Setup
+
+```bash
+git clone https://github.com/sadrmanesh/persian-rag
+cd persian-rag
+python -m venv .venv
+source .venv/bin/activate
+pip install -e ".[dev]"
+# optional integrations used by the full test suite
+pip install -e ".[dev,server,qdrant,pgvector,pdf,docx]"
+```
+
+Python 3.10 or newer is required.
+
+## Checks
+
+Run these before opening a pull request:
+
+```bash
+ruff check .
+ruff format --check .
+mypy src
+pytest
+```
+
+`ruff format .` rewrites formatting. Keep the public modules typed. New behavior needs a unit test. Persian fixtures belong in `tests/fixtures/` or inline in the test when they are only a few words.
+
+Tests that download model weights must be marked `heavy` and skipped unless `PERSIAN_RAG_RUN_HEAVY=1`. Do not make the default suite depend on network access. Tests for optional extras use `pytest.importorskip` or fakes so the suite also passes with only the core dependencies installed. Tests that need a live PostgreSQL server are marked `postgres` and read the connection string from `PERSIAN_RAG_PG_DSN`.
+
+Documentation lives in `docs/` with Persian pages next to English ones (`page.md` and `page.fa.md`). Build it with `pip install -e ".[docs]"` and `mkdocs build --strict`.
+
+## Pull requests
+
+- Describe the user-facing change and how you tested it.
+- Do not add benchmark numbers unless the change includes the data, the script, and the command used to produce them.
+- Keep API keys, indexes, and virtual environments out of the commit.
+- Version bumps belong in `src/persian_rag/version.py`, `pyproject.toml`, and `CHANGELOG.md` together.
+
+## Scope of 0.2.x
+
+Bug fixes and small, tested improvements to normalization, chunking, retrieval, loaders, evaluation, the server, and the CLI are in scope. New vector-store backends, parsers, and hosted services are better discussed in an issue first so the public API stays small.

persian_rag-0.2.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Naser Sadrmanesh
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

persian_rag-0.2.0/PKG-INFO

@@ -0,0 +1,347 @@
+Metadata-Version: 2.5
+Name: persian-rag
+Version: 0.2.0
+Summary: Persian-first retrieval-augmented generation toolkit
+Project-URL: Homepage, https://github.com/sadrmanesh/persian-rag
+Project-URL: Repository, https://github.com/sadrmanesh/persian-rag
+Project-URL: Documentation, https://sadrmanesh.github.io/persian-rag/
+Project-URL: Changelog, https://github.com/sadrmanesh/persian-rag/blob/main/CHANGELOG.md
+Project-URL: Author, https://sadrmanesh.github.io
+Author-email: Naser Sadrmanesh <info.sadrmanesh@gmail.com>
+Maintainer-email: Naser Sadrmanesh <info.sadrmanesh@gmail.com>
+License: MIT License
+        
+        Copyright (c) 2026 Naser Sadrmanesh
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Keywords: bm25,embeddings,farsi,nlp,persian,pgvector,qdrant,rag,reranking,retrieval
+Classifier: Development Status :: 3 - Alpha
+Classifier: Framework :: FastAPI
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Natural Language :: Persian
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: Topic :: Text Processing :: Linguistic
+Classifier: Typing :: Typed
+Requires-Python: >=3.10
+Provides-Extra: all
+Requires-Dist: faiss-cpu>=1.8.0; extra == 'all'
+Requires-Dist: fastapi>=0.110.0; extra == 'all'
+Requires-Dist: numpy>=1.26.0; extra == 'all'
+Requires-Dist: psycopg[binary]>=3.1.0; extra == 'all'
+Requires-Dist: pypdf>=4.0.0; extra == 'all'
+Requires-Dist: python-docx>=1.1.0; extra == 'all'
+Requires-Dist: qdrant-client>=1.10.0; extra == 'all'
+Requires-Dist: sentence-transformers>=3.0.0; extra == 'all'
+Requires-Dist: uvicorn>=0.29.0; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: httpx>=0.27.0; extra == 'dev'
+Requires-Dist: mypy>=1.11.0; extra == 'dev'
+Requires-Dist: pytest>=8.2.0; extra == 'dev'
+Requires-Dist: ruff>=0.6.0; extra == 'dev'
+Provides-Extra: docs
+Requires-Dist: mkdocs-material>=9.5.0; extra == 'docs'
+Requires-Dist: mkdocs-static-i18n>=1.2.0; extra == 'docs'
+Requires-Dist: mkdocs>=1.6.0; extra == 'docs'
+Provides-Extra: docx
+Requires-Dist: python-docx>=1.1.0; extra == 'docx'
+Provides-Extra: faiss
+Requires-Dist: faiss-cpu>=1.8.0; extra == 'faiss'
+Requires-Dist: numpy>=1.26.0; extra == 'faiss'
+Provides-Extra: pdf
+Requires-Dist: pypdf>=4.0.0; extra == 'pdf'
+Provides-Extra: pgvector
+Requires-Dist: psycopg[binary]>=3.1.0; extra == 'pgvector'
+Provides-Extra: qdrant
+Requires-Dist: qdrant-client>=1.10.0; extra == 'qdrant'
+Provides-Extra: rerank
+Requires-Dist: sentence-transformers>=3.0.0; extra == 'rerank'
+Provides-Extra: sentence-transformers
+Requires-Dist: sentence-transformers>=3.0.0; extra == 'sentence-transformers'
+Provides-Extra: server
+Requires-Dist: fastapi>=0.110.0; extra == 'server'
+Requires-Dist: uvicorn>=0.29.0; extra == 'server'
+Description-Content-Type: text/markdown
+
+# persian-rag
+
+[![CI](https://github.com/sadrmanesh/persian-rag/actions/workflows/ci.yml/badge.svg)](https://github.com/sadrmanesh/persian-rag/actions/workflows/ci.yml)
+[![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12%20%7C%203.13-blue)](pyproject.toml)
+[![License: MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+
+Persian-first retrieval-augmented generation for Python.
+
+persian-rag is a toolkit for building RAG systems over Farsi text. It normalizes Arabic and Persian characters, half-spaces, digits, and punctuation; loads text, Markdown, PDF, and Word files (repairing reversed PDF text); chunks and embeds passages; retrieves them with BM25, dense vectors, or both; reranks them; and measures the result. Generation is a function you supply, sync or async.
+
+The core install is pure Python with no dependencies. Every heavy piece (models, vector databases, parsers, the HTTP server) is an optional extra.
+
+**Documentation:** English and Persian guides live in [`docs/`](docs/) and build with MkDocs (see [Documentation](#documentation)).
+
+## خلاصه فارسی
+
+persian-rag یک جعبه ابزار پایتون برای ساخت سامانه های بازیابی و تولید پاسخ روی متن فارسی است. نویسه های عربی و فارسی، فاصله ها، رقم ها و نشانه گذاری را یکدست می کند. فایل های متنی، مارک داون، PDF و Word را می خواند و ترتیب وارونه متن فارسی در PDF را اصلاح می کند. متن را تکه تکه می کند و با BM25، بردارهای معنایی یا ترکیب هر دو جست و جو می کند. نتیجه را با رتبه بندی دوباره بهتر می کند و کیفیت بازیابی و پاسخ را می سنجد. تولید پاسخ به تابعی سپرده می شود که خودتان می دهید و می تواند همگام یا ناهمگام باشد. راهنمای کامل فارسی در پوشه docs است.
+
+## Features
+
+- **Normalization** for Persian: Arabic Yeh and Kaf, presentation forms, ZWNJ, digits, optional diacritics, and punctuation.
+- **Loaders** for `.txt`, `.md`, `.pdf`, and `.docx`. PDF text in visual (reversed) order is detected and repaired, presentation forms are folded, and bidi control marks are removed.
+- **Chunking**: fixed windows with overlap, sentence packing, and recursive separators.
+- **Embeddings** through a small protocol, with adapters for `sentence-transformers` and any OpenAI-compatible API, plus a persistent embedding cache.
+- **Vector stores**: in-memory, FAISS, Qdrant, and PostgreSQL with pgvector.
+- **Hybrid retrieval** with BM25 and [reciprocal rank fusion](https://plg.uwaterloo.ca/gvcormac/cormacksigir09-rrf.pdf), and an optional query result cache.
+- **Reranking** with a cross-encoder (default `BAAI/bge-reranker-v2-m3`) or any LLM (listwise).
+- **Evaluation**: hit rate, MRR, recall, and nDCG at several cut-offs, plus pluggable answer faithfulness scorers.
+- **Async**: `aretrieve` and `aquery`, with native support for `async` model functions.
+- **HTTP API** built on FastAPI, and a CLI for `ingest`, `query`, `eval`, and `serve`.
+- Fully typed (`mypy --strict`), tested offline, MIT licensed.
+
+## Installation
+
+```bash
+pip install persian-rag
+```
+
+Pick the extras you need:
+
+| Extra | Installs | Enables |
+| --- | --- | --- |
+| `sentence-transformers` | sentence-transformers | `SentenceTransformerEmbedder` |
+| `rerank` | sentence-transformers | `CrossEncoderReranker` |
+| `faiss` | faiss-cpu, numpy | `FaissVectorStore` |
+| `qdrant` | qdrant-client | `QdrantVectorStore` |
+| `pgvector` | psycopg 3 | `PgVectorStore` |
+| `pdf` | pypdf | PDF loading |
+| `docx` | python-docx | Word loading |
+| `server` | fastapi, uvicorn | `persian_rag.server`, `persian-rag serve` |
+| `all` | everything above | |
+
+```bash
+pip install "persian-rag[pdf,docx,qdrant,server]"
+```
+
+Requires Python 3.10 or newer. The OpenAI-compatible adapters use only the standard library: set `OPENAI_API_KEY`, and `OPENAI_BASE_URL` for compatible servers.
+
+## Quick start
+
+```python
+from persian_rag import HashingEmbedder, RAGPipeline, build_retriever
+
+texts = [
+    "حافظ در شهر شیراز زاده شد و دیوان او مشهور است.",
+    "فوتبال ورزشی گروهی با دو تیم است.",
+]
+retriever = build_retriever(texts, HashingEmbedder())
+
+
+def answer(prompt: str) -> str:
+    # Replace with a call to your model.
+    return "شیراز" if "شیراز" in prompt else "نامشخص"
+
+
+result = RAGPipeline(retriever, answer).query("حافظ در کدام شهر زاده شد؟")
+print(result.answer)
+for item in result.contexts:
+    print(round(item.score, 4), item.text)
+```
+
+`HashingEmbedder` is a deterministic character n-gram hash for tests and offline checks. For semantic search use `SentenceTransformerEmbedder` (for example `"sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2"`) or `OpenAICompatibleEmbedder`. The same flow is in [`examples/quickstart.py`](examples/quickstart.py).
+
+## Usage
+
+### Normalization
+
+```python
+from persian_rag import NormalizeConfig, normalize
+
+normalize("يك كتاب با اعداد ٤٥٦")
+# 'یک کتاب با اعداد ۴۵۶'
+
+normalize("کِتاب", NormalizeConfig(remove_diacritics=True, digits="ascii"))
+```
+
+`normalize` is idempotent for a fixed config. Documents and queries must use the same `NormalizeConfig`; `build_retriever` and `HybridRetriever` take care of that.
+
+| Option | Default | Effect |
+| --- | --- | --- |
+| `unify_arabic_chars` | `True` | `ي`/`ى` to `ی`, `ك` to `ک`, `أ`/`إ`/`ٱ` to `ا`, `ؤ` to `و`, `ة` to `ه`. `آ` and `ئ` stay. |
+| `compatibility_forms` | `True` | NFKC, so presentation forms fold into base letters. |
+| `zwnj` | `"normalize"` | Collapse repeated half-spaces; `"remove"` deletes them; `"space"` turns them into spaces. |
+| `correct_half_spaces` | `True` | Join prefixes such as `می` and suffixes such as `ها` and `ترین` with a half-space. |
+| `digits` | `"persian"` | `"persian"`, `"ascii"`, or `"keep"`. Arabic-Indic digits are included. |
+| `remove_diacritics` | `False` | Strip Arabic vowel marks. |
+| `remove_tatweel` | `True` | Remove kashida (`ـ`). |
+| `clean_punctuation` | `True` | Collapse repeats and tidy spaces around punctuation. |
+| `punctuation` | `"persian"` | Map `,;?%` to `،؛؟٪`, the reverse with `"ascii"`, or `"keep"`. |
+
+### Loading PDF and Word files
+
+```python
+from persian_rag import load_documents, load_pdf
+
+documents = load_documents("./library")  # .txt, .md, .pdf, .docx
+pages = load_pdf("report.pdf", split_pages=True)  # one document per page
+```
+
+Many PDF generators store Persian lines in visual order, so extracted words come out reversed (`باتک` instead of `کتاب`). With `rtl_fix="auto"` (the default) each page is checked using two signals: which contextual glyph forms start and end each word, and whether frequent Persian words appear forwards or backwards. Reversed lines are flipped while numbers and Latin words keep their direction. Use `rtl_fix="reverse"` to force the repair or `"none"` to disable it. `clean_extracted_text` exposes the same cleanup for text from other extractors.
+
+Word documents keep paragraphs and tables in body order; table rows become `cell | cell` lines.
+
+### Retrieval and reranking
+
+```python
+from persian_rag import CrossEncoderReranker, HybridRetriever, LLMReranker, RAGPipeline
+
+hits = retriever.retrieve("حافظ اهل کجا بود؟", k=5, mode="hybrid")  # or "dense" / "sparse"
+
+pipeline = RAGPipeline(
+    retriever,
+    my_llm,
+    reranker=CrossEncoderReranker(),  # needs the rerank extra
+    rerank_candidates=20,
+)
+answer = pipeline.query("حافظ اهل کجا بود؟", k=4)
+```
+
+The retriever fetches `rerank_candidates` chunks and the reranker keeps the best `k`. Reranked results carry the reranker score in `score` and the first-stage score in `retrieval_score`. `LLMReranker(llm)` asks a model to order numbered passages and tolerates messy replies; any object with a `rerank(query, results, top_k)` method also works.
+
+Hybrid scores are reciprocal rank fusion. Dense mode returns the store score; sparse mode returns Okapi BM25.
+
+### Vector stores
+
+```python
+from persian_rag.vectorstores import PgVectorStore, QdrantVectorStore
+
+qdrant = QdrantVectorStore("persian_docs", dimension=384, url="http://localhost:6333")
+local = QdrantVectorStore("persian_docs", dimension=384)  # in-process, in memory
+
+pg = PgVectorStore(
+    "postgresql://user:pass@localhost/db",
+    dimension=384,
+    table="persian_chunks",
+    create_index=True,  # HNSW
+)
+
+retriever = build_retriever(texts, embedder, store=qdrant)
+```
+
+All stores share one protocol (`add`, `search`, `__len__`); Qdrant and pgvector also support `delete`. Scores are always larger-is-closer.
+
+### Caching and async
+
+```python
+from persian_rag import CachedEmbedder, LRUCache, SQLiteEmbeddingCache
+
+embedder = CachedEmbedder(OpenAICompatibleEmbedder(), SQLiteEmbeddingCache("emb.sqlite"))
+retriever = build_retriever(texts, embedder)
+retriever.result_cache = LRUCache(max_size=512, ttl=300)
+
+answer = await pipeline.aquery("پرسش")  # async LLMs are awaited, sync ones run in a thread
+```
+
+Embedding cache keys include the embedder settings, so changing the model never returns stale vectors. The result cache is cleared whenever chunks are added.
+
+### Evaluation
+
+```python
+from persian_rag import evaluate_answers, evaluate_retrieval, load_eval_examples
+
+examples = load_eval_examples("eval.jsonl")
+report = evaluate_retrieval(retriever, examples, k_values=(1, 3, 5, 10))
+print(report.format_table())
+
+answers = evaluate_answers(pipeline, examples)  # token-overlap faithfulness by default
+```
+
+Each JSONL line holds `question`, `relevant_ids` (document ids, or chunk ids with `match="chunk"`), and optionally `reference_answer`. Faithfulness scorers implement `score(question, answer, contexts)`: `TokenOverlapFaithfulness` is offline and deterministic, `LLMFaithfulnessJudge(llm)` asks a model for a 0 to 1 score.
+
+### HTTP API
+
+```python
+import uvicorn
+from persian_rag.server import create_app
+
+uvicorn.run(create_app(pipeline), host="127.0.0.1", port=8000)
+```
+
+Endpoints: `GET /health`, `POST /retrieve` (`{"query": "...", "k": 5, "mode": "hybrid"}`), and `POST /query` (`{"question": "...", "k": 5}`). Interactive docs are served at `/docs`.
+
+### Command line
+
+```bash
+persian-rag ingest ./library --index .persian-rag --embedding-cache .cache/emb.sqlite
+persian-rag query "حافظ اهل کجا بود؟" --index .persian-rag --json
+persian-rag eval eval.jsonl --index .persian-rag --k 1 3 5 10
+persian-rag serve --index .persian-rag --port 8000
+```
+
+`ingest` reads `.txt`, `.md`, `.pdf`, and `.docx`, skips hidden files, and writes a UTF-8 JSON index without API keys. Useful flags: `--embedder hashing|sentence-transformers|openai`, `--strategy`, `--chunk-size`, `--rtl-fix`, `--split-pages`, `--mode`, `--rerank cross-encoder|llm`, `--llm openai`, and `--prompt-lang fa|en`. Run `persian-rag <command> --help` for the rest.
+
+## Documentation
+
+```bash
+pip install -e ".[docs]"
+mkdocs serve
+```
+
+The site has matching English and Persian pages for every topic.
+
+## Development
+
+```bash
+pip install -e ".[dev,server,qdrant,pgvector,pdf,docx]"
+ruff check .
+ruff format --check .
+mypy src
+pytest
+```
+
+Tests run offline. Optional-dependency tests skip when the extra is missing. The pgvector integration test runs when `PERSIAN_RAG_PG_DSN` points at PostgreSQL with the `vector` extension, and heavy model tests run with `PERSIAN_RAG_RUN_HEAVY=1`.
+
+Releases are published to PyPI by the `Release` workflow when a `v*` tag is pushed, using trusted publishing (repository variable `PYPI_TRUSTED_PUBLISHING=true`) or a `PYPI_API_TOKEN` secret. Without either, the build still runs and publishing is skipped.
+
+## Roadmap
+
+- Morphological normalization for Persian clitics beyond the affix list.
+- OpenSearch and other durable store connectors.
+- HTML loading and incremental re-indexing of changed files.
+- Citation-style answers with span-level grounding.
+- Evaluation recipes for public Persian QA datasets.
+
+## Contributing
+
+See [CONTRIBUTING.md](CONTRIBUTING.md). Issues and proposals are welcome in Persian or English.
+
+## License
+
+[MIT](LICENSE). Copyright (c) 2026 Naser Sadrmanesh.
+
+## Author
+
+Naser Sadrmanesh
+
+- GitHub: [https://github.com/sadrmanesh](https://github.com/sadrmanesh)
+- Site: [https://sadrmanesh.github.io](https://sadrmanesh.github.io)
+- Email: [info.sadrmanesh@gmail.com](mailto:info.sadrmanesh@gmail.com)