vision-language-engineering-lab 0.2.1__tar.gz → 0.3.2__tar.gz

vision_language_engineering_lab-0.3.2/PKG-INFO

@@ -0,0 +1,1784 @@
+Metadata-Version: 2.4
+Name: vision-language-engineering-lab
+Version: 0.3.2
+Summary: Production-oriented Vision-Language engineering with CLIP, Qwen3-VL, visual document understanding and multimodal RAG.
+Author: GHANMI Helmi
+License-Expression: Apache-2.0
+Project-URL: Homepage, https://github.com/helmighanmi/vision-language-engineering-lab
+Project-URL: Repository, https://github.com/helmighanmi/vision-language-engineering-lab
+Project-URL: Issues, https://github.com/helmighanmi/vision-language-engineering-lab/issues
+Project-URL: Documentation, https://github.com/helmighanmi/vision-language-engineering-lab/tree/main/docs
+Keywords: vision-language-model,vlm,multimodal-ai,clip,qwen3-vl,multimodal-rag,visual-rag,document-ai,image-retrieval
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: <3.14,>=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: numpy<3,>=1.26
+Requires-Dist: pillow<14,>=13
+Provides-Extra: clip
+Requires-Dist: torch<3,>=2.8; extra == "clip"
+Requires-Dist: torchvision<1,>=0.23; extra == "clip"
+Requires-Dist: transformers<6,>=4.57; extra == "clip"
+Provides-Extra: qwen
+Requires-Dist: accelerate<2,>=1.10; extra == "qwen"
+Requires-Dist: huggingface-hub<2,>=0.34; extra == "qwen"
+Requires-Dist: qwen-vl-utils<1,>=0.0.14; extra == "qwen"
+Requires-Dist: torch<3,>=2.8; extra == "qwen"
+Requires-Dist: torchvision<1,>=0.23; extra == "qwen"
+Requires-Dist: transformers<6,>=4.57; extra == "qwen"
+Provides-Extra: retrieval
+Requires-Dist: sentence-transformers<7,>=5.4; extra == "retrieval"
+Provides-Extra: qwen-retrieval
+Requires-Dist: sentence-transformers[image]<7,>=5.4; extra == "qwen-retrieval"
+Requires-Dist: torch<3,>=2.8; extra == "qwen-retrieval"
+Requires-Dist: torchvision<1,>=0.23; extra == "qwen-retrieval"
+Requires-Dist: transformers<6,>=5.0; extra == "qwen-retrieval"
+Requires-Dist: accelerate<2,>=1.10; extra == "qwen-retrieval"
+Requires-Dist: huggingface-hub<2,>=0.34; extra == "qwen-retrieval"
+Requires-Dist: qwen-vl-utils<1,>=0.0.14; extra == "qwen-retrieval"
+Provides-Extra: config
+Requires-Dist: PyYAML<7,>=6.0.2; extra == "config"
+Provides-Extra: notebooks
+Requires-Dist: jupyterlab<5,>=4.6.3; extra == "notebooks"
+Requires-Dist: jupyter-server<3,>=2.21; extra == "notebooks"
+Requires-Dist: matplotlib<4,>=3.9; extra == "notebooks"
+Requires-Dist: pandas<3,>=2.2; extra == "notebooks"
+Provides-Extra: dev
+Requires-Dist: mypy<2,>=1.18; extra == "dev"
+Requires-Dist: huggingface-hub<2,>=0.34; extra == "dev"
+Requires-Dist: PyYAML<7,>=6.0.2; extra == "dev"
+Requires-Dist: types-PyYAML<7,>=6; extra == "dev"
+Requires-Dist: pip-audit<3,>=2.9; extra == "dev"
+Requires-Dist: pytest<10,>=9.0.3; extra == "dev"
+Requires-Dist: pytest-cov<8,>=7.1; extra == "dev"
+Requires-Dist: ruff<1,>=0.12; extra == "dev"
+Provides-Extra: all
+Requires-Dist: PyYAML<7,>=6.0.2; extra == "all"
+Requires-Dist: accelerate<2,>=1.10; extra == "all"
+Requires-Dist: huggingface-hub<2,>=0.34; extra == "all"
+Requires-Dist: qwen-vl-utils<1,>=0.0.14; extra == "all"
+Requires-Dist: sentence-transformers[image]<7,>=5.4; extra == "all"
+Requires-Dist: torch<3,>=2.8; extra == "all"
+Requires-Dist: torchvision<1,>=0.23; extra == "all"
+Requires-Dist: transformers<6,>=5.0; extra == "all"
+Dynamic: license-file
+
+<!--
+Path: README.md
+Author: GHANMI Helmi
+Current Role: AI Engineer
+Past Role: Researcher in Applied Mathematics
+Research Profile: https://www.researchgate.net/profile/Ghanmi-Helmi
+-->
+
+# Vision-Language Engineering Lab
+
+**Production-oriented Vision-Language engineering with CLIP, configurable Qwen3-VL, document understanding, multimodal retrieval, reranking, and RAG.**
+
+`vision-language-engineering-lab` is a Python package for building and studying Vision-Language applications with reusable engineering patterns rather than notebook-only demos.
+
+The implementation lives under `src/vlm_engineering/`. Notebooks are analysis/demo clients, `examples/` provides small API examples, and `scenarios/` contains end-to-end application recipes.
+
+> **v0.3.2 compatibility target:** Python 3.11, 3.12, and 3.13.
+
+v0.3.1 introduced the two-stage multimodal retrieval stack:
+
+```text
+query
+  ↓
+Qwen3-VL-Embedding
+  ↓
+vector retrieval / candidate_k
+  ↓
+Qwen3-VL-Reranker
+  ↓
+top_k multimodal evidence
+  ↓
+original text + images
+  ↓
+Qwen3-VL generation
+  ↓
+grounded answer
+```
+
+Contributions are welcome. See [Contributing](#20-contributing) and [`CONTRIBUTING.md`](CONTRIBUTING.md).
+
+---
+### Interactive architecture map
+
+Explore the package structure, services, and multimodal RAG flow:
+
+[Open the interactive architecture map](https://helmighanmi.github.io/vision-language-engineering-lab/)
+---
+## 1. Quick start
+
+### Install from PyPI
+
+Core package:
+
+```bash
+python -m pip install vision-language-engineering-lab
+```
+
+Qwen3-VL generation:
+
+```bash
+python -m pip install "vision-language-engineering-lab[qwen]"
+```
+
+Qwen multimodal embedding and reranking:
+
+```bash
+python -m pip install "vision-language-engineering-lab[qwen-retrieval]"
+```
+
+All runtime features:
+
+```bash
+python -m pip install "vision-language-engineering-lab[all]"
+```
+
+### Install from the repository
+
+```bash
+git clone https://github.com/helmighanmi/vision-language-engineering-lab.git
+cd vision-language-engineering-lab
+
+python -m venv .venv
+source .venv/bin/activate
+
+python -m pip install --upgrade pip setuptools wheel
+python -m pip install -e ".[all]"
+```
+
+Windows PowerShell:
+
+```powershell
+py -m venv .venv
+.venv\Scripts\Activate.ps1
+
+python -m pip install --upgrade pip setuptools wheel
+python -m pip install -e ".[all]"
+```
+
+### Example 1 — Real-image smoke test with Qwen3-VL 2B
+
+A repository clone includes `data/diagram_random_clean.png` as a real-image smoke-test asset:
+
+```bash
+vlm-lab describe data/diagram_random_clean.png \
+  --model-size 2b \
+  --prompt "Describe this image accurately. List the main objects, text, and important visual relationships."
+```
+
+Equivalent Python API:
+
+```python
+from vlm_engineering import QwenVLModel
+
+model = QwenVLModel(model_size="2b")
+
+answer = model.generate(
+    "data/diagram_random_clean.png",
+    (
+        "Describe this image accurately. "
+        "List the main objects, text, and important visual relationships."
+    ),
+)
+
+print(answer)
+```
+
+The default generative model is:
+
+```text
+Qwen/Qwen3-VL-2B-Instruct
+```
+
+If you installed from PyPI rather than cloning the repository, use your own image path:
+
+```bash
+vlm-lab describe /path/to/your/image.png \
+  --model-size 2b \
+  --prompt "Describe this image accurately."
+```
+
+### Example 2 — Multimodal embeddings
+
+```python
+from vlm_engineering import QwenVLEmbedder
+
+embedder = QwenVLEmbedder(
+    model_size="2b",
+    dimensions=1024,
+)
+
+text_vector = embedder.embed_text(
+    "Find an architecture diagram containing a cache."
+)
+
+image_vector = embedder.embed_image(
+    "data/diagram_random_clean.png"
+)
+
+mixed_vectors = embedder.encode(
+    [
+        {"text": "Redis caching architecture"},
+        {"image": "data/diagram_random_clean.png"},
+        {
+            "text": "Architecture diagram",
+            "image": "data/diagram_random_clean.png",
+        },
+    ]
+)
+
+print(text_vector.shape)
+print(image_vector.shape)
+print(mixed_vectors.shape)
+```
+
+The retrieval input contract is intentionally explicit:
+
+```python
+{"text": "..."}
+{"image": "..."}
+{"text": "...", "image": "..."}
+```
+
+Unknown fields, empty inputs, missing local images, corrupt files, and unsupported inputs are rejected before expensive model inference whenever possible.
+
+### Example 3 — Multimodal reranking
+
+```python
+from vlm_engineering import QwenVLReranker
+
+documents = [
+    {"text": "Redis provides application caching."},
+    {"text": "PostgreSQL stores persistent application data."},
+    {"image": "data/diagram_random_clean.png"},
+]
+
+reranker = QwenVLReranker(model_size="2b")
+
+scores = reranker.score(
+    "Find the evidence related to caching.",
+    documents,
+)
+
+ranked = reranker.rerank(
+    "Find the evidence related to caching.",
+    documents,
+    top_k=2,
+)
+
+print(scores)
+for result in ranked:
+    print(result.rank, result.score, result.item)
+```
+
+Raw scores and normalized scores are both supported. Normalized reranker scores are relevance transformations, not calibrated probabilities.
+
+### Example 4 — True multimodal RAG
+
+```python
+from vlm_engineering import (
+    MultimodalRAGPipeline,
+    NativePageContent,
+    QwenVLEmbedder,
+    QwenVLModel,
+    QwenVLReranker,
+    VisualAnalysis,
+    VisualChunkBuilder,
+)
+
+chunk = VisualChunkBuilder().build(
+    NativePageContent(
+        document_id="demo",
+        page=1,
+        source_file="architecture.pdf",
+        image_ref="data/diagram_random_clean.png",
+    ),
+    VisualAnalysis(
+        page_type="diagram",
+        title="Architecture",
+        summary="Technical architecture diagram.",
+    ),
+)
+
+pipeline = MultimodalRAGPipeline(
+    embedder=QwenVLEmbedder(model_size="2b"),
+    generator=QwenVLModel(model_size="2b"),
+    reranker=QwenVLReranker(model_size="2b"),
+    candidate_k=12,
+    top_k=3,
+    max_new_tokens=256,
+)
+
+pipeline.index_chunks([chunk])
+
+answer = pipeline.answer(
+    "Which components are connected?"
+)
+
+print(answer.answer)
+```
+
+The pipeline performs:
+
+```text
+multimodal embedding
+    ↓
+candidate retrieval
+    ↓
+multimodal reranking
+    ↓
+selected original evidence
+    ↓
+Qwen3-VL generation
+```
+
+### Example 5 — Select 2B / 4B / 8B generative Qwen3-VL
+
+```python
+from vlm_engineering import QwenVLModel
+
+model_2b = QwenVLModel(model_size="2b")
+model_4b = QwenVLModel(model_size="4b")
+model_8b = QwenVLModel(model_size="8b")
+
+print(model_2b.model_source)
+print(model_4b.model_source)
+print(model_8b.model_source)
+```
+
+| Preset | Hugging Face model | Typical use |
+|---|---|---|
+| `2b` | `Qwen/Qwen3-VL-2B-Instruct` | default development / lower-resource inference |
+| `4b` | `Qwen/Qwen3-VL-4B-Instruct` | balanced quality/resource option |
+| `8b` | `Qwen/Qwen3-VL-8B-Instruct` | stronger hardware / higher-capacity inference |
+
+### Example 6 — Local/offline model loading
+
+```python
+from vlm_engineering import QwenVLModel
+
+model = QwenVLModel.from_local(
+    "models/Qwen3-VL-2B-Instruct"
+)
+
+answer = model.generate(
+    "data/diagram_random_clean.png",
+    "Describe the diagram using only the visible evidence.",
+)
+
+print(answer)
+```
+
+`from_local()` uses local-files-only loading and does not silently fall back to the Hugging Face Hub.
+
+### Example 7 — Structured document analysis
+
+```python
+from vlm_engineering import QwenVLModel
+from vlm_engineering.documents import analyze_visual_document
+
+model = QwenVLModel(model_size="4b")
+
+analysis = analyze_visual_document(
+    model,
+    "data/architecture.png",
+)
+
+print(analysis.to_dict())
+```
+
+The structured result can contain page type, title, summary, entities, relations, important text, and uncertainties.
+
+### Example 8 — Text-only visual RAG
+
+When an existing RAG platform supports only text embeddings, the VLM can first turn visual evidence into faithful retrieval text:
+
+```python
+from vlm_engineering.retrieval import TextEmbedder
+
+embedder = TextEmbedder()
+
+embeddings = embedder.encode(
+    [
+        "Redis provides application caching.",
+        "PostgreSQL stores persistent application data.",
+    ]
+)
+
+print(embeddings.shape)
+```
+
+### Example 9 — CLI
+
+```bash
+# List generative Qwen presets
+vlm-lab models
+
+# Describe an image
+vlm-lab describe data/diagram_random_clean.png --model-size 2b
+
+# Structured analysis
+vlm-lab analyze data/architecture.png --model-size 4b
+
+# Multimodal embedding
+vlm-lab embed \
+  --text "Find a cache architecture" \
+  --model-size 2b \
+  --dimensions 1024
+
+# Reranking
+vlm-lab rerank \
+  --query "Find the cache architecture" \
+  --documents-json candidates.json \
+  --model-size 2b
+
+# Validate a runtime profile without loading large models
+vlm-lab validate-config \
+  configs/qwen_multimodal_rag.example.yaml
+```
+
+For complete workflows, see [`scenarios/README.md`](scenarios/README.md).
+
+---
+
+## 2. What this project demonstrates
+
+```text
+CLIP foundations
+   -> image/text embeddings and zero-shot similarity
+
+Qwen3-VL Instruct
+   -> image description
+   -> VQA
+   -> diagram/table/screenshot understanding
+   -> structured visual analysis
+
+Document understanding
+   -> native/OCR evidence + visual semantics
+   -> traceable VisualChunk objects
+
+Text-only visual RAG
+   -> VLM descriptions
+   -> normal text embeddings
+   -> retrieval
+   -> original-image grounding
+
+Multimodal retrieval
+   -> Qwen3-VL-Embedding
+   -> vector recall
+   -> Qwen3-VL-Reranker
+   -> top-k multimodal evidence
+
+Multimodal RAG
+   -> embedding
+   -> candidate retrieval
+   -> reranking
+   -> original text/images
+   -> Qwen3-VL answer
+```
+
+The project intentionally separates:
+
+- reusable package code under `src/vlm_engineering/`
+- deterministic tests under `tests/`
+- small API examples under `examples/`
+- end-to-end recipes under `scenarios/`
+- exploratory notebooks under `notebooks/`
+- architecture and operational documentation under `docs/`
+
+---
+
+## 3. Python compatibility
+
+v0.3.1 targets:
+
+| Python | Support |
+|---|---:|
+| 3.11 | ✅ Supported |
+| 3.12 | ✅ Supported |
+| 3.13 | ✅ Supported |
+
+The package metadata declares:
+
+```text
+>=3.11,<3.14
+```
+
+GitHub Actions validates supported Python versions independently.
+
+Ruff targets Python 3.11 syntax so package code does not accidentally depend on Python 3.12+ syntax.
+
+### Google Colab
+
+When the active Colab runtime Python version is supported:
+
+```python
+!pip install "vision-language-engineering-lab[all]"
+```
+
+Then:
+
+```python
+from vlm_engineering import QwenVLModel
+
+model = QwenVLModel(model_size="2b")
+print(model.model_source)
+```
+
+For Qwen workloads, enable a GPU runtime where available.
+
+---
+
+## 4. Installation options
+
+### Requirements
+
+- Python **3.11, 3.12, or 3.13**
+- Git for source installs
+- enough disk space for selected models
+- a GPU is strongly recommended for practical Qwen3-VL inference
+- CPU-only execution may work for some paths but can be slow or require offloading
+
+### Create a virtual environment
+
+Linux/macOS/GitHub Codespaces:
+
+```bash
+python -m venv .venv
+source .venv/bin/activate
+python -m pip install --upgrade pip setuptools wheel
+```
+
+Windows PowerShell:
+
+```powershell
+py -m venv .venv
+.venv\Scripts\Activate.ps1
+python -m pip install --upgrade pip setuptools wheel
+```
+
+### Choose what to install
+
+Development/testing:
+
+```bash
+python -m pip install -e ".[dev]"
+```
+
+CLIP:
+
+```bash
+python -m pip install -e ".[clip]"
+```
+
+Qwen3-VL generation:
+
+```bash
+python -m pip install -e ".[qwen]"
+```
+
+Text retrieval:
+
+```bash
+python -m pip install -e ".[retrieval]"
+```
+
+Qwen multimodal retrieval:
+
+```bash
+python -m pip install -e ".[qwen-retrieval]"
+```
+
+Runtime YAML profiles:
+
+```bash
+python -m pip install -e ".[config]"
+```
+
+All runtime features:
+
+```bash
+python -m pip install -e ".[all]"
+```
+
+Full contributor environment:
+
+```bash
+python -m pip install -e ".[all,dev,notebooks]"
+```
+
+A compatibility `requirements.txt` is also provided:
+
+```bash
+python -m pip install -r requirements.txt
+```
+
+`pyproject.toml` remains the dependency source of truth.
+
+Confirm the CLI:
+
+```bash
+vlm-lab --help
+vlm-lab models
+```
+
+---
+
+## 5. Qwen models used by this project
+
+### Generative VLM
+
+| Preset | Hugging Face model | Role |
+|---|---|---|
+| `2b` | `Qwen/Qwen3-VL-2B-Instruct` | default generative VLM |
+| `4b` | `Qwen/Qwen3-VL-4B-Instruct` | balanced generative option |
+| `8b` | `Qwen/Qwen3-VL-8B-Instruct` | higher-capacity generative option |
+
+### Multimodal embedding
+
+| Preset | Hugging Face model | Role |
+|---|---|---|
+| `2b` | `Qwen/Qwen3-VL-Embedding-2B` | multimodal recall / shared embedding space |
+| `8b` | `Qwen/Qwen3-VL-Embedding-8B` | higher-capacity multimodal embeddings |
+
+### Multimodal reranking
+
+| Preset | Hugging Face model | Role |
+|---|---|---|
+| `2b` | `Qwen/Qwen3-VL-Reranker-2B` | second-stage candidate reranking |
+| `8b` | `Qwen/Qwen3-VL-Reranker-8B` | higher-capacity reranking |
+
+The size labels are parameter classes, not exact runtime-memory requirements.
+
+Real inference also needs memory for image processing, activations, framework overhead, KV cache/generation state, and model-specific runtime structures.
+
+`trust_remote_code` remains disabled by default where supported by the package.
+
+---
+
+## 6. Model selection and loading
+
+### Generative model
+
+```python
+from vlm_engineering import QwenVLModel
+
+model = QwenVLModel(model_size="2b")
+```
+
+Explicit Hub model:
+
+```python
+model = QwenVLModel.from_hub(
+    "Qwen/Qwen3-VL-4B-Instruct"
+)
+```
+
+Explicit local model:
+
+```python
+model = QwenVLModel.from_local(
+    "models/Qwen3-VL-4B-Instruct"
+)
+```
+
+### Embedding model
+
+```python
+from vlm_engineering import QwenVLEmbedder
+
+embedder = QwenVLEmbedder(
+    model_size="2b",
+    dimensions=1024,
+)
+```
+
+### Reranker
+
+```python
+from vlm_engineering import QwenVLReranker
+
+reranker = QwenVLReranker(
+    model_size="2b",
+)
+```
+
+Model selectors are intentionally explicit. Depending on the API/CLI path, use one of:
+
+```text
+model_size
+model_id
+model_path
+```
+
+Mutually exclusive model-source options prevent accidental fallback to a different model than the one requested.
+
+---
+
+## 7. Hugging Face cache and project-local models
+
+Two storage modes are supported.
+
+### Hub/cache mode
+
+For quick experimentation:
+
+```bash
+vlm-lab describe \
+  data/diagram_random_clean.png \
+  --model-size 2b
+```
+
+The Hugging Face Hub cache is typically reused automatically on subsequent runs.
+
+### Project-local generative model
+
+```bash
+vlm-lab download-model \
+  --model-size 2b \
+  --output models/Qwen3-VL-2B-Instruct
+```
+
+Then:
+
+```bash
+HF_HUB_OFFLINE=1 vlm-lab describe \
+  data/diagram_random_clean.png \
+  --model-path models/Qwen3-VL-2B-Instruct \
+  --prompt "Describe this image accurately."
+```
+
+### Project-local embedding model
+
+```bash
+vlm-lab download-model \
+  --embedding-size 2b \
+  --output models/Qwen3-VL-Embedding-2B
+```
+
+Example offline use:
+
+```bash
+HF_HUB_OFFLINE=1 vlm-lab embed \
+  --text "architecture diagram" \
+  --model-path models/Qwen3-VL-Embedding-2B
+```
+
+### Project-local reranker
+
+```bash
+vlm-lab download-model \
+  --reranker-size 2b \
+  --output models/Qwen3-VL-Reranker-2B
+```
+
+The package does **not** delete a user's shared Hugging Face cache automatically.
+
+Model directories under `models/` are intended for local deployment and remain outside source control.
+
+---
+
+## 8. Advanced model-loading options
+
+### `trust_remote_code`
+
+Default: **False**.
+
+Only enable repository-provided Python code for model repositories you have reviewed and trust.
+
+### Device / cache / revision
+
+The Qwen wrappers support configurable model-loading behavior appropriate to their backend, including device selection, cache location, revision selection, and local-only loading.
+
+For generative inference:
+
+```python
+from vlm_engineering import QwenVLModel
+
+model = QwenVLModel(
+    model_size="4b",
+    device_map="auto",
+    dtype="auto",
+)
+```
+
+For large multimodal retrieval models, conservative defaults are preferred. Embedding/reranking default to small batch sizes to reduce accidental memory pressure.
+
+---
+
+## 9. Core Qwen3-VL use cases
+
+### Image captioning / rich description
+
+```bash
+vlm-lab describe photo.jpg \
+  --prompt "Write a factual, detailed description."
+```
+
+### Visual question answering
+
+```bash
+vlm-lab describe dashboard.png \
+  --model-size 4b \
+  --prompt "Which metric increased the most and what visual evidence supports it?"
+```
+
+### Diagram / architecture understanding
+
+```bash
+vlm-lab analyze architecture.png \
+  --model-size 4b
+```
+
+### Screenshot / UI understanding
+
+Use `describe` or `analyze` for dashboards, application screenshots, forms, and technical UI states.
+
+### Tables and document pages
+
+Render a page as an image and request structured extraction. For production document workflows, preserve native/OCR text separately and use the VLM for layout, grouping, relationships, and visual semantics.
+
+---
+
+## 10. VLM + OCR/native parsing: hybrid by default
+
+```text
+Native extraction / OCR                  Qwen3-VL
+exact text, IDs, numbers                 layout, arrows, groups,
+long deterministic text                 relationships, visual meaning
+             \                           /
+              \                         /
+                       fusion
+                         |
+                   RAG-ready chunk
+```
+
+A VLM can replace OCR in some workflows, but robust document pipelines often preserve native/OCR evidence for literal accuracy while adding VLM semantics where visual structure matters.
+
+---
+
+## 11. Build a RAG-ready visual chunk
+
+```bash
+vlm-lab chunk data/page_001.png \
+  --model-size 4b \
+  --document-id architecture-v1 \
+  --source-file architecture.pdf \
+  --page 1 \
+  --native-text-file data/page_001.txt
+```
+
+The resulting `VisualChunk` preserves traceability such as document ID, source file, page, image reference, entities, relations, metadata, and fused retrieval text.
+
+---
+
+## 12. Visual RAG with a text embedder
+
+```text
+image/page
+   ↓
+Qwen3-VL description / structured analysis
+   ↓
+retrieval text
+   ↓
+Sentence Transformers text embedder
+   ↓
+vector index / vector database
+```
+
+This path is useful when an existing RAG platform supports only text embeddings.
+
+```bash
+python examples/text_only_visual_rag.py
+```
+
+The original `image_ref` is retained so the final VLM can inspect the original image rather than relying only on generated descriptions.
+
+---
+
+## 13. Multimodal retrieval and RAG — v0.3.1
+
+### 13.1 Installation
+
+```bash
+python -m pip install -e ".[qwen-retrieval,config]"
+```
+
+The retrieval stack is designed around:
+
+```text
+Qwen3-VL-Embedding
+        ↓
+candidate recall
+        ↓
+Qwen3-VL-Reranker
+        ↓
+precision-oriented reranking
+        ↓
+Qwen3-VL generation
+```
+
+### 13.2 Multimodal input contract
+
+The public retrieval contract accepts:
+
+```python
+{"text": "..."}
+```
+
+```python
+{"image": "path/to/image.png"}
+```
+
+```python
+{
+    "text": "Find diagrams similar to this",
+    "image": "path/to/query.png",
+}
+```
+
+Plain text strings may be accepted as shorthand where documented.
+
+Local images are validated before expensive inference where practical. Unknown fields, empty inputs, missing files, directories passed as images, corrupt files, and unsupported inputs fail early with readable errors.
+
+### 13.3 Embeddings
+
+```python
+from vlm_engineering import QwenVLEmbedder
+
+embedder = QwenVLEmbedder(
+    model_size="2b",
+    dimensions=1024,
+)
+
+text_vector = embedder.embed_text(
+    "architecture diagram"
+)
+
+image_vector = embedder.embed_image(
+    "data/diagram_random_clean.png"
+)
+
+vectors = embedder.encode(
+    [
+        {"text": "Redis caching"},
+        {"image": "data/diagram_random_clean.png"},
+        {
+            "text": "Architecture",
+            "image": "data/diagram_random_clean.png",
+        },
+    ]
+)
+```
+
+Embeddings are normalized by default where configured by the wrapper.
+
+Supported Qwen embedding presets expose the dimensions supported by their model contract. Invalid dimensions are rejected before inference.
+
+### 13.4 Reranking
+
+```python
+from vlm_engineering import QwenVLReranker
+
+documents = [
+    {"text": "Redis provides caching."},
+    {"text": "PostgreSQL stores persistent state."},
+    {"image": "data/diagram_random_clean.png"},
+]
+
+reranker = QwenVLReranker(model_size="2b")
+
+scores = reranker.score(
+    "Find caching evidence",
+    documents,
+)
+
+ranked = reranker.rerank(
+    "Find caching evidence",
+    documents,
+    top_k=2,
+)
+```
+
+Ranking preserves the original candidate object in each result.
+
+Where normalized scoring is requested, the transformed score should be interpreted as a relevance score rather than a calibrated probability.
+
+### 13.5 Full multimodal RAG
+
+```python
+from vlm_engineering import (
+    MultimodalRAGPipeline,
+    QwenVLEmbedder,
+    QwenVLModel,
+    QwenVLReranker,
+)
+
+pipeline = MultimodalRAGPipeline(
+    embedder=QwenVLEmbedder(model_size="2b"),
+    reranker=QwenVLReranker(model_size="2b"),
+    generator=QwenVLModel(model_size="2b"),
+    candidate_k=12,
+    top_k=3,
+    max_new_tokens=256,
+)
+
+pipeline.index_chunks(chunks)
+
+answer = pipeline.answer(
+    "How does the cache interact with the rest of the architecture?"
+)
+
+print(answer.answer)
+```
+
+Important pipeline behavior:
+
+- `candidate_k >= top_k > 0`
+- re-indexing replaces previous in-memory state
+- empty re-indexing clears the previous collection
+- failed indexing does not intentionally leave a half-built index
+- original document/page/source metadata is preserved
+- original image references remain available for final grounding
+- retrieved evidence is reranked before generation
+- missing visual evidence is reported explicitly when the generation path requires it
+
+### 13.6 Multimodal retrieval queries
+
+A query can include text and image evidence where supported:
+
+```python
+results = pipeline.retrieve(
+    {
+        "text": "Find architecture similar to this diagram.",
+        "image": "data/query_diagram.png",
+    }
+)
+```
+
+### 13.7 CLI
+
+Embedding text:
+
+```bash
+vlm-lab embed \
+  --text "Find a cache diagram" \
+  --model-size 2b \
+  --dimensions 1024
+```
+
+Embedding an image:
+
+```bash
+vlm-lab embed \
+  --image data/diagram_random_clean.png \
+  --model-size 2b
+```
+
+Reranking:
+
+```bash
+vlm-lab rerank \
+  --query "Find the architecture" \
+  --documents-json candidates.json \
+  --model-size 2b
+```
+
+### 13.8 Runtime YAML
+
+Example profile:
+
+```yaml
+version: 1
+
+embedding:
+  model:
+    model_size: 2b
+    revision: null
+    device: null
+    cache_folder: null
+    trust_remote_code: false
+
+  batch_size: 1
+  dimensions: 1024
+  normalize_embeddings: true
+
+reranker:
+  model:
+    model_size: 2b
+    revision: null
+    device: null
+    cache_folder: null
+    trust_remote_code: false
+
+  batch_size: 1
+  normalize_scores: true
+
+rag:
+  top_k: 3
+  candidate_k: 12
+  max_new_tokens: 256
+```
+
+Validate without loading model weights:
+
+```bash
+vlm-lab validate-config \
+  configs/qwen_multimodal_rag.example.yaml
+```
+
+The configuration layer:
+
+- uses safe YAML loading
+- has an explicit schema version
+- validates field types
+- rejects unknown keys
+- rejects conflicting model selectors
+- validates embedding dimensions where possible without model loading
+- rejects invalid `candidate_k` / `top_k` combinations
+
+Example Python usage:
+
+```python
+from vlm_engineering import QwenVLModel
+from vlm_engineering.runtime_config import load_runtime_config
+
+config = load_runtime_config(
+    "configs/qwen_multimodal_rag.example.yaml"
+)
+
+pipeline = config.build_pipeline(
+    QwenVLModel(model_size="2b")
+)
+```
+
+### 13.9 Reliability and operational errors
+
+v0.3.1 strengthens user-facing validation around common operational failures, including:
+
+- missing optional dependencies
+- invalid multimodal input
+- missing/corrupt image files
+- invalid local model directories
+- incomplete local model snapshots where detectable
+- Hugging Face authentication/offline failures
+- resource / out-of-memory failures
+- disk-space failures
+- invalid embedding dimensions
+- unexpected embedding shapes
+- NaN / Inf embeddings
+- invalid reranker outputs
+- NaN / Inf reranker scores
+- malformed YAML profiles
+
+Expected operational problems are mapped to readable errors where possible. Unexpected programming errors should remain visible for debugging rather than being silently swallowed.
+
+### 13.10 Resource safety
+
+Qwen multimodal retrieval models are large.
+
+The retrieval wrappers use conservative batch defaults such as:
+
+```text
+batch_size = 1
+```
+
+Users with stronger hardware can tune batching after validating their memory envelope.
+
+For practical workloads:
+
+- GPU execution is recommended
+- CPU execution can be substantially slower
+- backend offloading may use CPU or disk depending on the model/backend
+- multiple images and high-resolution inputs increase memory requirements
+- local model storage requires several gigabytes depending on the selected model
+
+### 13.11 Real-model validation
+
+Large real-model tests are opt-in and excluded from normal PR CI.
+
+Run:
+
+```bash
+VLM_RUN_REAL_MODEL_TESTS=1 \
+python -m pytest \
+  -m real_model \
+  tests/e2e \
+  -v
+```
+
+Using explicit local/offline models:
+
+```bash
+export VLM_QWEN_EMBEDDING_MODEL_PATH=\
+models/Qwen3-VL-Embedding-2B
+
+export VLM_QWEN_RERANKER_MODEL_PATH=\
+models/Qwen3-VL-Reranker-2B
+
+export VLM_RUN_REAL_MODEL_TESTS=1
+
+python -m pytest \
+  -m real_model \
+  tests/e2e \
+  -v
+```
+
+Normal CI relies on deterministic fake/injected backends rather than downloading multi-gigabyte weights.
+
+### 13.12 Migration from v0.2.x
+
+Where retained by the package, earlier retrieval class names remain compatibility aliases.
+
+New code should prefer:
+
+```python
+from vlm_engineering import (
+    QwenVLEmbedder,
+    QwenVLReranker,
+    MultimodalRAGPipeline,
+)
+```
+
+For image inputs, prefer explicit multimodal objects:
+
+```python
+{"image": "path/to/image.png"}
+```
+
+rather than ambiguous bare path strings.
+
+---
+
+## 14. CLIP foundations
+
+```python
+from PIL import Image
+
+from vlm_engineering import CLIPEncoder
+
+image = Image.open(
+    "data/diagram_random_clean.png"
+).convert("RGB")
+
+clip = CLIPEncoder()
+
+result = clip.zero_shot_classify(
+    image,
+    ["cat", "dog", "airplane"],
+)
+
+print(result)
+```
+
+CLIP is useful for learning the shared image/text embedding-space idea that modern multimodal retrieval builds upon.
+
+---
+
+## 15. Examples and application scenarios
+
+Small API examples:
+
+```text
+examples/clip_zero_shot.py
+examples/qwen_describe_image.py
+examples/qwen_model_selection.py
+examples/qwen_local_model.py
+examples/structured_visual_chunk.py
+examples/text_only_visual_rag.py
+examples/qwen_multimodal_retrieval.py
+```
+
+End-to-end scenarios:
+
+```text
+scenarios/scenario_01_image_captioning.py
+scenarios/scenario_02_visual_question_answering.py
+scenarios/scenario_03_diagram_to_json.py
+scenarios/scenario_04_document_page_to_rag_chunk.py
+scenarios/scenario_05_text_only_visual_rag.py
+scenarios/scenario_06_true_multimodal_retrieval.py
+scenarios/scenario_07_compare_qwen_presets.py
+scenarios/scenario_08_hub_vs_local_loading.py
+scenarios/scenario_09_batch_structured_analysis.py
+```
+
+See [`scenarios/README.md`](scenarios/README.md) for copy/paste commands, model guidance, RAG patterns, batch processing, and offline usage.
+
+---
+
+## 16. Notebooks
+
+Notebooks are analysis clients, not the implementation layer:
+
+```text
+00_clip_foundations.ipynb
+01_qwen3_vl_quickstart.ipynb
+02_visual_document_understanding.ipynb
+03_multimodal_rag.ipynb
+```
+
+Reusable logic belongs under `src/vlm_engineering/` and can be used without Jupyter.
+
+---
+
+## 17. Docker and Docker Compose
+
+The repository includes `Dockerfile` and `docker-compose.yml` for reproducible CLI/container workflows.
+
+The normal image does not bake model weights into the container.
+
+### Build
+
+```bash
+docker compose build
+```
+
+### CLI
+
+```bash
+docker compose run --rm vlm-lab --help
+docker compose run --rm vlm-lab models
+```
+
+### Mounted directories
+
+```text
+Host                       Container
+./data                     /app/data
+./models                   /app/models
+hf-cache named volume      /home/appuser/.cache/huggingface
+```
+
+Explicit model directories under `./models` therefore survive one-shot container removal.
+
+### Generative inference
+
+```bash
+docker compose run --rm vlm-lab \
+  describe data/diagram_random_clean.png \
+  --model-size 2b
+```
+
+### Download a local model
+
+```bash
+docker compose run --rm vlm-lab \
+  download-model \
+  --model-size 2b \
+  --output models/Qwen3-VL-2B-Instruct
+```
+
+### Offline inference
+
+```bash
+docker compose run --rm \
+  -e HF_HUB_OFFLINE=1 \
+  vlm-lab \
+  describe data/diagram_random_clean.png \
+  --model-path models/Qwen3-VL-2B-Instruct
+```
+
+### Retrieval model downloads
+
+```bash
+docker compose run --rm vlm-lab \
+  download-model \
+  --embedding-size 2b \
+  --output models/Qwen3-VL-Embedding-2B
+```
+
+```bash
+docker compose run --rm vlm-lab \
+  download-model \
+  --reranker-size 2b \
+  --output models/Qwen3-VL-Reranker-2B
+```
+
+### Hugging Face authentication
+
+Pass credentials at runtime rather than baking them into the image:
+
+```bash
+export HF_TOKEN="..."
+
+docker compose run --rm \
+  -e HF_TOKEN \
+  vlm-lab \
+  describe data/diagram_random_clean.png \
+  --model-id your-org/your-compatible-model
+```
+
+Do not commit tokens.
+
+### Shell debugging
+
+```bash
+docker compose run --rm \
+  --entrypoint /bin/sh \
+  vlm-lab
+```
+
+### Direct Docker usage
+
+```bash
+docker build \
+  -t vision-language-engineering-lab:0.3.2 \
+  .
+```
+
+```bash
+docker run --rm \
+  vision-language-engineering-lab:0.3.2 \
+  models
+```
+
+### GPU execution
+
+The checked-in Docker setup is a portable baseline and does not force a particular GPU runtime.
+
+Configure the appropriate container GPU runtime on the host and verify GPU visibility before relying on Qwen workloads.
+
+---
+
+## 18. Quality gates and compatibility validation
+
+Before pushing changes:
+
+```bash
+python -m pip check
+python -m ruff check .
+python -m mypy .
+python -m pytest \
+  -m "not real_model" \
+  --cov=vlm_engineering \
+  --cov-report=term-missing
+python -m pip_audit .
+```
+
+The project uses:
+
+| Tool | Purpose |
+|---|---|
+| Ruff | linting, import organization, source-quality rules |
+| mypy | static type checking |
+| pytest | functional, integration, and contract testing |
+| pytest-cov | coverage enforcement |
+| `pip check` | installed dependency consistency |
+| `pip-audit` | known dependency-vulnerability auditing |
+
+For the v0.3.2 release branch, deterministic validation reached:
+
+```text
+347 passed
+2 skipped (opt-in real-model tests)
+95% coverage
+pip check: no broken requirements
+pip-audit: no known vulnerabilities
+```
+
+The deselected tests are resource-heavy / opt-in real-model validations rather than normal CI tests.
+
+CI validates Python 3.11, 3.12, and 3.13 while avoiding automatic multi-gigabyte model downloads in ordinary pull requests.
+
+---
+
+## 19. Troubleshooting
+
+### `ModuleNotFoundError`
+
+Activate the environment and install the appropriate extra:
+
+```bash
+source .venv/bin/activate
+python -m pip install -e ".[all]"
+```
+
+### `vlm-lab: command not found`
+
+```bash
+python -m pip show vision-language-engineering-lab
+which python
+```
+
+Then reinstall:
+
+```bash
+python -m pip install -e ".[all]"
+```
+
+### Missing Qwen retrieval dependencies
+
+```bash
+python -m pip install -e ".[qwen-retrieval]"
+```
+
+### Missing `torchvision`
+
+Install the relevant package extra rather than adding ad-hoc dependencies individually:
+
+```bash
+python -m pip install -e ".[qwen]"
+```
+
+or:
+
+```bash
+python -m pip install -e ".[qwen-retrieval]"
+```
+
+### CUDA / RAM / out-of-memory error
+
+Start with 2B models and batch size 1.
+
+Reduce:
+
+- batch size
+- image resolution
+- number of simultaneous images
+- output token length
+
+or move to hardware with more RAM/VRAM.
+
+### Model offloading is slow
+
+Automatic device placement can offload weights to CPU or disk on constrained machines. This is expected to be slower than running fully on an adequately sized GPU.
+
+### First run is slow
+
+The Hub may be downloading model weights.
+
+For predictable storage:
+
+```bash
+vlm-lab download-model \
+  --model-size 2b
+```
+
+Then use `--model-path`.
+
+### Fully offline inference
+
+Download the model first, then use a local model path and optionally:
+
+```bash
+export HF_HUB_OFFLINE=1
+```
+
+### Invalid/corrupt image
+
+Local image validation intentionally fails early when possible.
+
+Verify:
+
+```bash
+file path/to/image.png
+```
+
+and re-export the source image if necessary.
+
+### Hugging Face authentication
+
+For gated/private models, set:
+
+```bash
+export HF_TOKEN="..."
+```
+
+Do not commit credentials.
+
+### Invalid YAML configuration
+
+Validate before starting models:
+
+```bash
+vlm-lab validate-config \
+  configs/qwen_multimodal_rag.example.yaml
+```
+
+### Dependency compatibility
+
+```bash
+python -m pip check
+python -m pip show \
+  sentence-transformers \
+  torch \
+  torchvision \
+  transformers
+```
+
+### Which Qwen model should I choose?
+
+Start with the 2B preset while validating the pipeline.
+
+Move to larger models only when evaluation data demonstrates that the quality gain justifies the additional hardware cost.
+
+---
+
+## 20. Contributing
+
+**Contributions are welcome.**
+
+The project is intended to grow as an open Vision-Language engineering lab.
+
+Useful contribution areas include:
+
+- VLM integrations
+- multimodal retrieval strategies
+- document-understanding pipelines
+- RAG evaluation
+- persistent vector-database adapters
+- performance and memory improvements
+- deterministic and real-model testing
+- Docker/deployment improvements
+- documentation
+- application scenarios
+- bug fixes
+
+### Contribution workflow
+
+1. Fork or clone the repository.
+2. Create a focused branch.
+3. Implement the change.
+4. Add/update tests.
+5. Run quality gates.
+6. Push the branch.
+7. Open a Pull Request.
+
+Example:
+
+```bash
+git clone https://github.com/helmighanmi/vision-language-engineering-lab.git
+cd vision-language-engineering-lab
+
+python -m venv .venv
+source .venv/bin/activate
+
+python -m pip install --upgrade pip
+python -m pip install -e ".[all,dev]"
+```
+
+Validation:
+
+```bash
+python -m pip check
+python -m ruff check .
+python -m mypy .
+python -m pytest \
+  -m "not real_model" \
+  --cov=vlm_engineering \
+  --cov-report=term-missing
+python -m pip_audit .
+```
+
+See [`CONTRIBUTING.md`](CONTRIBUTING.md) for the complete contribution workflow.
+
+Questions, feature proposals, documentation improvements, and bug reports are welcome through GitHub Issues.
+
+---
+
+## 21. Documentation
+
+- `docs/model-loading.md` — model presets, Hub/custom/local modes and troubleshooting
+- `docs/qwen-model-selection.md` — model-size selection and resource guidance
+- `docs/rag-patterns.md` — visual and multimodal RAG patterns
+- `docs/multimodal-retrieval.md` — Qwen embedding/reranking design and resource guidance
+- `docs/architecture.md` — package architecture
+- `docs/testing.md` — deterministic and real-model testing
+- `docs/pdf/Vision_Language_Engineering_EN.pdf` — English teaching course
+- `docs/pdf/Ingenierie_Vision_Langage_FR.pdf` — French teaching course
+
+The README and Markdown documentation are the operational source of truth for the current package API/CLI.
+
+---
+
+## 22. Repository structure
+
+```text
+vision-language-engineering-lab/
+├── .github/                 CI, security, and publishing workflows
+├── configs/                 validated runtime YAML profiles
+├── data/                    smoke-test/user images
+├── docs/                    architecture, retrieval, RAG, testing guidance
+├── examples/                small runnable API examples
+├── models/                  explicit local model downloads (gitignored)
+├── notebooks/               analysis/demonstration clients
+├── scenarios/               end-to-end application cookbook
+├── src/
+│   └── vlm_engineering/     production Python package
+├── tests/
+│   ├── contract/            metadata/dependency/public-contract tests
+│   ├── e2e/                 opt-in real-model tests
+│   └── unit/                deterministic package tests
+├── Dockerfile
+├── docker-compose.yml
+├── pyproject.toml
+├── requirements.txt
+├── README.md
+├── CHANGELOG.md
+├── CONTRIBUTING.md
+├── SECURITY.md
+└── LICENSE
+```
+
+---
+
+## 23. Implemented vs roadmap
+
+### Implemented
+
+- CLIP foundations
+- Qwen3-VL 2B / 4B / 8B generation
+- image description and visual question answering
+- structured visual/document analysis
+- traceable visual chunks
+- text-only visual RAG
+- Qwen3-VL multimodal embeddings
+- Qwen3-VL multimodal reranking
+- two-stage multimodal retrieval
+- multimodal RAG with original visual evidence
+- project-local model downloads
+- local/offline execution
+- runtime YAML validation
+- CLI workflows
+- Docker workflows
+- Python 3.11–3.13 CI
+- deterministic tests and opt-in real-model tests
+
+### Roadmap
+
+Potential future directions include:
+
+- persistent vector database adapters
+- retrieval/RAG evaluation and benchmarks
+- multi-image and richer multimodal sessions
+- video / temporal reasoning
+- richer document parsing and OCR workflows
+- visual grounding / bounding boxes
+- quantized / low-resource deployment
+- async and streaming interfaces
+- serving/API layer
+- observability and caching
+- visual-agent perception workflows
+
+Roadmap items are not part of the current public API until implemented and validated.
+
+---
+
+## License and third-party material
+
+Project code is distributed under Apache-2.0.
+
+Model weights are downloaded separately and retain their respective licenses. Users are responsible for reviewing the license terms of third-party models, datasets, and external assets used with this package.