multixtract 0.1.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- multixtract-0.1.1/.gitignore +28 -0
- multixtract-0.1.1/CHANGELOG.md +63 -0
- multixtract-0.1.1/LICENSE +21 -0
- multixtract-0.1.1/PKG-INFO +419 -0
- multixtract-0.1.1/README.md +306 -0
- multixtract-0.1.1/pyproject.toml +95 -0
- multixtract-0.1.1/src/multixtract/__init__.py +52 -0
- multixtract-0.1.1/src/multixtract/chunking.py +297 -0
- multixtract-0.1.1/src/multixtract/cli.py +75 -0
- multixtract-0.1.1/src/multixtract/extraction.py +44 -0
- multixtract-0.1.1/src/multixtract/extractors/__init__.py +84 -0
- multixtract-0.1.1/src/multixtract/extractors/_image_utils.py +157 -0
- multixtract-0.1.1/src/multixtract/extractors/docx.py +375 -0
- multixtract-0.1.1/src/multixtract/extractors/eml.py +187 -0
- multixtract-0.1.1/src/multixtract/extractors/epub.py +142 -0
- multixtract-0.1.1/src/multixtract/extractors/excel.py +321 -0
- multixtract-0.1.1/src/multixtract/extractors/html.py +127 -0
- multixtract-0.1.1/src/multixtract/extractors/image.py +149 -0
- multixtract-0.1.1/src/multixtract/extractors/legacy.py +148 -0
- multixtract-0.1.1/src/multixtract/extractors/markdown.py +104 -0
- multixtract-0.1.1/src/multixtract/extractors/pdf.py +347 -0
- multixtract-0.1.1/src/multixtract/extractors/pptx.py +303 -0
- multixtract-0.1.1/src/multixtract/extractors/registry.py +72 -0
- multixtract-0.1.1/src/multixtract/extractors/rtf.py +54 -0
- multixtract-0.1.1/src/multixtract/extractors/text.py +50 -0
- multixtract-0.1.1/src/multixtract/filters.py +181 -0
- multixtract-0.1.1/src/multixtract/interfaces.py +151 -0
- multixtract-0.1.1/src/multixtract/pipeline.py +250 -0
- multixtract-0.1.1/src/multixtract/providers/__init__.py +38 -0
- multixtract-0.1.1/src/multixtract/providers/_utils.py +32 -0
- multixtract-0.1.1/src/multixtract/providers/azure.py +91 -0
- multixtract-0.1.1/src/multixtract/providers/llama.py +151 -0
- multixtract-0.1.1/src/multixtract/providers/openai.py +125 -0
- multixtract-0.1.1/src/multixtract/providers/qwen2vl.py +158 -0
- multixtract-0.1.1/src/multixtract/providers/smolvlm.py +152 -0
- multixtract-0.1.1/src/multixtract/providers/storage.py +97 -0
- multixtract-0.1.1/src/multixtract/py.typed +1 -0
- multixtract-0.1.1/src/multixtract/vision.py +120 -0
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*.egg-info/
|
|
5
|
+
.eggs/
|
|
6
|
+
build/
|
|
7
|
+
dist/
|
|
8
|
+
.venv/
|
|
9
|
+
venv/
|
|
10
|
+
|
|
11
|
+
# Test / coverage
|
|
12
|
+
.pytest_cache/
|
|
13
|
+
.coverage
|
|
14
|
+
htmlcov/
|
|
15
|
+
|
|
16
|
+
# IDE
|
|
17
|
+
.vscode/
|
|
18
|
+
.idea/
|
|
19
|
+
|
|
20
|
+
# Secrets — never commit credentials
|
|
21
|
+
.env
|
|
22
|
+
*.key
|
|
23
|
+
secrets.toml
|
|
24
|
+
|
|
25
|
+
# Local extraction output
|
|
26
|
+
output_folder/
|
|
27
|
+
extracted_images/
|
|
28
|
+
*.json
|
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project will be documented here.
|
|
4
|
+
Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
|
|
5
|
+
This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
6
|
+
|
|
7
|
+
## [0.1.0] — 2026-07-29
|
|
8
|
+
|
|
9
|
+
### Added
|
|
10
|
+
|
|
11
|
+
**Core extraction**
|
|
12
|
+
- `extract_document()` — text, tables, and filtered images from PDF, Word (.docx), PowerPoint (.pptx), Excel/CSV (.xlsx/.csv)
|
|
13
|
+
- Legacy `.doc` / `.ppt` support via headless LibreOffice conversion
|
|
14
|
+
- `chunk_document()` — sliding-window text chunks, per-table Markdown chunks, per-image chunks
|
|
15
|
+
- `split_text_into_chunks()` — standalone sentence-boundary-aware text splitter
|
|
16
|
+
- `table_to_markdown()` — serialize extracted tables to Markdown
|
|
17
|
+
|
|
18
|
+
**Image filtering (`ImageFilterPipeline`)**
|
|
19
|
+
- Dimension filter (configurable major/minor pixel thresholds)
|
|
20
|
+
- Solid-colour and tiny-icon rejection (perceptual quality checks)
|
|
21
|
+
- Reference-logo deduplication via perceptual hash (`reference_img_dir`)
|
|
22
|
+
- Cross-page duplicate tracking via xref/media-path deduplication
|
|
23
|
+
- Vector image pre-conversion (EMF/WMF/SVG → PNG via LibreOffice)
|
|
24
|
+
- JPEG-XR / WDP decoding via `imagecodecs` (`[imaging]` extra)
|
|
25
|
+
|
|
26
|
+
**Pipeline (`Pipeline`, `PipelineConfig`)**
|
|
27
|
+
- End-to-end orchestration: extract → filter → vision → chunk → embed → store
|
|
28
|
+
- All providers optional: `vision=None`, `embedder=None`, `store=None`
|
|
29
|
+
- Parallel vision calls (`vision_workers`), batched embeddings
|
|
30
|
+
- Resume support via `skip_if_exists` (skips documents already in the store)
|
|
31
|
+
- Configurable storage sub-folder layout
|
|
32
|
+
|
|
33
|
+
**Vision providers** (cloud)
|
|
34
|
+
- `OpenAIVisionModel` — GPT-4o and compatible models (`[openai]`)
|
|
35
|
+
- `AzureOpenAIVisionModel` — Azure OpenAI deployment (`[azure]`)
|
|
36
|
+
- Custom `system_prompt`, `max_tokens`, `temperature`, shared `client` support
|
|
37
|
+
|
|
38
|
+
**Vision providers** (local / offline)
|
|
39
|
+
- `Qwen2VLVisionModel` — Qwen2.5-VL-7B/3B, recommended for document understanding (`[qwen2vl]`)
|
|
40
|
+
- `SmolVLMVisionModel` — SmolVLM 2.2B, CPU-friendly, better accuracy than Moondream2 (`[smolvlm]`)
|
|
41
|
+
- `Llama32VisionModel` — Llama 3.2 Vision 11B/90B (`[llama]`)
|
|
42
|
+
|
|
43
|
+
**Embedding providers**
|
|
44
|
+
- `OpenAIEmbedder` — OpenAI embeddings API (`[openai]`)
|
|
45
|
+
- `AzureOpenAIEmbedder` — Azure OpenAI embeddings deployment (`[azure]`)
|
|
46
|
+
|
|
47
|
+
**Storage providers**
|
|
48
|
+
- `LocalDiskStore` — writes JSON and image files to the local filesystem
|
|
49
|
+
- `AzureBlobStore` — Azure Blob Storage with key, service principal, or `DefaultAzureCredential` auth (`[azure]`)
|
|
50
|
+
|
|
51
|
+
**Extensibility**
|
|
52
|
+
- `DocumentExtractor` protocol — add any file format with `register_extractor()`
|
|
53
|
+
- `VisionModel` protocol — plug in any vision model (Tesseract, Claude, Gemini, …)
|
|
54
|
+
- `Embedder` protocol — plug in any embedding model (sentence-transformers, Cohere, …)
|
|
55
|
+
- `BlobStore` protocol — plug in any storage backend (S3, GCS, …)
|
|
56
|
+
|
|
57
|
+
**CLI**
|
|
58
|
+
- `multixtract <file>` — extract any supported document to JSON; vision/embeddings enabled when `OPENAI_API_KEY` is set
|
|
59
|
+
|
|
60
|
+
**Packaging**
|
|
61
|
+
- `py.typed` marker — fully typed, compatible with mypy and pyright
|
|
62
|
+
- Optional extras: `[pdf]`, `[docx]`, `[pptx]`, `[xlsx]`, `[imaging]`, `[openai]`, `[azure]`, `[qwen2vl]`, `[smolvlm]`, `[llama]`, `[dev]`, `[all]`
|
|
63
|
+
- Core dependencies: Pillow, ImageHash only
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Namrata Srivastava
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,419 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: multixtract
|
|
3
|
+
Version: 0.1.1
|
|
4
|
+
Summary: Vendor-neutral PDF/document extraction: text, tables, and vision-described images, chunked and embedded for search & RAG.
|
|
5
|
+
Project-URL: Homepage, https://github.com/srivnamrata/multixtract
|
|
6
|
+
Project-URL: Documentation, https://github.com/srivnamrata/multixtract#readme
|
|
7
|
+
Project-URL: Issues, https://github.com/srivnamrata/multixtract/issues
|
|
8
|
+
Project-URL: Changelog, https://github.com/srivnamrata/multixtract/blob/main/CHANGELOG.md
|
|
9
|
+
Author-email: Namrata Srivastava <srivnamrata@yahoo.co.in>
|
|
10
|
+
License: MIT License
|
|
11
|
+
|
|
12
|
+
Copyright (c) 2026 Namrata Srivastava
|
|
13
|
+
|
|
14
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
15
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
16
|
+
in the Software without restriction, including without limitation the rights
|
|
17
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
18
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
19
|
+
furnished to do so, subject to the following conditions:
|
|
20
|
+
|
|
21
|
+
The above copyright notice and this permission notice shall be included in all
|
|
22
|
+
copies or substantial portions of the Software.
|
|
23
|
+
|
|
24
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
25
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
26
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
27
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
28
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
29
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
30
|
+
SOFTWARE.
|
|
31
|
+
License-File: LICENSE
|
|
32
|
+
Keywords: chunking,embeddings,extraction,ocr,pdf,rag,vision
|
|
33
|
+
Classifier: Development Status :: 3 - Alpha
|
|
34
|
+
Classifier: Environment :: Console
|
|
35
|
+
Classifier: Intended Audience :: Developers
|
|
36
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
37
|
+
Classifier: Operating System :: OS Independent
|
|
38
|
+
Classifier: Programming Language :: Python :: 3
|
|
39
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
40
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
41
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
42
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
43
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
44
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
45
|
+
Classifier: Topic :: Scientific/Engineering :: Image Recognition
|
|
46
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
47
|
+
Classifier: Topic :: Text Processing
|
|
48
|
+
Classifier: Topic :: Text Processing :: Indexing
|
|
49
|
+
Requires-Python: >=3.9
|
|
50
|
+
Requires-Dist: imagehash>=4.3
|
|
51
|
+
Requires-Dist: pillow>=10.0
|
|
52
|
+
Provides-Extra: all
|
|
53
|
+
Requires-Dist: accelerate<2.0,>=0.27; extra == 'all'
|
|
54
|
+
Requires-Dist: azure-identity<2.0,>=1.15; extra == 'all'
|
|
55
|
+
Requires-Dist: azure-storage-blob<13.0,>=12.19; extra == 'all'
|
|
56
|
+
Requires-Dist: beautifulsoup4>=4.12; extra == 'all'
|
|
57
|
+
Requires-Dist: ebooklib>=0.18; extra == 'all'
|
|
58
|
+
Requires-Dist: imagecodecs>=2023.1.23; extra == 'all'
|
|
59
|
+
Requires-Dist: openai<2.0,>=1.57.0; extra == 'all'
|
|
60
|
+
Requires-Dist: openpyxl>=3.1; extra == 'all'
|
|
61
|
+
Requires-Dist: pdfplumber>=0.10; extra == 'all'
|
|
62
|
+
Requires-Dist: pymupdf>=1.23; extra == 'all'
|
|
63
|
+
Requires-Dist: python-docx>=1.1; extra == 'all'
|
|
64
|
+
Requires-Dist: python-pptx>=0.6.21; extra == 'all'
|
|
65
|
+
Requires-Dist: striprtf>=0.0.26; extra == 'all'
|
|
66
|
+
Requires-Dist: torch<3.0,>=2.1; extra == 'all'
|
|
67
|
+
Requires-Dist: transformers<5.0,>=4.45; extra == 'all'
|
|
68
|
+
Requires-Dist: transformers<5.0,>=4.49; extra == 'all'
|
|
69
|
+
Provides-Extra: azure
|
|
70
|
+
Requires-Dist: azure-identity<2.0,>=1.15; extra == 'azure'
|
|
71
|
+
Requires-Dist: azure-storage-blob<13.0,>=12.19; extra == 'azure'
|
|
72
|
+
Requires-Dist: openai<2.0,>=1.57.0; extra == 'azure'
|
|
73
|
+
Provides-Extra: dev
|
|
74
|
+
Requires-Dist: build>=1.2; extra == 'dev'
|
|
75
|
+
Requires-Dist: mkdocs-include-markdown-plugin>=6.0; extra == 'dev'
|
|
76
|
+
Requires-Dist: mkdocs-material>=9.5; extra == 'dev'
|
|
77
|
+
Requires-Dist: pytest-cov>=5.0; extra == 'dev'
|
|
78
|
+
Requires-Dist: pytest>=7.4; extra == 'dev'
|
|
79
|
+
Requires-Dist: ruff>=0.4; extra == 'dev'
|
|
80
|
+
Requires-Dist: twine>=5.0; extra == 'dev'
|
|
81
|
+
Provides-Extra: docx
|
|
82
|
+
Requires-Dist: python-docx>=1.1; extra == 'docx'
|
|
83
|
+
Provides-Extra: epub
|
|
84
|
+
Requires-Dist: beautifulsoup4>=4.12; extra == 'epub'
|
|
85
|
+
Requires-Dist: ebooklib>=0.18; extra == 'epub'
|
|
86
|
+
Provides-Extra: html
|
|
87
|
+
Requires-Dist: beautifulsoup4>=4.12; extra == 'html'
|
|
88
|
+
Provides-Extra: imaging
|
|
89
|
+
Requires-Dist: imagecodecs>=2023.1.23; extra == 'imaging'
|
|
90
|
+
Provides-Extra: llama
|
|
91
|
+
Requires-Dist: accelerate<2.0,>=0.27; extra == 'llama'
|
|
92
|
+
Requires-Dist: torch<3.0,>=2.1; extra == 'llama'
|
|
93
|
+
Requires-Dist: transformers<5.0,>=4.45; extra == 'llama'
|
|
94
|
+
Provides-Extra: openai
|
|
95
|
+
Requires-Dist: openai<2.0,>=1.57.0; extra == 'openai'
|
|
96
|
+
Provides-Extra: pdf
|
|
97
|
+
Requires-Dist: pdfplumber>=0.10; extra == 'pdf'
|
|
98
|
+
Requires-Dist: pymupdf>=1.23; extra == 'pdf'
|
|
99
|
+
Provides-Extra: pptx
|
|
100
|
+
Requires-Dist: python-pptx>=0.6.21; extra == 'pptx'
|
|
101
|
+
Provides-Extra: qwen2vl
|
|
102
|
+
Requires-Dist: accelerate<2.0,>=0.27; extra == 'qwen2vl'
|
|
103
|
+
Requires-Dist: torch<3.0,>=2.1; extra == 'qwen2vl'
|
|
104
|
+
Requires-Dist: transformers<5.0,>=4.49; extra == 'qwen2vl'
|
|
105
|
+
Provides-Extra: rtf
|
|
106
|
+
Requires-Dist: striprtf>=0.0.26; extra == 'rtf'
|
|
107
|
+
Provides-Extra: smolvlm
|
|
108
|
+
Requires-Dist: torch<3.0,>=2.1; extra == 'smolvlm'
|
|
109
|
+
Requires-Dist: transformers<5.0,>=4.49; extra == 'smolvlm'
|
|
110
|
+
Provides-Extra: xlsx
|
|
111
|
+
Requires-Dist: openpyxl>=3.1; extra == 'xlsx'
|
|
112
|
+
Description-Content-Type: text/markdown
|
|
113
|
+
|
|
114
|
+
# multixtract
|
|
115
|
+
|
|
116
|
+
Vendor-neutral document extraction for search & RAG. Pull **text, tables, and images** out of PDFs, Word, PowerPoint, and Excel/CSV files, let any **vision model** describe the images, **chunk** everything into bite-size pieces, **embed** them, and store the result anywhere.
|
|
117
|
+
|
|
118
|
+
The core is tiny (just `Pillow` + `ImageHash`). Every **format parser** and every **cloud SDK** is an **optional extra** — install only what you need and plug in OpenAI, Azure OpenAI, a local model, Azure Blob, S3, or local disk.
|
|
119
|
+
|
|
120
|
+
## Install
|
|
121
|
+
|
|
122
|
+
```bash
|
|
123
|
+
pip install multixtract # core only — framework + image filters
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
### Format extractors
|
|
127
|
+
|
|
128
|
+
Install the formats you need (each lazy-loads its parser; calling an extractor without its extra raises a clear `pip install` hint):
|
|
129
|
+
|
|
130
|
+
| Extra | Formats | Pulls in |
|
|
131
|
+
|---|---|---|
|
|
132
|
+
| `[pdf]` | `.pdf` | PyMuPDF, pdfplumber |
|
|
133
|
+
| `[docx]` | `.docx` (+ legacy `.doc`\*) | python-docx |
|
|
134
|
+
| `[pptx]` | `.pptx` (+ legacy `.ppt`\*) | python-pptx |
|
|
135
|
+
| `[xlsx]` | `.xlsx`, `.xlsm`, `.csv` | openpyxl |
|
|
136
|
+
| `[imaging]` | decode `.wdp` / JPEG-XR images embedded in pptx/xlsx | imagecodecs |
|
|
137
|
+
|
|
138
|
+
```bash
|
|
139
|
+
pip install "multixtract[pdf]" # just PDFs
|
|
140
|
+
pip install "multixtract[pdf,docx,pptx,xlsx]" # all document formats
|
|
141
|
+
```
|
|
142
|
+
|
|
143
|
+
\* Legacy `.doc` / `.ppt` are converted via a system **LibreOffice** install (headless) then parsed natively (`.doc`→docx, `.ppt`→pptx). EMF/WMF/SVG vector images also require LibreOffice.
|
|
144
|
+
|
|
145
|
+
### Providers
|
|
146
|
+
|
|
147
|
+
| Extra | Adds |
|
|
148
|
+
|---|---|
|
|
149
|
+
| `[openai]` | OpenAI vision & embeddings |
|
|
150
|
+
| `[azure]` | Azure OpenAI + Azure Blob Storage |
|
|
151
|
+
| `[qwen2vl]` | **Qwen2.5-VL** — recommended local vision model (leads 7B class on DocVQA/ChartQA; GPU 16–24 GB recommended) |
|
|
152
|
+
| `[smolvlm]` | **SmolVLM 2.2B** — CPU-friendly local vision model, better accuracy than Moondream |
|
|
153
|
+
| `[llama]` | **Llama 3.2 Vision** — strong free alternative (11B; GPU 16 GB recommended) |
|
|
154
|
+
| `[all]` | all formats + imaging + all providers |
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
pip install "multixtract[openai]" # + OpenAI vision & embeddings
|
|
158
|
+
pip install "multixtract[azure]" # + Azure OpenAI & Azure Blob Storage
|
|
159
|
+
pip install "multixtract[qwen2vl]" # + Qwen2.5-VL local vision (GPU recommended)
|
|
160
|
+
pip install "multixtract[smolvlm]" # + SmolVLM 2.2B local vision (CPU-friendly)
|
|
161
|
+
pip install "multixtract[all]" # everything
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
## Quick start
|
|
165
|
+
|
|
166
|
+
```python
|
|
167
|
+
from multixtract import Pipeline
|
|
168
|
+
from multixtract.providers import OpenAIVisionModel, OpenAIEmbedder
|
|
169
|
+
from multixtract.providers.storage import LocalDiskStore
|
|
170
|
+
|
|
171
|
+
pipeline = Pipeline(
|
|
172
|
+
vision=OpenAIVisionModel(api_key="sk-...", model="gpt-4o"),
|
|
173
|
+
embedder=OpenAIEmbedder(api_key="sk-...", model="text-embedding-3-large", dim=1024),
|
|
174
|
+
store=LocalDiskStore("./output_folder"),
|
|
175
|
+
)
|
|
176
|
+
|
|
177
|
+
result = pipeline.process("report.pdf") # also .docx / .pptx / .xlsx / .csv
|
|
178
|
+
print(result.document) # {metadata, pgs:[{txt, tables, imgs:[...]}]}
|
|
179
|
+
print(result.chunks) # [{chunk_id, chunk_type, content, embedding, ...}]
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
## Recipes — use only the parts you need
|
|
183
|
+
|
|
184
|
+
Extraction, vision (OCR/description), chunking, and embedding are fully **decoupled**. Call only the steps you want — no `Pipeline` required.
|
|
185
|
+
|
|
186
|
+
### Extract only — no chunking, no embedding
|
|
187
|
+
|
|
188
|
+
```python
|
|
189
|
+
from multixtract import extract_document # needs multixtract[pdf]
|
|
190
|
+
|
|
191
|
+
document, images = extract_document("report.pdf") # .docx / .pptx / .xlsx / .csv too
|
|
192
|
+
|
|
193
|
+
for page in document["pgs"]:
|
|
194
|
+
print(f"--- page {page['pg_num']} ---")
|
|
195
|
+
print(page["txt"]) # plain text
|
|
196
|
+
for table in page["tables"]: # each table is a list of row-lists
|
|
197
|
+
print(table)
|
|
198
|
+
|
|
199
|
+
# `images` = filtered, de-duplicated images ready for analysis.
|
|
200
|
+
# NOTE: no vision model was called — these are raw image bytes + metadata.
|
|
201
|
+
for img in images:
|
|
202
|
+
print(img["image_id"], img["page_number"], img["width"], "x", img["height"])
|
|
203
|
+
```
|
|
204
|
+
|
|
205
|
+
No API keys, no cloud SDKs, no `chunk_document` — just text, tables, and the filtered image bytes.
|
|
206
|
+
|
|
207
|
+
### Extract + chunk, but don't embed
|
|
208
|
+
|
|
209
|
+
```python
|
|
210
|
+
from multixtract import extract_document, chunk_document
|
|
211
|
+
|
|
212
|
+
document, _ = extract_document("timetable.pdf")
|
|
213
|
+
chunks = chunk_document(document, base_name="timetable") # each chunk has embedding=None
|
|
214
|
+
```
|
|
215
|
+
|
|
216
|
+
### OCR images with a vision model — no embedding
|
|
217
|
+
|
|
218
|
+
OCR text comes from a `VisionModel` (e.g. GPT-4o vision), which also returns a caption and a longer description. Run it directly on the filtered images and skip the embedder/chunker entirely.
|
|
219
|
+
|
|
220
|
+
```python
|
|
221
|
+
from multixtract import extract_document
|
|
222
|
+
from multixtract.providers import OpenAIVisionModel # needs multixtract[openai]
|
|
223
|
+
|
|
224
|
+
vision = OpenAIVisionModel(api_key="sk-...", model="gpt-4o")
|
|
225
|
+
|
|
226
|
+
document, images = extract_document("scanned.pdf") # needs multixtract[pdf]
|
|
227
|
+
for img in images:
|
|
228
|
+
result = vision.analyze(
|
|
229
|
+
image_bytes=img["image_bytes"],
|
|
230
|
+
ext=img["ext"],
|
|
231
|
+
width=img["width"],
|
|
232
|
+
height=img["height"],
|
|
233
|
+
)
|
|
234
|
+
print(img["image_id"], "| OCR:", result.ocr_text)
|
|
235
|
+
print(" caption:", result.caption)
|
|
236
|
+
print(" description:", result.description)
|
|
237
|
+
```
|
|
238
|
+
|
|
239
|
+
On **Azure OpenAI**, swap in the Azure provider (`multixtract[azure]`) and pass your endpoint + deployment. Keep secrets out of code — inject them via environment variables or a secrets manager:
|
|
240
|
+
|
|
241
|
+
```python
|
|
242
|
+
from multixtract.providers import AzureOpenAIVisionModel
|
|
243
|
+
|
|
244
|
+
vision = AzureOpenAIVisionModel(
|
|
245
|
+
endpoint="https://<resource>.openai.azure.com",
|
|
246
|
+
api_key=AZURE_OPENAI_KEY, # injected, never hard-coded
|
|
247
|
+
deployment="gpt-4o",
|
|
248
|
+
)
|
|
249
|
+
# vision.analyze(...) exactly as above
|
|
250
|
+
```
|
|
251
|
+
|
|
252
|
+
### Bring your own OCR — fully offline (no cloud)
|
|
253
|
+
|
|
254
|
+
A `VisionModel` is just any object with an `analyze()` method (structural typing — no subclassing or cloud SDK needed). Here's a zero-cloud one backed by [Tesseract](https://github.com/tesseract-ocr/tesseract) (`pip install pytesseract`, plus a system `tesseract` binary):
|
|
255
|
+
|
|
256
|
+
```python
|
|
257
|
+
import io
|
|
258
|
+
import pytesseract
|
|
259
|
+
from PIL import Image
|
|
260
|
+
from multixtract import extract_document
|
|
261
|
+
from multixtract.interfaces import VisionResult
|
|
262
|
+
|
|
263
|
+
class TesseractVisionModel:
|
|
264
|
+
"""Offline OCR-only VisionModel — no network, no API key."""
|
|
265
|
+
def analyze(self, image_bytes, ext="png", width=0, height=0) -> VisionResult:
|
|
266
|
+
try:
|
|
267
|
+
text = pytesseract.image_to_string(Image.open(io.BytesIO(image_bytes)))
|
|
268
|
+
except Exception:
|
|
269
|
+
return VisionResult() # never break the caller
|
|
270
|
+
return VisionResult(ocr_text=text.strip())
|
|
271
|
+
|
|
272
|
+
vision = TesseractVisionModel()
|
|
273
|
+
document, images = extract_document("scanned.pdf") # needs multixtract[pdf]
|
|
274
|
+
for img in images:
|
|
275
|
+
print(img["image_id"], "| OCR:", vision.analyze(img["image_bytes"], img["ext"]).ocr_text)
|
|
276
|
+
```
|
|
277
|
+
|
|
278
|
+
Because it satisfies the same `VisionModel` interface as the cloud providers, you can also drop it straight into the full pipeline — `Pipeline(vision=TesseractVisionModel(), embedder=..., store=...)` — for offline OCR end-to-end.
|
|
279
|
+
|
|
280
|
+
### Local vision models — offline, no API key
|
|
281
|
+
|
|
282
|
+
Three local model options are available. All return the same `VisionResult` structure and work as drop-in replacements for the cloud providers.
|
|
283
|
+
|
|
284
|
+
#### Qwen2.5-VL (recommended)
|
|
285
|
+
|
|
286
|
+
Best accuracy for document images — leads the 7B class on DocVQA, ChartQA, TextVQA, and OCR benchmarks as of 2025. Requires a GPU with 16–24 GB VRAM for BF16; use the 3B variant or `load_in_4bit=True` for smaller cards.
|
|
287
|
+
|
|
288
|
+
```bash
|
|
289
|
+
pip install "multixtract[qwen2vl]"
|
|
290
|
+
```
|
|
291
|
+
|
|
292
|
+
```python
|
|
293
|
+
from multixtract import extract_document
|
|
294
|
+
from multixtract.providers import Qwen2VLVisionModel
|
|
295
|
+
|
|
296
|
+
# Default: 7B. Use "Qwen/Qwen2.5-VL-3B-Instruct" for lower VRAM.
|
|
297
|
+
vision = Qwen2VLVisionModel()
|
|
298
|
+
document, images = extract_document("report.pdf")
|
|
299
|
+
for img in images:
|
|
300
|
+
r = vision.analyze(img["image_bytes"], ext=img["ext"])
|
|
301
|
+
print(r.caption, "|", r.description)
|
|
302
|
+
|
|
303
|
+
# Drop into the full pipeline:
|
|
304
|
+
Pipeline(vision=Qwen2VLVisionModel(), embedder=my_embedder, store=my_store).process("report.pdf")
|
|
305
|
+
```
|
|
306
|
+
|
|
307
|
+
#### SmolVLM 2.2B (CPU-friendly)
|
|
308
|
+
|
|
309
|
+
At 2.2B parameters, SmolVLM runs on CPU without impractical wait times and delivers meaningfully better DocVQA and ChartQA accuracy than Moondream2. No `trust_remote_code` required. Use it when a GPU is unavailable.
|
|
310
|
+
|
|
311
|
+
```bash
|
|
312
|
+
pip install "multixtract[smolvlm]"
|
|
313
|
+
```
|
|
314
|
+
|
|
315
|
+
```python
|
|
316
|
+
from multixtract.providers import SmolVLMVisionModel
|
|
317
|
+
|
|
318
|
+
vision = SmolVLMVisionModel() # ~4 GB download on first use
|
|
319
|
+
vision = SmolVLMVisionModel("HuggingFaceTB/SmolVLM-500M-Instruct") # 500M for extreme constraints
|
|
320
|
+
document, images = extract_document("report.pdf")
|
|
321
|
+
for img in images:
|
|
322
|
+
r = vision.analyze(img["image_bytes"], ext=img["ext"])
|
|
323
|
+
print(r.caption, "|", r.ocr_text)
|
|
324
|
+
```
|
|
325
|
+
|
|
326
|
+
#### Llama 3.2 Vision
|
|
327
|
+
|
|
328
|
+
Strong free alternative, especially for users already in the Meta/Llama ecosystem. Requires ≥16 GB VRAM for the 11B model.
|
|
329
|
+
|
|
330
|
+
```python
|
|
331
|
+
from multixtract.providers import Llama32VisionModel # pip install "multixtract[llama]"
|
|
332
|
+
|
|
333
|
+
vision = Llama32VisionModel() # 11B default
|
|
334
|
+
vision = Llama32VisionModel("meta-llama/Llama-3.2-90B-Vision-Instruct") # 90B, highest accuracy
|
|
335
|
+
vision = Llama32VisionModel(load_in_4bit=True) # 4-bit, needs bitsandbytes
|
|
336
|
+
```
|
|
337
|
+
|
|
338
|
+
## Architecture
|
|
339
|
+
|
|
340
|
+
```
|
|
341
|
+
file → extract (text/tables/images) → filter images → vision describe
|
|
342
|
+
→ chunk (text/table/image) → embed → store (JSON)
|
|
343
|
+
```
|
|
344
|
+
|
|
345
|
+
The right extractor is chosen by file extension via a registry; the pipeline talks only to three **interfaces** — it never imports a vendor directly:
|
|
346
|
+
|
|
347
|
+
| Interface | Job | Built-in implementations |
|
|
348
|
+
|---|---|---|
|
|
349
|
+
| `VisionModel` | image → caption + OCR + description | `OpenAIVisionModel`, `AzureOpenAIVisionModel`, `Llama32VisionModel` |
|
|
350
|
+
| `Embedder` | text → vector | `OpenAIEmbedder`, `AzureOpenAIEmbedder` |
|
|
351
|
+
| `BlobStore` | save bytes/JSON | `LocalDiskStore`, `AzureBlobStore` |
|
|
352
|
+
|
|
353
|
+
Write your own by implementing the same methods (e.g. a local vision model, a sentence-transformers embedder, or an S3 store). Add a new format by implementing `DocumentExtractor` and calling `register_extractor`.
|
|
354
|
+
|
|
355
|
+
## Features
|
|
356
|
+
|
|
357
|
+
* **Multi-format**: PDF, Word, PowerPoint, Excel/CSV (+ legacy `.doc`/`.ppt` via LibreOffice)
|
|
358
|
+
* Cross-page image **deduplication** via xref tracking
|
|
359
|
+
* **Image filters**: solid-color / tiny-icon / dimension / reference-logo (perceptual hash)
|
|
360
|
+
* **Sliding-window** text chunking (~500 tokens, ~50 overlap) at sentence boundaries
|
|
361
|
+
* Tables serialized to **Markdown**; images embedded once and reused
|
|
362
|
+
* **Parallel** vision calls, **batched** embeddings
|
|
363
|
+
|
|
364
|
+
## Development
|
|
365
|
+
|
|
366
|
+
```bash
|
|
367
|
+
pip install -e ".[dev,pdf,docx,pptx,xlsx]"
|
|
368
|
+
pytest
|
|
369
|
+
ruff check src tests
|
|
370
|
+
```
|
|
371
|
+
|
|
372
|
+
## Troubleshooting
|
|
373
|
+
|
|
374
|
+
**LibreOffice not found / vector images skipped**
|
|
375
|
+
EMF, WMF, and SVG images embedded in PPTX/XLSX are converted via LibreOffice.
|
|
376
|
+
Install it system-wide (`apt install libreoffice` / `brew install libreoffice` /
|
|
377
|
+
[libreoffice.org](https://www.libreoffice.org/download/download/)) and ensure
|
|
378
|
+
`soffice` is on `PATH`. Without it, vector images are silently skipped; other
|
|
379
|
+
image types are unaffected.
|
|
380
|
+
|
|
381
|
+
**`.doc` / `.ppt` legacy files not extracted**
|
|
382
|
+
Legacy binary formats require LibreOffice for conversion to DOCX/PPTX before
|
|
383
|
+
extraction. The same `soffice` dependency applies.
|
|
384
|
+
|
|
385
|
+
**`transformers` / `torch` import errors or CUDA failures**
|
|
386
|
+
Local vision models (Qwen2.5-VL, Llama 3.2 Vision, SmolVLM) require a compatible
|
|
387
|
+
`torch` + CUDA environment. Confirm with:
|
|
388
|
+
```python
|
|
389
|
+
import torch; print(torch.cuda.is_available(), torch.version.cuda)
|
|
390
|
+
```
|
|
391
|
+
If CUDA is unavailable, SmolVLM (`[smolvlm]`) is the recommended model that runs
|
|
392
|
+
on CPU at practical speeds. Qwen2.5-VL and Llama 3.2 Vision require a GPU with ≥16 GB
|
|
393
|
+
VRAM in BF16; use `load_in_4bit=True` for smaller cards.
|
|
394
|
+
|
|
395
|
+
**`pip install multixtract[qwen2vl]` takes a long time**
|
|
396
|
+
`torch` is a large package (~2 GB). Pull a GPU-specific wheel with:
|
|
397
|
+
```bash
|
|
398
|
+
pip install "multixtract[qwen2vl]" --extra-index-url https://download.pytorch.org/whl/cu121
|
|
399
|
+
```
|
|
400
|
+
Replace `cu121` with your CUDA version (`cu118`, `cu124`, etc.).
|
|
401
|
+
|
|
402
|
+
**Azure `DefaultAzureCredential` fails locally**
|
|
403
|
+
`DefaultAzureCredential` tries several auth paths in order. For local dev the
|
|
404
|
+
easiest is `az login` (Azure CLI). For managed identity in production, ensure
|
|
405
|
+
the compute resource has an assigned identity and the necessary role on the
|
|
406
|
+
target resource.
|
|
407
|
+
|
|
408
|
+
|
|
409
|
+
**PyMuPDF / pdfplumber version conflicts**
|
|
410
|
+
If you see `ImportError` from `fitz`, ensure `PyMuPDF>=1.23` is installed.
|
|
411
|
+
`pdfplumber` and `PyMuPDF` can coexist; both are required for the `[pdf]` extra.
|
|
412
|
+
|
|
413
|
+
## Acknowledgements
|
|
414
|
+
|
|
415
|
+
Built with assistance from [Claude](https://claude.ai) (Anthropic) for code review, bug analysis, and quality improvements.
|
|
416
|
+
|
|
417
|
+
## License
|
|
418
|
+
|
|
419
|
+
MIT — see `LICENSE`.
|