mare-retrieval 0.2.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- mare_retrieval-0.2.0/LICENSE +22 -0
- mare_retrieval-0.2.0/MANIFEST.in +4 -0
- mare_retrieval-0.2.0/PKG-INFO +352 -0
- mare_retrieval-0.2.0/README.md +319 -0
- mare_retrieval-0.2.0/examples/sample_corpus.json +41 -0
- mare_retrieval-0.2.0/pyproject.toml +50 -0
- mare_retrieval-0.2.0/setup.cfg +4 -0
- mare_retrieval-0.2.0/setup.py +24 -0
- mare_retrieval-0.2.0/src/mare/__init__.py +21 -0
- mare_retrieval-0.2.0/src/mare/api.py +65 -0
- mare_retrieval-0.2.0/src/mare/ask.py +56 -0
- mare_retrieval-0.2.0/src/mare/demo.py +59 -0
- mare_retrieval-0.2.0/src/mare/engine.py +44 -0
- mare_retrieval-0.2.0/src/mare/fusion.py +61 -0
- mare_retrieval-0.2.0/src/mare/ingest.py +152 -0
- mare_retrieval-0.2.0/src/mare/retrievers/__init__.py +2 -0
- mare_retrieval-0.2.0/src/mare/retrievers/base.py +17 -0
- mare_retrieval-0.2.0/src/mare/retrievers/image.py +70 -0
- mare_retrieval-0.2.0/src/mare/retrievers/text.py +76 -0
- mare_retrieval-0.2.0/src/mare/router.py +85 -0
- mare_retrieval-0.2.0/src/mare/streamlit_app.py +100 -0
- mare_retrieval-0.2.0/src/mare/types.py +52 -0
- mare_retrieval-0.2.0/src/mare_retrieval.egg-info/PKG-INFO +352 -0
- mare_retrieval-0.2.0/src/mare_retrieval.egg-info/SOURCES.txt +30 -0
- mare_retrieval-0.2.0/src/mare_retrieval.egg-info/dependency_links.txt +1 -0
- mare_retrieval-0.2.0/src/mare_retrieval.egg-info/entry_points.txt +5 -0
- mare_retrieval-0.2.0/src/mare_retrieval.egg-info/requires.txt +13 -0
- mare_retrieval-0.2.0/src/mare_retrieval.egg-info/top_level.txt +1 -0
- mare_retrieval-0.2.0/tests/test_api.py +36 -0
- mare_retrieval-0.2.0/tests/test_ask.py +92 -0
- mare_retrieval-0.2.0/tests/test_engine.py +45 -0
- mare_retrieval-0.2.0/tests/test_ingest.py +41 -0
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
22
|
+
|
|
@@ -0,0 +1,352 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: mare-retrieval
|
|
3
|
+
Version: 0.2.0
|
|
4
|
+
Summary: Modality-Aware Retrieval Engine inspired by IRPAPERS-style multimodal retrieval.
|
|
5
|
+
Author: Saisandeep Kantareddy
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/SaiSandeepKantareddy/MARE
|
|
8
|
+
Project-URL: Repository, https://github.com/SaiSandeepKantareddy/MARE
|
|
9
|
+
Project-URL: Issues, https://github.com/SaiSandeepKantareddy/MARE/issues
|
|
10
|
+
Keywords: retrieval,rag,pdf,document-ai,multimodal
|
|
11
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
16
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
17
|
+
Classifier: Topic :: Text Processing :: Indexing
|
|
18
|
+
Requires-Python: >=3.9
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
License-File: LICENSE
|
|
21
|
+
Requires-Dist: pypdf>=4.0
|
|
22
|
+
Requires-Dist: pypdfium2>=4.30.0
|
|
23
|
+
Provides-Extra: dev
|
|
24
|
+
Requires-Dist: pytest>=8.0; extra == "dev"
|
|
25
|
+
Provides-Extra: publish
|
|
26
|
+
Requires-Dist: build>=1.2.0; extra == "publish"
|
|
27
|
+
Requires-Dist: twine>=5.0.0; extra == "publish"
|
|
28
|
+
Provides-Extra: ui
|
|
29
|
+
Requires-Dist: streamlit<2.0,>=1.12; extra == "ui"
|
|
30
|
+
Requires-Dist: altair<5; extra == "ui"
|
|
31
|
+
Dynamic: license-file
|
|
32
|
+
Dynamic: requires-python
|
|
33
|
+
|
|
34
|
+
# MARE
|
|
35
|
+
|
|
36
|
+
MARE is an open-source Python library for evidence-first document retrieval.
|
|
37
|
+
|
|
38
|
+
It is inspired by the direction highlighted in the IRPAPERS paper, which shows that page-image retrieval and text retrieval have complementary failure modes on scientific documents. Instead of flattening everything into one retrieval path, MARE treats routing, retrieval, fusion, and observability as separate system concerns.
|
|
39
|
+
|
|
40
|
+
## What this repo is
|
|
41
|
+
|
|
42
|
+
- A lightweight Python package between a query and modality-specific indexes
|
|
43
|
+
- A baseline router that decides whether a query should hit text, image, layout, or a hybrid path
|
|
44
|
+
- A late-fusion layer that combines modality-specific scores
|
|
45
|
+
- An explainable debug surface that tells you why a modality was selected
|
|
46
|
+
|
|
47
|
+
## What this repo is not
|
|
48
|
+
|
|
49
|
+
- Not a chatbot wrapper
|
|
50
|
+
- Not a full PDF parsing stack yet
|
|
51
|
+
- Not a claim that heuristic routing is state of the art
|
|
52
|
+
|
|
53
|
+
## Why now
|
|
54
|
+
|
|
55
|
+
IRPAPERS asks a useful systems question: when should we retrieve over OCR text, page images, layout structure, or some combination? The paper reports that text-based and image-based retrieval each solve queries the other misses, and that fusion improves retrieval quality over either modality alone.
|
|
56
|
+
|
|
57
|
+
This repo turns that observation into an MVP developer layer.
|
|
58
|
+
|
|
59
|
+
Paper: https://arxiv.org/pdf/2602.17687
|
|
60
|
+
|
|
61
|
+
## Architecture
|
|
62
|
+
|
|
63
|
+
```text
|
|
64
|
+
query
|
|
65
|
+
-> router
|
|
66
|
+
-> modality-specific retrievers
|
|
67
|
+
-> text index
|
|
68
|
+
-> image index
|
|
69
|
+
-> layout index
|
|
70
|
+
-> fusion
|
|
71
|
+
-> explainable results
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Current implementation choices:
|
|
75
|
+
|
|
76
|
+
- Router: keyword heuristic baseline
|
|
77
|
+
- Text retrieval: token-overlap cosine baseline
|
|
78
|
+
- Image retrieval: caption and visual-tag overlap baseline
|
|
79
|
+
- Layout retrieval: layout-hint overlap baseline
|
|
80
|
+
- Fusion: weighted late fusion
|
|
81
|
+
|
|
82
|
+
The point of `v0.1` is not raw benchmark quality. It is to package the control plane cleanly enough that stronger models can drop in later.
|
|
83
|
+
|
|
84
|
+
## Repo layout
|
|
85
|
+
|
|
86
|
+
```text
|
|
87
|
+
src/mare/
|
|
88
|
+
engine.py
|
|
89
|
+
router.py
|
|
90
|
+
fusion.py
|
|
91
|
+
types.py
|
|
92
|
+
retrievers/
|
|
93
|
+
examples/
|
|
94
|
+
tests/
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
## Quickstart
|
|
98
|
+
|
|
99
|
+
Clone and install:
|
|
100
|
+
|
|
101
|
+
```bash
|
|
102
|
+
git clone https://github.com/SaiSandeepKantareddy/MARE.git
|
|
103
|
+
cd MARE
|
|
104
|
+
python3 -m venv .venv
|
|
105
|
+
source .venv/bin/activate
|
|
106
|
+
pip install -e ".[dev]"
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
Or install directly from GitHub:
|
|
110
|
+
|
|
111
|
+
```bash
|
|
112
|
+
pip install "git+https://github.com/SaiSandeepKantareddy/MARE.git"
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
The intended package install after PyPI release is:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
pip install mare-retrieval
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Then use it as a library:
|
|
122
|
+
|
|
123
|
+
```python
|
|
124
|
+
from mare import MAREApp
|
|
125
|
+
|
|
126
|
+
app = MAREApp.from_pdf("manual.pdf", reuse=True)
|
|
127
|
+
best = app.best_match("partially reinstall the set screws if they fall out")
|
|
128
|
+
|
|
129
|
+
print(best.page)
|
|
130
|
+
print(best.snippet)
|
|
131
|
+
print(best.page_image_path)
|
|
132
|
+
```
|
|
133
|
+
|
|
134
|
+
Or try the sample corpus from the CLI:
|
|
135
|
+
|
|
136
|
+
```bash
|
|
137
|
+
mare-demo --query "show me the architecture diagram of transformer"
|
|
138
|
+
```
|
|
139
|
+
|
|
140
|
+
Or without installing the package yet:
|
|
141
|
+
|
|
142
|
+
```bash
|
|
143
|
+
PYTHONPATH=src python3 -m mare.demo --query "show me the architecture diagram of transformer"
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
## Simplest way to use it
|
|
147
|
+
|
|
148
|
+
Use one command:
|
|
149
|
+
|
|
150
|
+
```bash
|
|
151
|
+
python3 ask.py "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf" "partially reinstall the set screws if they fall out"
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
That will:
|
|
155
|
+
|
|
156
|
+
- ingest the PDF if needed
|
|
157
|
+
- retrieve the best matching page
|
|
158
|
+
- print the page number
|
|
159
|
+
- print the exact snippet
|
|
160
|
+
- print the rendered page image path
|
|
161
|
+
|
|
162
|
+
If you want to reuse a previously generated corpus:
|
|
163
|
+
|
|
164
|
+
```bash
|
|
165
|
+
python3 ask.py --reuse "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf" "partially reinstall the set screws if they fall out"
|
|
166
|
+
```
|
|
167
|
+
|
|
168
|
+
If the PDF filename is awkward, rename it first:
|
|
169
|
+
|
|
170
|
+
```bash
|
|
171
|
+
mv ./*.pdf ./manual.pdf
|
|
172
|
+
PYTHONPATH=src python3 ask.py ./manual.pdf "partially reinstall the set screws if they fall out"
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
## Public Python API
|
|
176
|
+
|
|
177
|
+
The package is meant to be importable, not just runnable from scripts.
|
|
178
|
+
|
|
179
|
+
```python
|
|
180
|
+
from mare import MAREApp, load_corpus, load_pdf
|
|
181
|
+
```
|
|
182
|
+
|
|
183
|
+
Create an app from a PDF:
|
|
184
|
+
|
|
185
|
+
```python
|
|
186
|
+
app = load_pdf("manual.pdf", reuse=True)
|
|
187
|
+
hit = app.best_match("what does MagSafe 3 refer to")
|
|
188
|
+
```
|
|
189
|
+
|
|
190
|
+
Create an app from an existing JSON corpus:
|
|
191
|
+
|
|
192
|
+
```python
|
|
193
|
+
app = load_corpus("generated/manual.json")
|
|
194
|
+
results = app.retrieve("show me the comparison table", top_k=3)
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
Core methods:
|
|
198
|
+
|
|
199
|
+
- `MAREApp.from_pdf(...)`
|
|
200
|
+
- `MAREApp.from_corpus(...)`
|
|
201
|
+
- `MAREApp.from_documents(...)`
|
|
202
|
+
- `app.explain(query)`
|
|
203
|
+
- `app.retrieve(query)`
|
|
204
|
+
- `app.best_match(query)`
|
|
205
|
+
|
|
206
|
+
## Packaging and release
|
|
207
|
+
|
|
208
|
+
MARE is now structured as a regular Python package with:
|
|
209
|
+
|
|
210
|
+
- `pyproject.toml` metadata
|
|
211
|
+
- legacy-friendly `setup.py`
|
|
212
|
+
- console entry points
|
|
213
|
+
- a PyPI publishing workflow
|
|
214
|
+
|
|
215
|
+
Release notes and PyPI steps live in `PUBLISHING.md`.
|
|
216
|
+
|
|
217
|
+
## Visual demo
|
|
218
|
+
|
|
219
|
+
If you want to show this to users visually, run the Streamlit demo:
|
|
220
|
+
|
|
221
|
+
```bash
|
|
222
|
+
pip install -e ".[ui]"
|
|
223
|
+
PYTHONPATH=src streamlit run src/mare/streamlit_app.py
|
|
224
|
+
```
|
|
225
|
+
|
|
226
|
+
The demo lets a user:
|
|
227
|
+
|
|
228
|
+
- upload a PDF
|
|
229
|
+
- ask a question
|
|
230
|
+
- see the best matching page
|
|
231
|
+
- read the exact evidence snippet
|
|
232
|
+
- view the rendered page image
|
|
233
|
+
|
|
234
|
+
The technical retrieval plan is hidden under a `Debug details` expander so the default experience stays user-facing.
|
|
235
|
+
|
|
236
|
+
## Ingest a real PDF
|
|
237
|
+
|
|
238
|
+
You can convert a PDF into a page-level JSON corpus and then run retrieval on it.
|
|
239
|
+
|
|
240
|
+
```bash
|
|
241
|
+
python3 -m venv .venv
|
|
242
|
+
source .venv/bin/activate
|
|
243
|
+
pip install -e ".[dev]"
|
|
244
|
+
mare-ingest "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf"
|
|
245
|
+
mare-demo --corpus "generated/MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.json" --query "what does MagSafe 3 refer to"
|
|
246
|
+
```
|
|
247
|
+
|
|
248
|
+
Without installing the package first:
|
|
249
|
+
|
|
250
|
+
```bash
|
|
251
|
+
PYTHONPATH=src python3 -m mare.ingest "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf"
|
|
252
|
+
PYTHONPATH=src python3 -m mare.demo --corpus "generated/MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.json" --query "what does MagSafe 3 refer to"
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
What the ingest step does right now:
|
|
256
|
+
|
|
257
|
+
- reads each PDF page with `pypdf`
|
|
258
|
+
- renders each PDF page to `generated/<pdf-name>/page-N.png`
|
|
259
|
+
- extracts page text
|
|
260
|
+
- creates one retrieval document per page
|
|
261
|
+
- adds lightweight layout hints when terms like `Table` or `Figure` appear
|
|
262
|
+
- writes a JSON corpus that the retriever can search immediately
|
|
263
|
+
|
|
264
|
+
This is still a simple baseline. OCR, figure extraction, and true layout modeling are the next step.
|
|
265
|
+
|
|
266
|
+
## What you get back
|
|
267
|
+
|
|
268
|
+
The retriever now returns:
|
|
269
|
+
|
|
270
|
+
- the matching page number
|
|
271
|
+
- why that page matched
|
|
272
|
+
- a short exact snippet from the page text
|
|
273
|
+
- the rendered page image path
|
|
274
|
+
|
|
275
|
+
That makes it easier to validate whether retrieval found the right instruction and jump to the exact page image.
|
|
276
|
+
|
|
277
|
+
Example output:
|
|
278
|
+
|
|
279
|
+
```json
|
|
280
|
+
{
|
|
281
|
+
"query": "show me the architecture diagram of transformer",
|
|
282
|
+
"intent": "visual_lookup",
|
|
283
|
+
"selected_modalities": ["image"],
|
|
284
|
+
"discarded_modalities": ["text", "layout"],
|
|
285
|
+
"confidence": 0.8,
|
|
286
|
+
"rationale": "Detected modality cues in query tokens. Selected image based on keyword overlap with routing hints.",
|
|
287
|
+
"results": [
|
|
288
|
+
{
|
|
289
|
+
"doc_id": "paper-transformer-p4",
|
|
290
|
+
"title": "Attention Is All You Need",
|
|
291
|
+
"page": 4,
|
|
292
|
+
"score": 0.6,
|
|
293
|
+
"reason": "image:Matched visual cues: architecture, diagram, transformer"
|
|
294
|
+
}
|
|
295
|
+
]
|
|
296
|
+
}
|
|
297
|
+
```
|
|
298
|
+
|
|
299
|
+
## Why the explainability matters
|
|
300
|
+
|
|
301
|
+
The debug surface is a core feature, not an afterthought. For production retrieval systems, we need to answer:
|
|
302
|
+
|
|
303
|
+
- Which modality did the router choose?
|
|
304
|
+
- Which modalities were skipped?
|
|
305
|
+
- Why did a page rank highly?
|
|
306
|
+
- What tradeoff did fusion make?
|
|
307
|
+
|
|
308
|
+
That is the wedge for MARE: make multimodal retrieval inspectable before trying to make it magical.
|
|
309
|
+
|
|
310
|
+
## Local sample data
|
|
311
|
+
|
|
312
|
+
`examples/sample_corpus.json` contains a tiny IR-paper-style corpus so the routing and fusion path is runnable out of the box.
|
|
313
|
+
|
|
314
|
+
There is also a local PDF in this workspace:
|
|
315
|
+
|
|
316
|
+
- `MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf`
|
|
317
|
+
|
|
318
|
+
That file can now be ingested into a JSON page corpus with `mare-ingest`.
|
|
319
|
+
|
|
320
|
+
## Roadmap
|
|
321
|
+
|
|
322
|
+
### v0.1
|
|
323
|
+
|
|
324
|
+
- text + image + layout routing
|
|
325
|
+
- weighted late fusion
|
|
326
|
+
- explainable retrieval output
|
|
327
|
+
- tests and runnable demo
|
|
328
|
+
|
|
329
|
+
### v0.2
|
|
330
|
+
|
|
331
|
+
- pluggable embedding backends
|
|
332
|
+
- PDF page ingestion
|
|
333
|
+
- OCR and caption extraction adapters
|
|
334
|
+
- score normalization per modality
|
|
335
|
+
|
|
336
|
+
### v0.3
|
|
337
|
+
|
|
338
|
+
- learned router
|
|
339
|
+
- benchmark harness for IRPAPERS-style evaluation
|
|
340
|
+
- cost-aware routing budgets
|
|
341
|
+
- reranking and cross-modal evidence aggregation
|
|
342
|
+
|
|
343
|
+
## Suggested next open-source moves
|
|
344
|
+
|
|
345
|
+
- Add adapters for FAISS, Qdrant, and Weaviate
|
|
346
|
+
- Add page extraction from PDFs
|
|
347
|
+
- Add a benchmark runner that computes Recall@k per modality
|
|
348
|
+
- Add a small web debug UI for route inspection
|
|
349
|
+
|
|
350
|
+
## License
|
|
351
|
+
|
|
352
|
+
MIT
|
|
@@ -0,0 +1,319 @@
|
|
|
1
|
+
# MARE
|
|
2
|
+
|
|
3
|
+
MARE is an open-source Python library for evidence-first document retrieval.
|
|
4
|
+
|
|
5
|
+
It is inspired by the direction highlighted in the IRPAPERS paper, which shows that page-image retrieval and text retrieval have complementary failure modes on scientific documents. Instead of flattening everything into one retrieval path, MARE treats routing, retrieval, fusion, and observability as separate system concerns.
|
|
6
|
+
|
|
7
|
+
## What this repo is
|
|
8
|
+
|
|
9
|
+
- A lightweight Python package between a query and modality-specific indexes
|
|
10
|
+
- A baseline router that decides whether a query should hit text, image, layout, or a hybrid path
|
|
11
|
+
- A late-fusion layer that combines modality-specific scores
|
|
12
|
+
- An explainable debug surface that tells you why a modality was selected
|
|
13
|
+
|
|
14
|
+
## What this repo is not
|
|
15
|
+
|
|
16
|
+
- Not a chatbot wrapper
|
|
17
|
+
- Not a full PDF parsing stack yet
|
|
18
|
+
- Not a claim that heuristic routing is state of the art
|
|
19
|
+
|
|
20
|
+
## Why now
|
|
21
|
+
|
|
22
|
+
IRPAPERS asks a useful systems question: when should we retrieve over OCR text, page images, layout structure, or some combination? The paper reports that text-based and image-based retrieval each solve queries the other misses, and that fusion improves retrieval quality over either modality alone.
|
|
23
|
+
|
|
24
|
+
This repo turns that observation into an MVP developer layer.
|
|
25
|
+
|
|
26
|
+
Paper: https://arxiv.org/pdf/2602.17687
|
|
27
|
+
|
|
28
|
+
## Architecture
|
|
29
|
+
|
|
30
|
+
```text
|
|
31
|
+
query
|
|
32
|
+
-> router
|
|
33
|
+
-> modality-specific retrievers
|
|
34
|
+
-> text index
|
|
35
|
+
-> image index
|
|
36
|
+
-> layout index
|
|
37
|
+
-> fusion
|
|
38
|
+
-> explainable results
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Current implementation choices:
|
|
42
|
+
|
|
43
|
+
- Router: keyword heuristic baseline
|
|
44
|
+
- Text retrieval: token-overlap cosine baseline
|
|
45
|
+
- Image retrieval: caption and visual-tag overlap baseline
|
|
46
|
+
- Layout retrieval: layout-hint overlap baseline
|
|
47
|
+
- Fusion: weighted late fusion
|
|
48
|
+
|
|
49
|
+
The point of `v0.1` is not raw benchmark quality. It is to package the control plane cleanly enough that stronger models can drop in later.
|
|
50
|
+
|
|
51
|
+
## Repo layout
|
|
52
|
+
|
|
53
|
+
```text
|
|
54
|
+
src/mare/
|
|
55
|
+
engine.py
|
|
56
|
+
router.py
|
|
57
|
+
fusion.py
|
|
58
|
+
types.py
|
|
59
|
+
retrievers/
|
|
60
|
+
examples/
|
|
61
|
+
tests/
|
|
62
|
+
```
|
|
63
|
+
|
|
64
|
+
## Quickstart
|
|
65
|
+
|
|
66
|
+
Clone and install:
|
|
67
|
+
|
|
68
|
+
```bash
|
|
69
|
+
git clone https://github.com/SaiSandeepKantareddy/MARE.git
|
|
70
|
+
cd MARE
|
|
71
|
+
python3 -m venv .venv
|
|
72
|
+
source .venv/bin/activate
|
|
73
|
+
pip install -e ".[dev]"
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
Or install directly from GitHub:
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
pip install "git+https://github.com/SaiSandeepKantareddy/MARE.git"
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
The intended package install after PyPI release is:
|
|
83
|
+
|
|
84
|
+
```bash
|
|
85
|
+
pip install mare-retrieval
|
|
86
|
+
```
|
|
87
|
+
|
|
88
|
+
Then use it as a library:
|
|
89
|
+
|
|
90
|
+
```python
|
|
91
|
+
from mare import MAREApp
|
|
92
|
+
|
|
93
|
+
app = MAREApp.from_pdf("manual.pdf", reuse=True)
|
|
94
|
+
best = app.best_match("partially reinstall the set screws if they fall out")
|
|
95
|
+
|
|
96
|
+
print(best.page)
|
|
97
|
+
print(best.snippet)
|
|
98
|
+
print(best.page_image_path)
|
|
99
|
+
```
|
|
100
|
+
|
|
101
|
+
Or try the sample corpus from the CLI:
|
|
102
|
+
|
|
103
|
+
```bash
|
|
104
|
+
mare-demo --query "show me the architecture diagram of transformer"
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
Or without installing the package yet:
|
|
108
|
+
|
|
109
|
+
```bash
|
|
110
|
+
PYTHONPATH=src python3 -m mare.demo --query "show me the architecture diagram of transformer"
|
|
111
|
+
```
|
|
112
|
+
|
|
113
|
+
## Simplest way to use it
|
|
114
|
+
|
|
115
|
+
Use one command:
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
python3 ask.py "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf" "partially reinstall the set screws if they fall out"
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
That will:
|
|
122
|
+
|
|
123
|
+
- ingest the PDF if needed
|
|
124
|
+
- retrieve the best matching page
|
|
125
|
+
- print the page number
|
|
126
|
+
- print the exact snippet
|
|
127
|
+
- print the rendered page image path
|
|
128
|
+
|
|
129
|
+
If you want to reuse a previously generated corpus:
|
|
130
|
+
|
|
131
|
+
```bash
|
|
132
|
+
python3 ask.py --reuse "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf" "partially reinstall the set screws if they fall out"
|
|
133
|
+
```
|
|
134
|
+
|
|
135
|
+
If the PDF filename is awkward, rename it first:
|
|
136
|
+
|
|
137
|
+
```bash
|
|
138
|
+
mv ./*.pdf ./manual.pdf
|
|
139
|
+
PYTHONPATH=src python3 ask.py ./manual.pdf "partially reinstall the set screws if they fall out"
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
## Public Python API
|
|
143
|
+
|
|
144
|
+
The package is meant to be importable, not just runnable from scripts.
|
|
145
|
+
|
|
146
|
+
```python
|
|
147
|
+
from mare import MAREApp, load_corpus, load_pdf
|
|
148
|
+
```
|
|
149
|
+
|
|
150
|
+
Create an app from a PDF:
|
|
151
|
+
|
|
152
|
+
```python
|
|
153
|
+
app = load_pdf("manual.pdf", reuse=True)
|
|
154
|
+
hit = app.best_match("what does MagSafe 3 refer to")
|
|
155
|
+
```
|
|
156
|
+
|
|
157
|
+
Create an app from an existing JSON corpus:
|
|
158
|
+
|
|
159
|
+
```python
|
|
160
|
+
app = load_corpus("generated/manual.json")
|
|
161
|
+
results = app.retrieve("show me the comparison table", top_k=3)
|
|
162
|
+
```
|
|
163
|
+
|
|
164
|
+
Core methods:
|
|
165
|
+
|
|
166
|
+
- `MAREApp.from_pdf(...)`
|
|
167
|
+
- `MAREApp.from_corpus(...)`
|
|
168
|
+
- `MAREApp.from_documents(...)`
|
|
169
|
+
- `app.explain(query)`
|
|
170
|
+
- `app.retrieve(query)`
|
|
171
|
+
- `app.best_match(query)`
|
|
172
|
+
|
|
173
|
+
## Packaging and release
|
|
174
|
+
|
|
175
|
+
MARE is now structured as a regular Python package with:
|
|
176
|
+
|
|
177
|
+
- `pyproject.toml` metadata
|
|
178
|
+
- legacy-friendly `setup.py`
|
|
179
|
+
- console entry points
|
|
180
|
+
- a PyPI publishing workflow
|
|
181
|
+
|
|
182
|
+
Release notes and PyPI steps live in `PUBLISHING.md`.
|
|
183
|
+
|
|
184
|
+
## Visual demo
|
|
185
|
+
|
|
186
|
+
If you want to show this to users visually, run the Streamlit demo:
|
|
187
|
+
|
|
188
|
+
```bash
|
|
189
|
+
pip install -e ".[ui]"
|
|
190
|
+
PYTHONPATH=src streamlit run src/mare/streamlit_app.py
|
|
191
|
+
```
|
|
192
|
+
|
|
193
|
+
The demo lets a user:
|
|
194
|
+
|
|
195
|
+
- upload a PDF
|
|
196
|
+
- ask a question
|
|
197
|
+
- see the best matching page
|
|
198
|
+
- read the exact evidence snippet
|
|
199
|
+
- view the rendered page image
|
|
200
|
+
|
|
201
|
+
The technical retrieval plan is hidden under a `Debug details` expander so the default experience stays user-facing.
|
|
202
|
+
|
|
203
|
+
## Ingest a real PDF
|
|
204
|
+
|
|
205
|
+
You can convert a PDF into a page-level JSON corpus and then run retrieval on it.
|
|
206
|
+
|
|
207
|
+
```bash
|
|
208
|
+
python3 -m venv .venv
|
|
209
|
+
source .venv/bin/activate
|
|
210
|
+
pip install -e ".[dev]"
|
|
211
|
+
mare-ingest "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf"
|
|
212
|
+
mare-demo --corpus "generated/MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.json" --query "what does MagSafe 3 refer to"
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
Without installing the package first:
|
|
216
|
+
|
|
217
|
+
```bash
|
|
218
|
+
PYTHONPATH=src python3 -m mare.ingest "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf"
|
|
219
|
+
PYTHONPATH=src python3 -m mare.demo --corpus "generated/MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.json" --query "what does MagSafe 3 refer to"
|
|
220
|
+
```
|
|
221
|
+
|
|
222
|
+
What the ingest step does right now:
|
|
223
|
+
|
|
224
|
+
- reads each PDF page with `pypdf`
|
|
225
|
+
- renders each PDF page to `generated/<pdf-name>/page-N.png`
|
|
226
|
+
- extracts page text
|
|
227
|
+
- creates one retrieval document per page
|
|
228
|
+
- adds lightweight layout hints when terms like `Table` or `Figure` appear
|
|
229
|
+
- writes a JSON corpus that the retriever can search immediately
|
|
230
|
+
|
|
231
|
+
This is still a simple baseline. OCR, figure extraction, and true layout modeling are the next step.
|
|
232
|
+
|
|
233
|
+
## What you get back
|
|
234
|
+
|
|
235
|
+
The retriever now returns:
|
|
236
|
+
|
|
237
|
+
- the matching page number
|
|
238
|
+
- why that page matched
|
|
239
|
+
- a short exact snippet from the page text
|
|
240
|
+
- the rendered page image path
|
|
241
|
+
|
|
242
|
+
That makes it easier to validate whether retrieval found the right instruction and jump to the exact page image.
|
|
243
|
+
|
|
244
|
+
Example output:
|
|
245
|
+
|
|
246
|
+
```json
|
|
247
|
+
{
|
|
248
|
+
"query": "show me the architecture diagram of transformer",
|
|
249
|
+
"intent": "visual_lookup",
|
|
250
|
+
"selected_modalities": ["image"],
|
|
251
|
+
"discarded_modalities": ["text", "layout"],
|
|
252
|
+
"confidence": 0.8,
|
|
253
|
+
"rationale": "Detected modality cues in query tokens. Selected image based on keyword overlap with routing hints.",
|
|
254
|
+
"results": [
|
|
255
|
+
{
|
|
256
|
+
"doc_id": "paper-transformer-p4",
|
|
257
|
+
"title": "Attention Is All You Need",
|
|
258
|
+
"page": 4,
|
|
259
|
+
"score": 0.6,
|
|
260
|
+
"reason": "image:Matched visual cues: architecture, diagram, transformer"
|
|
261
|
+
}
|
|
262
|
+
]
|
|
263
|
+
}
|
|
264
|
+
```
|
|
265
|
+
|
|
266
|
+
## Why the explainability matters
|
|
267
|
+
|
|
268
|
+
The debug surface is a core feature, not an afterthought. For production retrieval systems, we need to answer:
|
|
269
|
+
|
|
270
|
+
- Which modality did the router choose?
|
|
271
|
+
- Which modalities were skipped?
|
|
272
|
+
- Why did a page rank highly?
|
|
273
|
+
- What tradeoff did fusion make?
|
|
274
|
+
|
|
275
|
+
That is the wedge for MARE: make multimodal retrieval inspectable before trying to make it magical.
|
|
276
|
+
|
|
277
|
+
## Local sample data
|
|
278
|
+
|
|
279
|
+
`examples/sample_corpus.json` contains a tiny IR-paper-style corpus so the routing and fusion path is runnable out of the box.
|
|
280
|
+
|
|
281
|
+
There is also a local PDF in this workspace:
|
|
282
|
+
|
|
283
|
+
- `MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf`
|
|
284
|
+
|
|
285
|
+
That file can now be ingested into a JSON page corpus with `mare-ingest`.
|
|
286
|
+
|
|
287
|
+
## Roadmap
|
|
288
|
+
|
|
289
|
+
### v0.1
|
|
290
|
+
|
|
291
|
+
- text + image + layout routing
|
|
292
|
+
- weighted late fusion
|
|
293
|
+
- explainable retrieval output
|
|
294
|
+
- tests and runnable demo
|
|
295
|
+
|
|
296
|
+
### v0.2
|
|
297
|
+
|
|
298
|
+
- pluggable embedding backends
|
|
299
|
+
- PDF page ingestion
|
|
300
|
+
- OCR and caption extraction adapters
|
|
301
|
+
- score normalization per modality
|
|
302
|
+
|
|
303
|
+
### v0.3
|
|
304
|
+
|
|
305
|
+
- learned router
|
|
306
|
+
- benchmark harness for IRPAPERS-style evaluation
|
|
307
|
+
- cost-aware routing budgets
|
|
308
|
+
- reranking and cross-modal evidence aggregation
|
|
309
|
+
|
|
310
|
+
## Suggested next open-source moves
|
|
311
|
+
|
|
312
|
+
- Add adapters for FAISS, Qdrant, and Weaviate
|
|
313
|
+
- Add page extraction from PDFs
|
|
314
|
+
- Add a benchmark runner that computes Recall@k per modality
|
|
315
|
+
- Add a small web debug UI for route inspection
|
|
316
|
+
|
|
317
|
+
## License
|
|
318
|
+
|
|
319
|
+
MIT
|