mare-retrieval 0.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (32) hide show
  1. mare_retrieval-0.2.0/LICENSE +22 -0
  2. mare_retrieval-0.2.0/MANIFEST.in +4 -0
  3. mare_retrieval-0.2.0/PKG-INFO +352 -0
  4. mare_retrieval-0.2.0/README.md +319 -0
  5. mare_retrieval-0.2.0/examples/sample_corpus.json +41 -0
  6. mare_retrieval-0.2.0/pyproject.toml +50 -0
  7. mare_retrieval-0.2.0/setup.cfg +4 -0
  8. mare_retrieval-0.2.0/setup.py +24 -0
  9. mare_retrieval-0.2.0/src/mare/__init__.py +21 -0
  10. mare_retrieval-0.2.0/src/mare/api.py +65 -0
  11. mare_retrieval-0.2.0/src/mare/ask.py +56 -0
  12. mare_retrieval-0.2.0/src/mare/demo.py +59 -0
  13. mare_retrieval-0.2.0/src/mare/engine.py +44 -0
  14. mare_retrieval-0.2.0/src/mare/fusion.py +61 -0
  15. mare_retrieval-0.2.0/src/mare/ingest.py +152 -0
  16. mare_retrieval-0.2.0/src/mare/retrievers/__init__.py +2 -0
  17. mare_retrieval-0.2.0/src/mare/retrievers/base.py +17 -0
  18. mare_retrieval-0.2.0/src/mare/retrievers/image.py +70 -0
  19. mare_retrieval-0.2.0/src/mare/retrievers/text.py +76 -0
  20. mare_retrieval-0.2.0/src/mare/router.py +85 -0
  21. mare_retrieval-0.2.0/src/mare/streamlit_app.py +100 -0
  22. mare_retrieval-0.2.0/src/mare/types.py +52 -0
  23. mare_retrieval-0.2.0/src/mare_retrieval.egg-info/PKG-INFO +352 -0
  24. mare_retrieval-0.2.0/src/mare_retrieval.egg-info/SOURCES.txt +30 -0
  25. mare_retrieval-0.2.0/src/mare_retrieval.egg-info/dependency_links.txt +1 -0
  26. mare_retrieval-0.2.0/src/mare_retrieval.egg-info/entry_points.txt +5 -0
  27. mare_retrieval-0.2.0/src/mare_retrieval.egg-info/requires.txt +13 -0
  28. mare_retrieval-0.2.0/src/mare_retrieval.egg-info/top_level.txt +1 -0
  29. mare_retrieval-0.2.0/tests/test_api.py +36 -0
  30. mare_retrieval-0.2.0/tests/test_ask.py +92 -0
  31. mare_retrieval-0.2.0/tests/test_engine.py +45 -0
  32. mare_retrieval-0.2.0/tests/test_ingest.py +41 -0
@@ -0,0 +1,22 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
22
+
@@ -0,0 +1,4 @@
1
+ include README.md
2
+ include LICENSE
3
+ recursive-include examples *.json
4
+
@@ -0,0 +1,352 @@
1
+ Metadata-Version: 2.4
2
+ Name: mare-retrieval
3
+ Version: 0.2.0
4
+ Summary: Modality-Aware Retrieval Engine inspired by IRPAPERS-style multimodal retrieval.
5
+ Author: Saisandeep Kantareddy
6
+ License: MIT
7
+ Project-URL: Homepage, https://github.com/SaiSandeepKantareddy/MARE
8
+ Project-URL: Repository, https://github.com/SaiSandeepKantareddy/MARE
9
+ Project-URL: Issues, https://github.com/SaiSandeepKantareddy/MARE/issues
10
+ Keywords: retrieval,rag,pdf,document-ai,multimodal
11
+ Classifier: License :: OSI Approved :: MIT License
12
+ Classifier: Programming Language :: Python :: 3
13
+ Classifier: Programming Language :: Python :: 3.9
14
+ Classifier: Programming Language :: Python :: 3.10
15
+ Classifier: Programming Language :: Python :: 3.11
16
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
17
+ Classifier: Topic :: Text Processing :: Indexing
18
+ Requires-Python: >=3.9
19
+ Description-Content-Type: text/markdown
20
+ License-File: LICENSE
21
+ Requires-Dist: pypdf>=4.0
22
+ Requires-Dist: pypdfium2>=4.30.0
23
+ Provides-Extra: dev
24
+ Requires-Dist: pytest>=8.0; extra == "dev"
25
+ Provides-Extra: publish
26
+ Requires-Dist: build>=1.2.0; extra == "publish"
27
+ Requires-Dist: twine>=5.0.0; extra == "publish"
28
+ Provides-Extra: ui
29
+ Requires-Dist: streamlit<2.0,>=1.12; extra == "ui"
30
+ Requires-Dist: altair<5; extra == "ui"
31
+ Dynamic: license-file
32
+ Dynamic: requires-python
33
+
34
+ # MARE
35
+
36
+ MARE is an open-source Python library for evidence-first document retrieval.
37
+
38
+ It is inspired by the direction highlighted in the IRPAPERS paper, which shows that page-image retrieval and text retrieval have complementary failure modes on scientific documents. Instead of flattening everything into one retrieval path, MARE treats routing, retrieval, fusion, and observability as separate system concerns.
39
+
40
+ ## What this repo is
41
+
42
+ - A lightweight Python package between a query and modality-specific indexes
43
+ - A baseline router that decides whether a query should hit text, image, layout, or a hybrid path
44
+ - A late-fusion layer that combines modality-specific scores
45
+ - An explainable debug surface that tells you why a modality was selected
46
+
47
+ ## What this repo is not
48
+
49
+ - Not a chatbot wrapper
50
+ - Not a full PDF parsing stack yet
51
+ - Not a claim that heuristic routing is state of the art
52
+
53
+ ## Why now
54
+
55
+ IRPAPERS asks a useful systems question: when should we retrieve over OCR text, page images, layout structure, or some combination? The paper reports that text-based and image-based retrieval each solve queries the other misses, and that fusion improves retrieval quality over either modality alone.
56
+
57
+ This repo turns that observation into an MVP developer layer.
58
+
59
+ Paper: https://arxiv.org/pdf/2602.17687
60
+
61
+ ## Architecture
62
+
63
+ ```text
64
+ query
65
+ -> router
66
+ -> modality-specific retrievers
67
+ -> text index
68
+ -> image index
69
+ -> layout index
70
+ -> fusion
71
+ -> explainable results
72
+ ```
73
+
74
+ Current implementation choices:
75
+
76
+ - Router: keyword heuristic baseline
77
+ - Text retrieval: token-overlap cosine baseline
78
+ - Image retrieval: caption and visual-tag overlap baseline
79
+ - Layout retrieval: layout-hint overlap baseline
80
+ - Fusion: weighted late fusion
81
+
82
+ The point of `v0.1` is not raw benchmark quality. It is to package the control plane cleanly enough that stronger models can drop in later.
83
+
84
+ ## Repo layout
85
+
86
+ ```text
87
+ src/mare/
88
+ engine.py
89
+ router.py
90
+ fusion.py
91
+ types.py
92
+ retrievers/
93
+ examples/
94
+ tests/
95
+ ```
96
+
97
+ ## Quickstart
98
+
99
+ Clone and install:
100
+
101
+ ```bash
102
+ git clone https://github.com/SaiSandeepKantareddy/MARE.git
103
+ cd MARE
104
+ python3 -m venv .venv
105
+ source .venv/bin/activate
106
+ pip install -e ".[dev]"
107
+ ```
108
+
109
+ Or install directly from GitHub:
110
+
111
+ ```bash
112
+ pip install "git+https://github.com/SaiSandeepKantareddy/MARE.git"
113
+ ```
114
+
115
+ The intended package install after PyPI release is:
116
+
117
+ ```bash
118
+ pip install mare-retrieval
119
+ ```
120
+
121
+ Then use it as a library:
122
+
123
+ ```python
124
+ from mare import MAREApp
125
+
126
+ app = MAREApp.from_pdf("manual.pdf", reuse=True)
127
+ best = app.best_match("partially reinstall the set screws if they fall out")
128
+
129
+ print(best.page)
130
+ print(best.snippet)
131
+ print(best.page_image_path)
132
+ ```
133
+
134
+ Or try the sample corpus from the CLI:
135
+
136
+ ```bash
137
+ mare-demo --query "show me the architecture diagram of transformer"
138
+ ```
139
+
140
+ Or without installing the package yet:
141
+
142
+ ```bash
143
+ PYTHONPATH=src python3 -m mare.demo --query "show me the architecture diagram of transformer"
144
+ ```
145
+
146
+ ## Simplest way to use it
147
+
148
+ Use one command:
149
+
150
+ ```bash
151
+ python3 ask.py "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf" "partially reinstall the set screws if they fall out"
152
+ ```
153
+
154
+ That will:
155
+
156
+ - ingest the PDF if needed
157
+ - retrieve the best matching page
158
+ - print the page number
159
+ - print the exact snippet
160
+ - print the rendered page image path
161
+
162
+ If you want to reuse a previously generated corpus:
163
+
164
+ ```bash
165
+ python3 ask.py --reuse "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf" "partially reinstall the set screws if they fall out"
166
+ ```
167
+
168
+ If the PDF filename is awkward, rename it first:
169
+
170
+ ```bash
171
+ mv ./*.pdf ./manual.pdf
172
+ PYTHONPATH=src python3 ask.py ./manual.pdf "partially reinstall the set screws if they fall out"
173
+ ```
174
+
175
+ ## Public Python API
176
+
177
+ The package is meant to be importable, not just runnable from scripts.
178
+
179
+ ```python
180
+ from mare import MAREApp, load_corpus, load_pdf
181
+ ```
182
+
183
+ Create an app from a PDF:
184
+
185
+ ```python
186
+ app = load_pdf("manual.pdf", reuse=True)
187
+ hit = app.best_match("what does MagSafe 3 refer to")
188
+ ```
189
+
190
+ Create an app from an existing JSON corpus:
191
+
192
+ ```python
193
+ app = load_corpus("generated/manual.json")
194
+ results = app.retrieve("show me the comparison table", top_k=3)
195
+ ```
196
+
197
+ Core methods:
198
+
199
+ - `MAREApp.from_pdf(...)`
200
+ - `MAREApp.from_corpus(...)`
201
+ - `MAREApp.from_documents(...)`
202
+ - `app.explain(query)`
203
+ - `app.retrieve(query)`
204
+ - `app.best_match(query)`
205
+
206
+ ## Packaging and release
207
+
208
+ MARE is now structured as a regular Python package with:
209
+
210
+ - `pyproject.toml` metadata
211
+ - legacy-friendly `setup.py`
212
+ - console entry points
213
+ - a PyPI publishing workflow
214
+
215
+ Release notes and PyPI steps live in `PUBLISHING.md`.
216
+
217
+ ## Visual demo
218
+
219
+ If you want to show this to users visually, run the Streamlit demo:
220
+
221
+ ```bash
222
+ pip install -e ".[ui]"
223
+ PYTHONPATH=src streamlit run src/mare/streamlit_app.py
224
+ ```
225
+
226
+ The demo lets a user:
227
+
228
+ - upload a PDF
229
+ - ask a question
230
+ - see the best matching page
231
+ - read the exact evidence snippet
232
+ - view the rendered page image
233
+
234
+ The technical retrieval plan is hidden under a `Debug details` expander so the default experience stays user-facing.
235
+
236
+ ## Ingest a real PDF
237
+
238
+ You can convert a PDF into a page-level JSON corpus and then run retrieval on it.
239
+
240
+ ```bash
241
+ python3 -m venv .venv
242
+ source .venv/bin/activate
243
+ pip install -e ".[dev]"
244
+ mare-ingest "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf"
245
+ mare-demo --corpus "generated/MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.json" --query "what does MagSafe 3 refer to"
246
+ ```
247
+
248
+ Without installing the package first:
249
+
250
+ ```bash
251
+ PYTHONPATH=src python3 -m mare.ingest "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf"
252
+ PYTHONPATH=src python3 -m mare.demo --corpus "generated/MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.json" --query "what does MagSafe 3 refer to"
253
+ ```
254
+
255
+ What the ingest step does right now:
256
+
257
+ - reads each PDF page with `pypdf`
258
+ - renders each PDF page to `generated/<pdf-name>/page-N.png`
259
+ - extracts page text
260
+ - creates one retrieval document per page
261
+ - adds lightweight layout hints when terms like `Table` or `Figure` appear
262
+ - writes a JSON corpus that the retriever can search immediately
263
+
264
+ This is still a simple baseline. OCR, figure extraction, and true layout modeling are the next step.
265
+
266
+ ## What you get back
267
+
268
+ The retriever now returns:
269
+
270
+ - the matching page number
271
+ - why that page matched
272
+ - a short exact snippet from the page text
273
+ - the rendered page image path
274
+
275
+ That makes it easier to validate whether retrieval found the right instruction and jump to the exact page image.
276
+
277
+ Example output:
278
+
279
+ ```json
280
+ {
281
+ "query": "show me the architecture diagram of transformer",
282
+ "intent": "visual_lookup",
283
+ "selected_modalities": ["image"],
284
+ "discarded_modalities": ["text", "layout"],
285
+ "confidence": 0.8,
286
+ "rationale": "Detected modality cues in query tokens. Selected image based on keyword overlap with routing hints.",
287
+ "results": [
288
+ {
289
+ "doc_id": "paper-transformer-p4",
290
+ "title": "Attention Is All You Need",
291
+ "page": 4,
292
+ "score": 0.6,
293
+ "reason": "image:Matched visual cues: architecture, diagram, transformer"
294
+ }
295
+ ]
296
+ }
297
+ ```
298
+
299
+ ## Why the explainability matters
300
+
301
+ The debug surface is a core feature, not an afterthought. For production retrieval systems, we need to answer:
302
+
303
+ - Which modality did the router choose?
304
+ - Which modalities were skipped?
305
+ - Why did a page rank highly?
306
+ - What tradeoff did fusion make?
307
+
308
+ That is the wedge for MARE: make multimodal retrieval inspectable before trying to make it magical.
309
+
310
+ ## Local sample data
311
+
312
+ `examples/sample_corpus.json` contains a tiny IR-paper-style corpus so the routing and fusion path is runnable out of the box.
313
+
314
+ There is also a local PDF in this workspace:
315
+
316
+ - `MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf`
317
+
318
+ That file can now be ingested into a JSON page corpus with `mare-ingest`.
319
+
320
+ ## Roadmap
321
+
322
+ ### v0.1
323
+
324
+ - text + image + layout routing
325
+ - weighted late fusion
326
+ - explainable retrieval output
327
+ - tests and runnable demo
328
+
329
+ ### v0.2
330
+
331
+ - pluggable embedding backends
332
+ - PDF page ingestion
333
+ - OCR and caption extraction adapters
334
+ - score normalization per modality
335
+
336
+ ### v0.3
337
+
338
+ - learned router
339
+ - benchmark harness for IRPAPERS-style evaluation
340
+ - cost-aware routing budgets
341
+ - reranking and cross-modal evidence aggregation
342
+
343
+ ## Suggested next open-source moves
344
+
345
+ - Add adapters for FAISS, Qdrant, and Weaviate
346
+ - Add page extraction from PDFs
347
+ - Add a benchmark runner that computes Recall@k per modality
348
+ - Add a small web debug UI for route inspection
349
+
350
+ ## License
351
+
352
+ MIT
@@ -0,0 +1,319 @@
1
+ # MARE
2
+
3
+ MARE is an open-source Python library for evidence-first document retrieval.
4
+
5
+ It is inspired by the direction highlighted in the IRPAPERS paper, which shows that page-image retrieval and text retrieval have complementary failure modes on scientific documents. Instead of flattening everything into one retrieval path, MARE treats routing, retrieval, fusion, and observability as separate system concerns.
6
+
7
+ ## What this repo is
8
+
9
+ - A lightweight Python package between a query and modality-specific indexes
10
+ - A baseline router that decides whether a query should hit text, image, layout, or a hybrid path
11
+ - A late-fusion layer that combines modality-specific scores
12
+ - An explainable debug surface that tells you why a modality was selected
13
+
14
+ ## What this repo is not
15
+
16
+ - Not a chatbot wrapper
17
+ - Not a full PDF parsing stack yet
18
+ - Not a claim that heuristic routing is state of the art
19
+
20
+ ## Why now
21
+
22
+ IRPAPERS asks a useful systems question: when should we retrieve over OCR text, page images, layout structure, or some combination? The paper reports that text-based and image-based retrieval each solve queries the other misses, and that fusion improves retrieval quality over either modality alone.
23
+
24
+ This repo turns that observation into an MVP developer layer.
25
+
26
+ Paper: https://arxiv.org/pdf/2602.17687
27
+
28
+ ## Architecture
29
+
30
+ ```text
31
+ query
32
+ -> router
33
+ -> modality-specific retrievers
34
+ -> text index
35
+ -> image index
36
+ -> layout index
37
+ -> fusion
38
+ -> explainable results
39
+ ```
40
+
41
+ Current implementation choices:
42
+
43
+ - Router: keyword heuristic baseline
44
+ - Text retrieval: token-overlap cosine baseline
45
+ - Image retrieval: caption and visual-tag overlap baseline
46
+ - Layout retrieval: layout-hint overlap baseline
47
+ - Fusion: weighted late fusion
48
+
49
+ The point of `v0.1` is not raw benchmark quality. It is to package the control plane cleanly enough that stronger models can drop in later.
50
+
51
+ ## Repo layout
52
+
53
+ ```text
54
+ src/mare/
55
+ engine.py
56
+ router.py
57
+ fusion.py
58
+ types.py
59
+ retrievers/
60
+ examples/
61
+ tests/
62
+ ```
63
+
64
+ ## Quickstart
65
+
66
+ Clone and install:
67
+
68
+ ```bash
69
+ git clone https://github.com/SaiSandeepKantareddy/MARE.git
70
+ cd MARE
71
+ python3 -m venv .venv
72
+ source .venv/bin/activate
73
+ pip install -e ".[dev]"
74
+ ```
75
+
76
+ Or install directly from GitHub:
77
+
78
+ ```bash
79
+ pip install "git+https://github.com/SaiSandeepKantareddy/MARE.git"
80
+ ```
81
+
82
+ The intended package install after PyPI release is:
83
+
84
+ ```bash
85
+ pip install mare-retrieval
86
+ ```
87
+
88
+ Then use it as a library:
89
+
90
+ ```python
91
+ from mare import MAREApp
92
+
93
+ app = MAREApp.from_pdf("manual.pdf", reuse=True)
94
+ best = app.best_match("partially reinstall the set screws if they fall out")
95
+
96
+ print(best.page)
97
+ print(best.snippet)
98
+ print(best.page_image_path)
99
+ ```
100
+
101
+ Or try the sample corpus from the CLI:
102
+
103
+ ```bash
104
+ mare-demo --query "show me the architecture diagram of transformer"
105
+ ```
106
+
107
+ Or without installing the package yet:
108
+
109
+ ```bash
110
+ PYTHONPATH=src python3 -m mare.demo --query "show me the architecture diagram of transformer"
111
+ ```
112
+
113
+ ## Simplest way to use it
114
+
115
+ Use one command:
116
+
117
+ ```bash
118
+ python3 ask.py "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf" "partially reinstall the set screws if they fall out"
119
+ ```
120
+
121
+ That will:
122
+
123
+ - ingest the PDF if needed
124
+ - retrieve the best matching page
125
+ - print the page number
126
+ - print the exact snippet
127
+ - print the rendered page image path
128
+
129
+ If you want to reuse a previously generated corpus:
130
+
131
+ ```bash
132
+ python3 ask.py --reuse "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf" "partially reinstall the set screws if they fall out"
133
+ ```
134
+
135
+ If the PDF filename is awkward, rename it first:
136
+
137
+ ```bash
138
+ mv ./*.pdf ./manual.pdf
139
+ PYTHONPATH=src python3 ask.py ./manual.pdf "partially reinstall the set screws if they fall out"
140
+ ```
141
+
142
+ ## Public Python API
143
+
144
+ The package is meant to be importable, not just runnable from scripts.
145
+
146
+ ```python
147
+ from mare import MAREApp, load_corpus, load_pdf
148
+ ```
149
+
150
+ Create an app from a PDF:
151
+
152
+ ```python
153
+ app = load_pdf("manual.pdf", reuse=True)
154
+ hit = app.best_match("what does MagSafe 3 refer to")
155
+ ```
156
+
157
+ Create an app from an existing JSON corpus:
158
+
159
+ ```python
160
+ app = load_corpus("generated/manual.json")
161
+ results = app.retrieve("show me the comparison table", top_k=3)
162
+ ```
163
+
164
+ Core methods:
165
+
166
+ - `MAREApp.from_pdf(...)`
167
+ - `MAREApp.from_corpus(...)`
168
+ - `MAREApp.from_documents(...)`
169
+ - `app.explain(query)`
170
+ - `app.retrieve(query)`
171
+ - `app.best_match(query)`
172
+
173
+ ## Packaging and release
174
+
175
+ MARE is now structured as a regular Python package with:
176
+
177
+ - `pyproject.toml` metadata
178
+ - legacy-friendly `setup.py`
179
+ - console entry points
180
+ - a PyPI publishing workflow
181
+
182
+ Release notes and PyPI steps live in `PUBLISHING.md`.
183
+
184
+ ## Visual demo
185
+
186
+ If you want to show this to users visually, run the Streamlit demo:
187
+
188
+ ```bash
189
+ pip install -e ".[ui]"
190
+ PYTHONPATH=src streamlit run src/mare/streamlit_app.py
191
+ ```
192
+
193
+ The demo lets a user:
194
+
195
+ - upload a PDF
196
+ - ask a question
197
+ - see the best matching page
198
+ - read the exact evidence snippet
199
+ - view the rendered page image
200
+
201
+ The technical retrieval plan is hidden under a `Debug details` expander so the default experience stays user-facing.
202
+
203
+ ## Ingest a real PDF
204
+
205
+ You can convert a PDF into a page-level JSON corpus and then run retrieval on it.
206
+
207
+ ```bash
208
+ python3 -m venv .venv
209
+ source .venv/bin/activate
210
+ pip install -e ".[dev]"
211
+ mare-ingest "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf"
212
+ mare-demo --corpus "generated/MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.json" --query "what does MagSafe 3 refer to"
213
+ ```
214
+
215
+ Without installing the package first:
216
+
217
+ ```bash
218
+ PYTHONPATH=src python3 -m mare.ingest "MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf"
219
+ PYTHONPATH=src python3 -m mare.demo --corpus "generated/MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.json" --query "what does MagSafe 3 refer to"
220
+ ```
221
+
222
+ What the ingest step does right now:
223
+
224
+ - reads each PDF page with `pypdf`
225
+ - renders each PDF page to `generated/<pdf-name>/page-N.png`
226
+ - extracts page text
227
+ - creates one retrieval document per page
228
+ - adds lightweight layout hints when terms like `Table` or `Figure` appear
229
+ - writes a JSON corpus that the retriever can search immediately
230
+
231
+ This is still a simple baseline. OCR, figure extraction, and true layout modeling are the next step.
232
+
233
+ ## What you get back
234
+
235
+ The retriever now returns:
236
+
237
+ - the matching page number
238
+ - why that page matched
239
+ - a short exact snippet from the page text
240
+ - the rendered page image path
241
+
242
+ That makes it easier to validate whether retrieval found the right instruction and jump to the exact page image.
243
+
244
+ Example output:
245
+
246
+ ```json
247
+ {
248
+ "query": "show me the architecture diagram of transformer",
249
+ "intent": "visual_lookup",
250
+ "selected_modalities": ["image"],
251
+ "discarded_modalities": ["text", "layout"],
252
+ "confidence": 0.8,
253
+ "rationale": "Detected modality cues in query tokens. Selected image based on keyword overlap with routing hints.",
254
+ "results": [
255
+ {
256
+ "doc_id": "paper-transformer-p4",
257
+ "title": "Attention Is All You Need",
258
+ "page": 4,
259
+ "score": 0.6,
260
+ "reason": "image:Matched visual cues: architecture, diagram, transformer"
261
+ }
262
+ ]
263
+ }
264
+ ```
265
+
266
+ ## Why the explainability matters
267
+
268
+ The debug surface is a core feature, not an afterthought. For production retrieval systems, we need to answer:
269
+
270
+ - Which modality did the router choose?
271
+ - Which modalities were skipped?
272
+ - Why did a page rank highly?
273
+ - What tradeoff did fusion make?
274
+
275
+ That is the wedge for MARE: make multimodal retrieval inspectable before trying to make it magical.
276
+
277
+ ## Local sample data
278
+
279
+ `examples/sample_corpus.json` contains a tiny IR-paper-style corpus so the routing and fusion path is runnable out of the box.
280
+
281
+ There is also a local PDF in this workspace:
282
+
283
+ - `MacBook Pro (14-inch, M5 Pro or M5 Max) MagSafe 3 Board - Apple Support.pdf`
284
+
285
+ That file can now be ingested into a JSON page corpus with `mare-ingest`.
286
+
287
+ ## Roadmap
288
+
289
+ ### v0.1
290
+
291
+ - text + image + layout routing
292
+ - weighted late fusion
293
+ - explainable retrieval output
294
+ - tests and runnable demo
295
+
296
+ ### v0.2
297
+
298
+ - pluggable embedding backends
299
+ - PDF page ingestion
300
+ - OCR and caption extraction adapters
301
+ - score normalization per modality
302
+
303
+ ### v0.3
304
+
305
+ - learned router
306
+ - benchmark harness for IRPAPERS-style evaluation
307
+ - cost-aware routing budgets
308
+ - reranking and cross-modal evidence aggregation
309
+
310
+ ## Suggested next open-source moves
311
+
312
+ - Add adapters for FAISS, Qdrant, and Weaviate
313
+ - Add page extraction from PDFs
314
+ - Add a benchmark runner that computes Recall@k per modality
315
+ - Add a small web debug UI for route inspection
316
+
317
+ ## License
318
+
319
+ MIT