mineru 2.2.2__py3-none-any.whl → 2.5.1__py3-none-any.whl

{mineru-2.2.2.dist-info → mineru-2.5.1.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: mineru
-Version: 2.2.2
+Version: 2.5.1
 Summary: A practical tool for converting PDF to Markdown
 License: AGPL-3.0
 Project-URL: homepage, https://mineru.net/
@@ -36,6 +36,8 @@ Requires-Dist: fast-langdetect<0.3.0,>=0.2.3
 Requires-Dist: scikit-image<1.0.0,>=0.25.0
 Requires-Dist: openai<2,>=1.70.0
 Requires-Dist: beautifulsoup4<5,>=4.13.5
+Requires-Dist: magika<0.7.0,>=0.6.2
+Requires-Dist: mineru-vl-utils<1,>=0.1.7
 Provides-Extra: test
 Requires-Dist: mineru[core]; extra == "test"
 Requires-Dist: pytest; extra == "test"
@@ -43,12 +45,11 @@ Requires-Dist: pytest-cov; extra == "test"
 Requires-Dist: coverage; extra == "test"
 Requires-Dist: fuzzywuzzy; extra == "test"
 Provides-Extra: vlm
-Requires-Dist: transformers>=4.51.1; extra == "vlm"
-Requires-Dist: torch>=2.6.0; extra == "vlm"
+Requires-Dist: torch<2.8.0,>=2.6.0; extra == "vlm"
+Requires-Dist: transformers<5.0.0,>=4.51.1; extra == "vlm"
 Requires-Dist: accelerate>=1.5.1; extra == "vlm"
-Requires-Dist: pydantic; extra == "vlm"
-Provides-Extra: sglang
-Requires-Dist: sglang[all]<0.4.11,>=0.4.7; extra == "sglang"
+Provides-Extra: vllm
+Requires-Dist: vllm==0.10.1.1; extra == "vllm"
 Provides-Extra: pipeline
 Requires-Dist: matplotlib<4,>=3.10; extra == "pipeline"
 Requires-Dist: ultralytics<9,>=8.3.48; extra == "pipeline"
@@ -77,7 +78,7 @@ Requires-Dist: mineru[api]; extra == "core"
 Requires-Dist: mineru[gradio]; extra == "core"
 Provides-Extra: all
 Requires-Dist: mineru[core]; extra == "all"
-Requires-Dist: mineru[sglang]; extra == "all"
+Requires-Dist: mineru[vllm]; extra == "all"
 Dynamic: license-file
 
 <div align="center" xmlns="http://www.w3.org/1999/html">
@@ -126,25 +127,67 @@ Dynamic: license-file
 
 # Changelog
 
-- 2025/09/10 2.2.2 Released
-  - Fixed the issue where the new table recognition model would affect the overall parsing task when some table parsing failed
+- 2025/09/19 2.5.1 Released
 
-- 2025/09/08 2.2.1 Released  
-  - Fixed the issue where some newly added models were not downloaded when using the model download command.
+  We are officially releasing MinerU2.5, currently the most powerful multimodal large model for document parsing.
+  With only 1.2B parameters, MinerU2.5's accuracy on the OmniDocBench benchmark comprehensively surpasses top-tier multimodal models like Gemini 2.5 Pro, GPT-4o, and Qwen2.5-VL-72B. It also significantly outperforms leading specialized models such as dots.ocr, MonkeyOCR, and PP-StructureV3.
+  The model has been released on [HuggingFace](https://huggingface.co/opendatalab/MinerU2.5-2509-1.2B) and [ModelScope](https://modelscope.cn/models/opendatalab/MinerU2.5-2509-1.2B) platforms. Welcome to download and use!
+  - Core Highlights:
+    - SOTA Performance with Extreme Efficiency: As a 1.2B model, it achieves State-of-the-Art (SOTA) results that exceed models in the 10B and 100B+ classes, redefining the performance-per-parameter standard in document AI.
+    - Advanced Architecture for Across-the-Board Leadership: By combining a two-stage inference pipeline (decoupling layout analysis from content recognition) with a native high-resolution architecture, it achieves SOTA performance across five key areas: layout analysis, text recognition, formula recognition, table recognition, and reading order.
+  - Key Capability Enhancements:
+    - Layout Detection: Delivers more complete results by accurately covering non-body content like headers, footers, and page numbers. It also provides more precise element localization and natural format reconstruction for lists and references.
+    - Table Parsing: Drastically improves parsing for challenging cases, including rotated tables, borderless/semi-structured tables, and long/complex tables.
+    - Formula Recognition: Significantly boosts accuracy for complex, long-form, and hybrid Chinese-English formulas, greatly enhancing the parsing capability for mathematical documents.
 
-- 2025/09/05 2.2.0 Released
-  - Major Updates
-    - In this version, we focused on improving table parsing accuracy by introducing a new [wired table recognition model](https://github.com/RapidAI/TableStructureRec) and a brand-new hybrid table structure parsing algorithm, significantly enhancing the table recognition capabilities of the `pipeline` backend.
-    - We also added support for cross-page table merging, which is supported by both `pipeline` and `vlm` backends, further improving the completeness and accuracy of table parsing.
-  - Other Updates
-    - The `pipeline` backend now supports 270-degree rotated table parsing, bringing support for table parsing in 0/90/270-degree orientations
-    - `pipeline` added OCR capability support for Thai and Greek, and updated the English OCR model to the latest version. English recognition accuracy improved by 11%, Thai recognition model accuracy is 82.68%, and Greek recognition model accuracy is 89.28% (by PPOCRv5)
-    - Added `bbox` field (mapped to 0-1000 range) in the output `content_list.json`, making it convenient for users to directly obtain position information for each content block
-    - Removed the `pipeline_old_linux` installation option, no longer supporting legacy Linux systems such as `CentOS 7`, to provide better support for `uv`'s `sync`/`run` commands
+  Additionally, with the release of vlm 2.5, we have made some adjustments to the repository:
+  - The vlm backend has been upgraded to version 2.5, supporting the MinerU2.5 model and no longer compatible with the MinerU2.0-2505-0.9B model. The last version supporting the 2.0 model is mineru-2.2.2.
+  - VLM inference-related code has been moved to [mineru_vl_utils](https://github.com/opendatalab/mineru-vl-utils), reducing coupling with the main mineru repository and facilitating independent iteration in the future.
+  - The vlm accelerated inference framework has been switched from `sglang` to `vllm`, achieving full compatibility with the vllm ecosystem, allowing users to use the MinerU2.5 model and accelerated inference on any platform that supports the vllm framework.
+  - Due to major upgrades in the vlm model supporting more layout types, we have made some adjustments to the structure of the parsing intermediate file `middle.json` and result file `content_list.json`. Please refer to the [documentation](https://opendatalab.github.io/MinerU/reference/output_files/) for details.
+
+  Other repository optimizations:
+  - Removed file extension whitelist validation for input files. When input files are PDF documents or images, there are no longer requirements for file extensions, improving usability.
 
 <details>
   <summary>History Log</summary>
 
+  <details>
+    <summary>2025/09/10 2.2.2 Released</summary>
+    <ul>
+      <li>Fixed the issue where the new table recognition model would affect the overall parsing task when some table parsing failed</li>
+    </ul>
+  </details>  
+
+  <details>
+    <summary>2025/09/08 2.2.1 Released</summary>
+    <ul>
+      <li>Fixed the issue where some newly added models were not downloaded when using the model download command.</li>
+    </ul>
+  </details>  
+
+  <details>
+    <summary>2025/09/05 2.2.0 Released</summary>
+    <ul>
+      <li>
+        Major Updates
+        <ul>
+          <li>In this version, we focused on improving table parsing accuracy by introducing a new <a href="https://github.com/RapidAI/TableStructureRec">wired table recognition model</a> and a brand-new hybrid table structure parsing algorithm, significantly enhancing the table recognition capabilities of the <code>pipeline</code> backend.</li>
+          <li>We also added support for cross-page table merging, which is supported by both <code>pipeline</code> and <code>vlm</code> backends, further improving the completeness and accuracy of table parsing.</li>
+        </ul>
+      </li>
+      <li>
+        Other Updates
+        <ul>
+          <li>The <code>pipeline</code> backend now supports 270-degree rotated table parsing, bringing support for table parsing in 0/90/270-degree orientations</li>
+          <li><code>pipeline</code> added OCR capability support for Thai and Greek, and updated the English OCR model to the latest version. English recognition accuracy improved by 11%, Thai recognition model accuracy is 82.68%, and Greek recognition model accuracy is 89.28% (by PPOCRv5)</li>
+          <li>Added <code>bbox</code> field (mapped to 0-1000 range) in the output <code>content_list.json</code>, making it convenient for users to directly obtain position information for each content block</li>
+          <li>Removed the <code>pipeline_old_linux</code> installation option, no longer supporting legacy Linux systems such as <code>CentOS 7</code>, to provide better support for <code>uv</code>'s <code>sync</code>/<code>run</code> commands</li>
+        </ul>
+      </li>
+    </ul>
+  </details>
+
   <details>
     <summary>2025/08/01 2.1.10 Released</summary>
     <ul>
@@ -641,7 +684,7 @@ A WebUI developed based on Gradio, with a simple interface and only core parsing
         <td>Parsing Backend</td>
         <td>pipeline</td>
         <td>vlm-transformers</td>
-        <td>vlm-sglang</td>
+        <td>vlm-vllm</td>
     </tr>
     <tr>
         <td>Operating System</td>
@@ -690,8 +733,8 @@ uv pip install -e .[core]
 ```
 
 > [!TIP]
-> `mineru[core]` includes all core features except `sglang` acceleration, compatible with Windows / Linux / macOS systems, suitable for most users.
-> If you need to use `sglang` acceleration for VLM model inference or install a lightweight client on edge devices, please refer to the documentation [Extension Modules Installation Guide](https://opendatalab.github.io/MinerU/quick_start/extension_modules/).
+> `mineru[core]` includes all core features except `vLLM` acceleration, compatible with Windows / Linux / macOS systems, suitable for most users.
+> If you need to use `vLLM` acceleration for VLM model inference or install a lightweight client on edge devices, please refer to the documentation [Extension Modules Installation Guide](https://opendatalab.github.io/MinerU/quick_start/extension_modules/).
 
 ---
  
@@ -719,8 +762,8 @@ You can use MinerU for PDF parsing through various methods such as command line,
 - [x] Handwritten Text Recognition  
 - [x] Vertical Text Recognition  
 - [x] Latin Accent Mark Recognition
-- [ ] Code block recognition in the main text
-- [ ] [Chemical formula recognition](docs/chemical_knowledge_introduction/introduction.pdf)
+- [x] Code block recognition in the main text
+- [x] [Chemical formula recognition](docs/chemical_knowledge_introduction/introduction.pdf)(mineru.net)
 - [ ] Geometric shape recognition
 
 # Known Issues
@@ -768,6 +811,7 @@ Currently, some models in this project are trained based on YOLO. However, since
 - [pdftext](https://github.com/datalab-to/pdftext)
 - [pdfminer.six](https://github.com/pdfminer/pdfminer.six)
 - [pypdf](https://github.com/py-pdf/pypdf)
+- [magika](https://github.com/google/magika)
 
 # Citation
 
@@ -810,3 +854,4 @@ Currently, some models in this project are trained based on YOLO. However, since
 - [OmniDocBench (A Comprehensive Benchmark for Document Parsing and Evaluation)](https://github.com/opendatalab/OmniDocBench)
 - [Magic-HTML (Mixed web page extraction tool)](https://github.com/opendatalab/magic-html)
 - [Magic-Doc (Fast speed ppt/pptx/doc/docx/pdf extraction tool)](https://github.com/InternLM/magic-doc) 
+- [Dingo: A Comprehensive AI Data Quality Evaluation Tool](https://github.com/MigoXLab/dingo)

{mineru-2.2.2.dist-info → mineru-2.5.1.dist-info}/RECORD

@@ -1,5 +1,5 @@
 mineru/__init__.py,sha256=8CRrCQVuExa0BttRFh3Z40lFy2K5jN0sp67KWjOlj5c,50
-mineru/version.py,sha256=toAYzE_ok1SiBE0AqAVdW0O8YCXCwcx0w4JATYQuJOg,22
+mineru/version.py,sha256=PfQ9ThOuZlUZhThya-_PpR02LjazRR6LNSivpta03mM,22
 mineru/backend/__init__.py,sha256=8CRrCQVuExa0BttRFh3Z40lFy2K5jN0sp67KWjOlj5c,50
 mineru/backend/pipeline/__init__.py,sha256=8CRrCQVuExa0BttRFh3Z40lFy2K5jN0sp67KWjOlj5c,50
 mineru/backend/pipeline/batch_analyze.py,sha256=rp9nHYmuBBytlJIc3oRwqTtgFd5mhRak5UMhQ4mu02Y,21896
@@ -9,25 +9,19 @@ mineru/backend/pipeline/model_list.py,sha256=62_61GQ9ZgS5YmNSpzRkts9kLtbSu8dYrfn
 mineru/backend/pipeline/para_split.py,sha256=Kq95MmvkPm7rKxlCSGiTvVKyF7CErHI2eGGAs5sLl0Q,17119
 mineru/backend/pipeline/pipeline_analyze.py,sha256=rbO5AetOdnxR5ctkoDzFCFoElkz7Jgb7gi2Ct596NK8,6655
 mineru/backend/pipeline/pipeline_magic_model.py,sha256=w8jGx8f6yZN0Wf2yPP3L9rYKc9rogxreZCrUJzJvPO8,14974
-mineru/backend/pipeline/pipeline_middle_json_mkcontent.py,sha256=KlNdxI6-zij_NRzdD11UaAdQNtIo82FCYfeDRzgBeIk,14084
+mineru/backend/pipeline/pipeline_middle_json_mkcontent.py,sha256=xWWOFmYL6hB8PLrxQFyRJ72dAmTIDHtqiWV-WFUfR44,14081
 mineru/backend/vlm/__init__.py,sha256=8CRrCQVuExa0BttRFh3Z40lFy2K5jN0sp67KWjOlj5c,50
-mineru/backend/vlm/base_predictor.py,sha256=1QhREwTLNrQ2zQRRfJuyyb0PPoJRAi9JDkgBEBZpZ6U,6417
-mineru/backend/vlm/hf_predictor.py,sha256=oQE-vkTq1Yu7uFPBAwb-I8FWkyoxfYakOjRjAnnbc_M,7555
-mineru/backend/vlm/predictor.py,sha256=KWvyGB26IVwunf5d_HQao12LUAOLtMr9c48WfOoXbzU,3925
-mineru/backend/vlm/sglang_client_predictor.py,sha256=pXY1tUoV88dhtxKDkGUKJbEsaLPC_TmSwT3fcAD6GyU,15871
-mineru/backend/vlm/sglang_engine_predictor.py,sha256=gnAEaYKCx1_0PGuo7JRcOisMZmgruUucBvAhyrhHui8,8587
-mineru/backend/vlm/token_to_middle_json.py,sha256=JDnXI2f7YgcGRJ5ZrZtZevlX6gdiUVAsfT70vByWIoo,9622
-mineru/backend/vlm/utils.py,sha256=QwTEUpnMVVF30CiNAHJRN7n8ullqGYEH8TQyXy7my08,1361
-mineru/backend/vlm/vlm_analyze.py,sha256=zc6BpduUmIp0ysHKLAlAUVnKA-xRYo_8Pw6QXlVQrko,3588
-mineru/backend/vlm/vlm_magic_model.py,sha256=8e8EnpNZBKyPBLP1UPlRRVnbV9Sxd5u55OgaSdfpECQ,14256
-mineru/backend/vlm/vlm_middle_json_mkcontent.py,sha256=AU0udnwLnxSYxFOhzSn3zG5j3-ic-zuAJAnR_RBrNck,11138
+mineru/backend/vlm/model_output_to_middle_json.py,sha256=e4Yc98_Cth2cjVPybPGehD5cpjGcTka4D2qKKrP_qqo,5121
+mineru/backend/vlm/vlm_analyze.py,sha256=jR5DTHT0qtOLVos5-qSGPvlRCdYREoAshFej98FY5ao,8282
+mineru/backend/vlm/vlm_magic_model.py,sha256=o1WKwgArV0f4lp7ufmSq9zRZaM5bnfOdx1AQtKnosro,16379
+mineru/backend/vlm/vlm_middle_json_mkcontent.py,sha256=DX2TRpfSIqpuhWqou5QXNtCW40ddQi0kdQxXi4QgzKs,13375
 mineru/cli/__init__.py,sha256=8CRrCQVuExa0BttRFh3Z40lFy2K5jN0sp67KWjOlj5c,50
-mineru/cli/client.py,sha256=3F3d59YJZiVgTmI1B8vrZZ1icftsC5Tk8So7hkAkSqs,6565
-mineru/cli/common.py,sha256=63qK2NyAhzT9b0_YN3jbEKeAH7UjOlQnuagZqpinLuU,13812
-mineru/cli/fast_api.py,sha256=yGqzML1KZJc-YQnbjZVxVgPswe2VD7EoGfdHVkr0bbA,11256
-mineru/cli/gradio_app.py,sha256=ckwyzAAZVLRsyMEEFcVnmvoCKf0O3Y1RYi5NQI0JSSU,13458
+mineru/cli/client.py,sha256=uo7db9Wqj1Mc11MYuaM-bi54BfKKU3SFB9Urc8md5X4,6641
+mineru/cli/common.py,sha256=yJPdrwSYVidl2hTJ2Hn2YhnfH97GJ-QZi20dGFz7h5c,14025
+mineru/cli/fast_api.py,sha256=-GDT4gOCjKQrRjrx9WVJw-D-EC7Adv-F2rAiSWdl2CA,11328
+mineru/cli/gradio_app.py,sha256=m1ppNVVOS-gdMpQPWOgHQCBQkLZ4B6gKCZuUyhfsR1g,13482
 mineru/cli/models_download.py,sha256=TCKtzTRJ-ShaqZnRQID40QsILqp2b3basU142FMTmns,4775
-mineru/cli/vlm_sglang_server.py,sha256=EyRL0dt0-SvnG5lzyWB8OuVHHbc1MdeqSszNUk03Y6U,88
+mineru/cli/vlm_vllm_server.py,sha256=fQJyD-gIPQ41hR_6aIaDJczl66N310t0CiZEBAfX5mc,90
 mineru/data/__init__.py,sha256=8CRrCQVuExa0BttRFh3Z40lFy2K5jN0sp67KWjOlj5c,50
 mineru/data/data_reader_writer/__init__.py,sha256=9qnGNrsuGBMwwfsQy6oChdkz--a_LPdYWE0VZZr0yr4,490
 mineru/data/data_reader_writer/base.py,sha256=nqmAcdHOXMOJO6RAT3ILligDFaw8Op0STyCw5yOzAbI,1706
@@ -142,23 +136,15 @@ mineru/model/table/rec/slanet_plus/matcher_utils.py,sha256=9wt_ydeeViLd57bU6g3ln
 mineru/model/table/rec/slanet_plus/table_structure.py,sha256=Ve9eUdA0ivHf5bf9gwvHHfb7-E7drJLP3S3MPlh3uZ0,3844
 mineru/model/table/rec/slanet_plus/table_structure_utils.py,sha256=YYSkwN2WdLx7qkWMSGkPY7yXOH5ENVhg5CsRGhtZ5Wk,19281
 mineru/model/table/rec/unet_table/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-mineru/model/table/rec/unet_table/main.py,sha256=dHJodOnvl2dk7he3QhZpJy19rhxtBxIAEiOlAKx3dXY,15199
+mineru/model/table/rec/unet_table/main.py,sha256=6O5g2Gjml_NBMWKBK1rGdwpcjGIi6JHtjIhxjsGhXIY,15545
 mineru/model/table/rec/unet_table/table_recover.py,sha256=rSyeWyuP10M8dLKA5e0n4P2DXMYbVbmgLxEcdZA8_0E,9059
 mineru/model/table/rec/unet_table/table_structure_unet.py,sha256=beBMmBHAOR2lAuf2rcOKRSbFaJqwuIgMJWxWQsFmIRI,7908
 mineru/model/table/rec/unet_table/utils.py,sha256=CYAqJW0wePJk4NAemb8W203N7E32v0ujiWbxanDhd8I,16083
 mineru/model/table/rec/unet_table/utils_table_line_rec.py,sha256=zrCdPwI4M8nu0FEfd7lRJAe0z8kYq3KFbzwElM82USE,11174
 mineru/model/table/rec/unet_table/utils_table_recover.py,sha256=XksJsY82ZS0kqUnNT-jvaYzxJ3V3svMSzj0puwIau1k,10651
-mineru/model/vlm_hf_model/__init__.py,sha256=a24kdhTzsD__uL0h65MYehkAVgBZvk-BFZn9Cuv95c8,485
-mineru/model/vlm_hf_model/configuration_mineru2.py,sha256=DUr_xaG7jNHTgxMmnQ2aoYag-GgH9-Es5fL3XO8wAlQ,1529
-mineru/model/vlm_hf_model/image_processing_mineru2.py,sha256=MQg7723JeA1v1HlnzdkbMDfmwqbXZpCCPz71DFLfocA,10594
-mineru/model/vlm_hf_model/modeling_mineru2.py,sha256=hBqvzmVR3w1Bm6OhSRc0IvuC7AG5d0j73EXYsmw_2xA,20365
-mineru/model/vlm_sglang_model/__init__.py,sha256=ykNvPdCw-vmSy2UySBGhRkpDfhqEQ4hZ3FGoMU-nhQ8,558
-mineru/model/vlm_sglang_model/engine.py,sha256=E8FJBTclxQNDjnEluiz_QlambyPfo60nKm7ogUZFArg,10128
-mineru/model/vlm_sglang_model/image_processor.py,sha256=lUmHWSNSPED0RhYWwl6-z4R3KePPz12oBmkXBj-MRO4,8561
-mineru/model/vlm_sglang_model/logit_processor.py,sha256=TbNi-QEFXm9lTB94640_dNYZOthpnBcKJnPgwX2eSDs,3679
-mineru/model/vlm_sglang_model/model.py,sha256=faA8UkUNbITvKL8afpMRe4ZtukgtUgaM_uljZVk-2M0,22813
-mineru/model/vlm_sglang_model/server.py,sha256=UZIy61GvsqCtDfyQpMGTeiuJA2XiifmfbEfLIoWkqfg,2403
-mineru/resources/header.html,sha256=Ebc1VPxwLdbcmNHhHFlcb1G_WCwB60i7rX1pec2FsJM,3919
+mineru/model/vlm_vllm_model/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+mineru/model/vlm_vllm_model/server.py,sha256=v07x1esggP7Wbw0r8NeAbqG2kuJN9x5Xl2CmE2x0qzk,2003
+mineru/resources/header.html,sha256=NO8ZZdCYLqu_E72AtNcuRnA2NbFBamScjjGhtg9PKiM,4409
 mineru/resources/fasttext-langdetect/lid.176.ftz,sha256=jzRyz-hzintgmejpmcPL-uDc0VaWqsfXc4qAOdtgPoM,938013
 mineru/utils/__init__.py,sha256=8CRrCQVuExa0BttRFh3Z40lFy2K5jN0sp67KWjOlj5c,50
 mineru/utils/block_pre_proc.py,sha256=uGBmxf2MR9bplTnQI8xHjCI-kj3plRhJr0hcWKidbOQ,9632
@@ -167,9 +153,10 @@ mineru/utils/boxbase.py,sha256=moP660AmZq_udHEsfvFkTQdJ4gjrrBwN7t0Enx7CIL8,6903
 mineru/utils/cli_parser.py,sha256=4seFAu1kulsYnw6WM2q_cxgEOt2tErZVkI-LNEF_kGw,1445
 mineru/utils/config_reader.py,sha256=IRVWTpBnbnRpck6eXZUKw-fcLt7hon5S4uqWW-RBb1w,4075
 mineru/utils/cut_image.py,sha256=g3m4nfcJNWlxi-P0kpXTtlmspXkMcLCfGwmYuQ-Z2hE,751
-mineru/utils/draw_bbox.py,sha256=xTNlION8xY-6a6lgC85ePK1io2jddqFzfkIjAZrpnGc,19088
-mineru/utils/enum_class.py,sha256=3_OMpUKOXTh0djgx4cjbAQZJYuEHMaMPrB-K0egUXGk,2098
+mineru/utils/draw_bbox.py,sha256=FkgppjUzRhN-uxvChdkhHXcDavJEaApMD6qC6qoRwfQ,20292
+mineru/utils/enum_class.py,sha256=34lVsjeAYLha7Q-1qxY9seJFdK6fjuEphXfYFibghEY,2442
 mineru/utils/format_utils.py,sha256=2s89vHcSISjuolk8Hvg3K-5-rRbiT3Us7eFLzUKrNKs,10233
+mineru/utils/guess_suffix_or_lang.py,sha256=q7CbPpiaDFA2AmOS66oJ_HwzamEftNSsmOn5mbrtO9I,540
 mineru/utils/hash_utils.py,sha256=UPS_8NRBmVumdyOv16Lmv6Ly2xK8OVDJEe5gG6gKIFk,857
 mineru/utils/language.py,sha256=7RT3mxSa7jdpoC5ySd7ZddHA7TO7UsnmDOWiYZAxuyg,1433
 mineru/utils/llm_aided.py,sha256=0W6AlBpLfflON1ob6p72IgwdCJKFXhYpDWlrhrToR5s,4892
@@ -182,12 +169,12 @@ mineru/utils/pdf_image_tools.py,sha256=mioLEHOdDtM1YbspNaa0wWhnLw_4-H7rdHlIM40vr
 mineru/utils/pdf_reader.py,sha256=WeINm5SyWBUXT0wP9lzIbeHs8P6WUIkN6nVL5X4LzG4,3267
 mineru/utils/pdf_text_tool.py,sha256=KEztjfdqsIHHuiTEAMAL7Lr1OS3R7Ur-uTqGiCRjReQ,1364
 mineru/utils/run_async.py,sha256=rPeP4BCZerR8VByRDhiYzfZiahLVqoZEBVAS54dAjNg,1286
-mineru/utils/span_block_fix.py,sha256=2i1PLEAAo5O6dwPgDDTzPI_uVBAF2MgQV7zKCj7q0-0,8678
+mineru/utils/span_block_fix.py,sha256=0eVQjJCrT03woRt9hoh6Uu42Tp1dacfGTv2x3B9qq94,8797
 mineru/utils/span_pre_proc.py,sha256=h41q2uQajI0xQbc_30hqaju1dv3oVYxBAlKgURl8HIc,13692
 mineru/utils/table_merge.py,sha256=zYUpYLrfhBCnbHCYZi6rG8-s38NDnTbiNTObvLdYwJk,11494
-mineru-2.2.2.dist-info/licenses/LICENSE.md,sha256=jVa0BUaKrRH4erV2P5AeJ24I2WRv9chIGxditreJ6e0,34524
-mineru-2.2.2.dist-info/METADATA,sha256=tknI_mkDKcjDtiTNEWzI6YMxTKe4EjhqzocGcBjoyTc,60541
-mineru-2.2.2.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
-mineru-2.2.2.dist-info/entry_points.txt,sha256=1bcWxo6oE06gh07gIX6c-71Kb9CSLJ_i6FAmianbXd8,256
-mineru-2.2.2.dist-info/top_level.txt,sha256=zuGQfZcbsHv4I4oKI9gaKPqEWBFm6xJroKuug2LnKP8,7
-mineru-2.2.2.dist-info/RECORD,,
+mineru-2.5.1.dist-info/licenses/LICENSE.md,sha256=jVa0BUaKrRH4erV2P5AeJ24I2WRv9chIGxditreJ6e0,34524
+mineru-2.5.1.dist-info/METADATA,sha256=BnhjCe7hE1EMEZdmz-YQgX0Mq5LANyNWHayKFaJztw0,64460
+mineru-2.5.1.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+mineru-2.5.1.dist-info/entry_points.txt,sha256=luXmbhPiZK_tKlRgWuYOaW_V6EFpG-yJcAevVv9MEqE,252
+mineru-2.5.1.dist-info/top_level.txt,sha256=zuGQfZcbsHv4I4oKI9gaKPqEWBFm6xJroKuug2LnKP8,7
+mineru-2.5.1.dist-info/RECORD,,

{mineru-2.2.2.dist-info → mineru-2.5.1.dist-info}/entry_points.txt

@@ -3,4 +3,4 @@ mineru = mineru.cli:client.main
 mineru-api = mineru.cli.fast_api:main
 mineru-gradio = mineru.cli.gradio_app:main
 mineru-models-download = mineru.cli.models_download:download_models
-mineru-sglang-server = mineru.cli.vlm_sglang_server:main
+mineru-vllm-server = mineru.cli.vlm_vllm_server:main

mineru/backend/vlm/base_predictor.py

@@ -1,186 +0,0 @@
-import asyncio
-from abc import ABC, abstractmethod
-from typing import AsyncIterable, Iterable, List, Optional, Union
-
-DEFAULT_SYSTEM_PROMPT = (
-    "A conversation between a user and an LLM-based AI assistant. The assistant gives helpful and honest answers."
-)
-DEFAULT_USER_PROMPT = "Document Parsing:"
-DEFAULT_TEMPERATURE = 0.0
-DEFAULT_TOP_P = 0.8
-DEFAULT_TOP_K = 20
-DEFAULT_REPETITION_PENALTY = 1.0
-DEFAULT_PRESENCE_PENALTY = 0.0
-DEFAULT_NO_REPEAT_NGRAM_SIZE = 100
-DEFAULT_MAX_NEW_TOKENS = 16384
-
-
-class BasePredictor(ABC):
-    system_prompt = DEFAULT_SYSTEM_PROMPT
-
-    def __init__(
-        self,
-        temperature: float = DEFAULT_TEMPERATURE,
-        top_p: float = DEFAULT_TOP_P,
-        top_k: int = DEFAULT_TOP_K,
-        repetition_penalty: float = DEFAULT_REPETITION_PENALTY,
-        presence_penalty: float = DEFAULT_PRESENCE_PENALTY,
-        no_repeat_ngram_size: int = DEFAULT_NO_REPEAT_NGRAM_SIZE,
-        max_new_tokens: int = DEFAULT_MAX_NEW_TOKENS,
-    ) -> None:
-        self.temperature = temperature
-        self.top_p = top_p
-        self.top_k = top_k
-        self.repetition_penalty = repetition_penalty
-        self.presence_penalty = presence_penalty
-        self.no_repeat_ngram_size = no_repeat_ngram_size
-        self.max_new_tokens = max_new_tokens
-
-    @abstractmethod
-    def predict(
-        self,
-        image: str | bytes,
-        prompt: str = "",
-        temperature: Optional[float] = None,
-        top_p: Optional[float] = None,
-        top_k: Optional[int] = None,
-        repetition_penalty: Optional[float] = None,
-        presence_penalty: Optional[float] = None,
-        no_repeat_ngram_size: Optional[int] = None,
-        max_new_tokens: Optional[int] = None,
-    ) -> str: ...
-
-    @abstractmethod
-    def batch_predict(
-        self,
-        images: List[str] | List[bytes],
-        prompts: Union[List[str], str] = "",
-        temperature: Optional[float] = None,
-        top_p: Optional[float] = None,
-        top_k: Optional[int] = None,
-        repetition_penalty: Optional[float] = None,
-        presence_penalty: Optional[float] = None,
-        no_repeat_ngram_size: Optional[int] = None,
-        max_new_tokens: Optional[int] = None,
-    ) -> List[str]: ...
-
-    @abstractmethod
-    def stream_predict(
-        self,
-        image: str | bytes,
-        prompt: str = "",
-        temperature: Optional[float] = None,
-        top_p: Optional[float] = None,
-        top_k: Optional[int] = None,
-        repetition_penalty: Optional[float] = None,
-        presence_penalty: Optional[float] = None,
-        no_repeat_ngram_size: Optional[int] = None,
-        max_new_tokens: Optional[int] = None,
-    ) -> Iterable[str]: ...
-
-    async def aio_predict(
-        self,
-        image: str | bytes,
-        prompt: str = "",
-        temperature: Optional[float] = None,
-        top_p: Optional[float] = None,
-        top_k: Optional[int] = None,
-        repetition_penalty: Optional[float] = None,
-        presence_penalty: Optional[float] = None,
-        no_repeat_ngram_size: Optional[int] = None,
-        max_new_tokens: Optional[int] = None,
-    ) -> str:
-        return await asyncio.to_thread(
-            self.predict,
-            image,
-            prompt,
-            temperature,
-            top_p,
-            top_k,
-            repetition_penalty,
-            presence_penalty,
-            no_repeat_ngram_size,
-            max_new_tokens,
-        )
-
-    async def aio_batch_predict(
-        self,
-        images: List[str] | List[bytes],
-        prompts: Union[List[str], str] = "",
-        temperature: Optional[float] = None,
-        top_p: Optional[float] = None,
-        top_k: Optional[int] = None,
-        repetition_penalty: Optional[float] = None,
-        presence_penalty: Optional[float] = None,
-        no_repeat_ngram_size: Optional[int] = None,
-        max_new_tokens: Optional[int] = None,
-    ) -> List[str]:
-        return await asyncio.to_thread(
-            self.batch_predict,
-            images,
-            prompts,
-            temperature,
-            top_p,
-            top_k,
-            repetition_penalty,
-            presence_penalty,
-            no_repeat_ngram_size,
-            max_new_tokens,
-        )
-
-    async def aio_stream_predict(
-        self,
-        image: str | bytes,
-        prompt: str = "",
-        temperature: Optional[float] = None,
-        top_p: Optional[float] = None,
-        top_k: Optional[int] = None,
-        repetition_penalty: Optional[float] = None,
-        presence_penalty: Optional[float] = None,
-        no_repeat_ngram_size: Optional[int] = None,
-        max_new_tokens: Optional[int] = None,
-    ) -> AsyncIterable[str]:
-        queue = asyncio.Queue()
-        loop = asyncio.get_running_loop()
-
-        def synced_predict():
-            for chunk in self.stream_predict(
-                image=image,
-                prompt=prompt,
-                temperature=temperature,
-                top_p=top_p,
-                top_k=top_k,
-                repetition_penalty=repetition_penalty,
-                presence_penalty=presence_penalty,
-                no_repeat_ngram_size=no_repeat_ngram_size,
-                max_new_tokens=max_new_tokens,
-            ):
-                asyncio.run_coroutine_threadsafe(queue.put(chunk), loop)
-            asyncio.run_coroutine_threadsafe(queue.put(None), loop)
-
-        asyncio.create_task(
-            asyncio.to_thread(synced_predict),
-        )
-
-        while True:
-            chunk = await queue.get()
-            if chunk is None:
-                return
-            assert isinstance(chunk, str)
-            yield chunk
-
-    def build_prompt(self, prompt: str) -> str:
-        if prompt.startswith("<|im_start|>"):
-            return prompt
-        if not prompt:
-            prompt = DEFAULT_USER_PROMPT
-
-        return f"<|im_start|>system\n{self.system_prompt}<|im_end|><|im_start|>user\n<image>\n{prompt}<|im_end|><|im_start|>assistant\n"
-        # Modify here. We add <|box_start|> at the end of the prompt to force the model to generate bounding box.
-        # if "Document OCR" in prompt:
-        #     return f"<|im_start|>system\n{self.system_prompt}<|im_end|><|im_start|>user\n<image>\n{prompt}<|im_end|><|im_start|>assistant\n<|box_start|>"
-        # else:
-        #     return f"<|im_start|>system\n{self.system_prompt}<|im_end|><|im_start|>user\n<image>\n{prompt}<|im_end|><|im_start|>assistant\n"
-
-    def close(self):
-        pass