llm-markdownify 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- llm_markdownify-0.1.0/.gitignore +28 -0
- llm_markdownify-0.1.0/LICENSE +73 -0
- llm_markdownify-0.1.0/NOTICE +6 -0
- llm_markdownify-0.1.0/PKG-INFO +165 -0
- llm_markdownify-0.1.0/README.md +132 -0
- llm_markdownify-0.1.0/pyproject.toml +82 -0
- llm_markdownify-0.1.0/src/llm_markdownify/__init__.py +15 -0
- llm_markdownify-0.1.0/src/llm_markdownify/api.py +62 -0
- llm_markdownify-0.1.0/src/llm_markdownify/cli.py +74 -0
- llm_markdownify-0.1.0/src/llm_markdownify/config.py +72 -0
- llm_markdownify-0.1.0/src/llm_markdownify/grouping.py +52 -0
- llm_markdownify-0.1.0/src/llm_markdownify/llm.py +54 -0
- llm_markdownify-0.1.0/src/llm_markdownify/logging.py +23 -0
- llm_markdownify-0.1.0/src/llm_markdownify/markdownifier.py +78 -0
- llm_markdownify-0.1.0/src/llm_markdownify/pager.py +78 -0
- llm_markdownify-0.1.0/src/llm_markdownify/prompt_profiles.py +81 -0
- llm_markdownify-0.1.0/src/llm_markdownify/prompts.py +78 -0
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Python
|
|
2
|
+
__pycache__/
|
|
3
|
+
*.py[cod]
|
|
4
|
+
*.pyo
|
|
5
|
+
*.pyd
|
|
6
|
+
*.so
|
|
7
|
+
|
|
8
|
+
# Envs
|
|
9
|
+
.venv/
|
|
10
|
+
.uv/
|
|
11
|
+
.env
|
|
12
|
+
.env.*
|
|
13
|
+
|
|
14
|
+
# Build
|
|
15
|
+
build/
|
|
16
|
+
dist/
|
|
17
|
+
*.egg-info/
|
|
18
|
+
|
|
19
|
+
# IDE
|
|
20
|
+
.vscode/
|
|
21
|
+
.idea/
|
|
22
|
+
|
|
23
|
+
# Tests
|
|
24
|
+
.coverage
|
|
25
|
+
htmlcov/
|
|
26
|
+
.pytest_cache/
|
|
27
|
+
|
|
28
|
+
test_files/
|
|
@@ -0,0 +1,73 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction, and distribution as defined by Sections 1 through 9 of this document.
|
|
10
|
+
|
|
11
|
+
"Licensor" shall mean the copyright owner or entity authorized by the copyright owner that is granting the License.
|
|
12
|
+
|
|
13
|
+
"Legal Entity" shall mean the union of the acting entity and all other entities that control, are controlled by, or are under common control with that entity. For the purposes of this definition, "control" means (i) the power, direct or indirect, to cause the direction or management of such entity, whether by contract or otherwise, or (ii) ownership of fifty percent (50%) or more of the outstanding shares, or (iii) beneficial ownership of such entity.
|
|
14
|
+
|
|
15
|
+
"You" (or "Your") shall mean an individual or Legal Entity exercising permissions granted by this License.
|
|
16
|
+
|
|
17
|
+
"Source" form shall mean the preferred form for making modifications, including but not limited to software source code, documentation source, and configuration files.
|
|
18
|
+
|
|
19
|
+
"Object" form shall mean any form resulting from mechanical transformation or translation of a Source form, including but not limited to compiled object code, generated documentation, and conversions to other media types.
|
|
20
|
+
|
|
21
|
+
"Work" shall mean the work of authorship, whether in Source or Object form, made available under the License, as indicated by a copyright notice that is included in or attached to the work (an example is provided in the Appendix below).
|
|
22
|
+
|
|
23
|
+
"Derivative Works" shall mean any work, whether in Source or Object form, that is based on (or derived from) the Work and for which the editorial revisions, annotations, elaborations, or other modifications represent, as a whole, an original work of authorship. For the purposes of this License, Derivative Works shall not include works that remain separable from, or merely link (or bind by name) to the interfaces of, the Work and Derivative Works thereof.
|
|
24
|
+
|
|
25
|
+
"Contribution" shall mean any work of authorship, including the original version of the Work and any modifications or additions to that Work or Derivative Works thereof, that is intentionally submitted to Licensor for inclusion in the Work by the copyright owner or by an individual or Legal Entity authorized to submit on behalf of the copyright owner. For the purposes of this definition, "submitted" means any form of electronic, verbal, or written communication sent to the Licensor or its representatives, including but not limited to communication on electronic mailing lists, source code control systems, and issue tracking systems that are managed by, or on behalf of, the Licensor for the purpose of discussing and improving the Work, but excluding communication that is conspicuously marked or otherwise designated in writing by the copyright owner as "Not a Contribution."
|
|
26
|
+
|
|
27
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity on behalf of whom a Contribution has been received by Licensor and subsequently incorporated within the Work.
|
|
28
|
+
|
|
29
|
+
2. Grant of Copyright License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable copyright license to reproduce, prepare Derivative Works of, publicly display, publicly perform, sublicense, and distribute the Work and such Derivative Works in Source or Object form.
|
|
30
|
+
|
|
31
|
+
3. Grant of Patent License. Subject to the terms and conditions of this License, each Contributor hereby grants to You a perpetual, worldwide, non-exclusive, no-charge, royalty-free, irrevocable (except as stated in this section) patent license to make, have made, use, offer to sell, sell, import, and otherwise transfer the Work, where such license applies only to those patent claims licensable by such Contributor that are necessarily infringed by their Contribution(s) alone or by combination of their Contribution(s) with the Work to which such Contribution(s) was submitted. If You institute patent litigation against any entity (including a cross-claim or counterclaim in a lawsuit) alleging that the Work or a Contribution incorporated within the Work constitutes direct or contributory patent infringement, then any patent licenses granted to You under this License for that Work shall terminate as of the date such litigation is filed.
|
|
32
|
+
|
|
33
|
+
4. Redistribution. You may reproduce and distribute copies of the Work or Derivative Works thereof in any medium, with or without modifications, and in Source or Object form, provided that You meet the following conditions:
|
|
34
|
+
|
|
35
|
+
(a) You must give any other recipients of the Work or Derivative Works a copy of this License; and
|
|
36
|
+
|
|
37
|
+
(b) You must cause any modified files to carry prominent notices stating that You changed the files; and
|
|
38
|
+
|
|
39
|
+
(c) You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
|
|
40
|
+
|
|
41
|
+
(d) If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
|
|
42
|
+
|
|
43
|
+
You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
|
|
44
|
+
|
|
45
|
+
5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
|
|
46
|
+
|
|
47
|
+
6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
|
|
48
|
+
|
|
49
|
+
7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
|
|
50
|
+
|
|
51
|
+
8. Limitation of Liability. In no event and under no legal theory, whether in tort (including negligence), contract, or otherwise, unless required by applicable law (such as deliberate and grossly negligent acts) or agreed to in writing, shall any Contributor be liable to You for damages, including any direct, indirect, special, incidental, or consequential damages of any character arising as a result of this License or out of the use or inability to use the Work (including but not limited to damages for loss of goodwill, work stoppage, computer failure or malfunction, or any and all other commercial damages or losses), even if such Contributor has been advised of the possibility of such damages.
|
|
52
|
+
|
|
53
|
+
9. Accepting Warranty or Additional Liability. While redistributing the Work or Derivative Works thereof, You may choose to offer, and charge a fee for, acceptance of support, warranty, indemnity, or other liability obligations and/or rights consistent with this License. However, in accepting such obligations, You may act only on Your own behalf and on Your sole responsibility, not on behalf of any other Contributor, and only if You agree to indemnify, defend, and hold each Contributor harmless for any liability incurred by, or claims asserted against, such Contributor by reason of your accepting any such warranty or additional liability.
|
|
54
|
+
|
|
55
|
+
END OF TERMS AND CONDITIONS
|
|
56
|
+
|
|
57
|
+
APPENDIX: How to apply the Apache License to your work.
|
|
58
|
+
|
|
59
|
+
To apply the Apache License to your work, attach the following boilerplate notice, with the fields enclosed by brackets "[]" replaced with your own identifying information. (Don't include the brackets!) The text should be enclosed in the appropriate comment syntax for the file format. We also recommend that a file or class name and description of purpose be included on the same "printed page" as the copyright notice for easier identification within third-party archives.
|
|
60
|
+
|
|
61
|
+
Copyright [yyyy] [name of copyright owner]
|
|
62
|
+
|
|
63
|
+
Licensed under the Apache License, Version 2.0 (the "License");
|
|
64
|
+
you may not use this file except in compliance with the License.
|
|
65
|
+
You may obtain a copy of the License at
|
|
66
|
+
|
|
67
|
+
http://www.apache.org/licenses/LICENSE-2.0
|
|
68
|
+
|
|
69
|
+
Unless required by applicable law or agreed to in writing, software
|
|
70
|
+
distributed under the License is distributed on an "AS IS" BASIS,
|
|
71
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
72
|
+
See the License for the specific language governing permissions and
|
|
73
|
+
limitations under the License.
|
|
@@ -0,0 +1,165 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: llm-markdownify
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Convert documents (PDF, DOCX) to high-quality Markdown using Vision LLMs via LiteLLM
|
|
5
|
+
Project-URL: Homepage, https://github.com/sethupavan12/Markdownify
|
|
6
|
+
Project-URL: Repository, https://github.com/sethupavan12/Markdownify
|
|
7
|
+
Project-URL: Issues, https://github.com/sethupavan12/Markdownify/issues
|
|
8
|
+
Author: Sethu Pavan Venkata Reddy Pastula
|
|
9
|
+
License: Apache-2.0
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
License-File: NOTICE
|
|
12
|
+
Keywords: docx,litellm,llm,markdown,ocr,pdf,vision
|
|
13
|
+
Classifier: License :: OSI Approved :: Apache Software License
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Topic :: Text Processing :: Markup
|
|
17
|
+
Classifier: Topic :: Utilities
|
|
18
|
+
Requires-Python: >=3.10
|
|
19
|
+
Requires-Dist: litellm[proxy]>=1.43.0
|
|
20
|
+
Requires-Dist: pillow>=10.3.0
|
|
21
|
+
Requires-Dist: pydantic>=2.7.0
|
|
22
|
+
Requires-Dist: pypdfium2>=4.30.0
|
|
23
|
+
Requires-Dist: tqdm>=4.66.0
|
|
24
|
+
Requires-Dist: typer>=0.12.3
|
|
25
|
+
Provides-Extra: dev
|
|
26
|
+
Requires-Dist: pre-commit>=3.6.0; extra == 'dev'
|
|
27
|
+
Requires-Dist: pytest-cov>=5.0.0; extra == 'dev'
|
|
28
|
+
Requires-Dist: pytest>=8.2.0; extra == 'dev'
|
|
29
|
+
Requires-Dist: ruff>=0.5.6; extra == 'dev'
|
|
30
|
+
Provides-Extra: docx
|
|
31
|
+
Requires-Dist: docx2pdf>=0.1.8; extra == 'docx'
|
|
32
|
+
Description-Content-Type: text/markdown
|
|
33
|
+
|
|
34
|
+
## Markdownify
|
|
35
|
+
|
|
36
|
+
Mardownify is a super easy-to-use PDF to high-quality Markdown converter using Vision LLMs. It supports text, images, signatures, tables, charts, flowcharts and preserves document structure (Headings, numbered lists etc).
|
|
37
|
+
|
|
38
|
+
Tables become Markdown tables, charts become Mermaid diagrams, and images get concise summaries. Use as a CLI or Python library. Works with 100+ LLMs. Recommended to use with `gpt-5-mini` or `gpt-4.1-mini` or even better models for better performance.
|
|
39
|
+
|
|
40
|
+
### Features
|
|
41
|
+
- High-quality complex markdown generation powered by LLMs.
|
|
42
|
+
- Supports Text, Images, Tables, Charts.
|
|
43
|
+
- Built-in prompts tuned for clean Markdown, Mermaid, and structured headings along with ability to customise.
|
|
44
|
+
- Supports multi-page tables, charts and images.
|
|
45
|
+
- High-fidelity page rendering from PDF.
|
|
46
|
+
- Optional DOCX→PDF conversion using MS word installation.
|
|
47
|
+
- Works seamlessly with 100+ LLMs with LiteLLM Intergration.
|
|
48
|
+
|
|
49
|
+
### Install
|
|
50
|
+
```bash
|
|
51
|
+
uv pip install llm-markdownify
|
|
52
|
+
# or
|
|
53
|
+
pip install llm-markdownify
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
### Quickstart (CLI)
|
|
57
|
+
```bash
|
|
58
|
+
markdownify run input.pdf -o output.md --model gpt-5-mini
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
### Python API (one-liner)
|
|
62
|
+
```python
|
|
63
|
+
from llm_markdownify import convert
|
|
64
|
+
|
|
65
|
+
convert(
|
|
66
|
+
"input.pdf",
|
|
67
|
+
"output.md",
|
|
68
|
+
model="gpt-5-mini", # optional; can rely on env/provider defaults
|
|
69
|
+
dpi=72,
|
|
70
|
+
profile="contracts", # or path to JSON profile
|
|
71
|
+
)
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
Optional DOCX support (macOS/Windows via Word):
|
|
75
|
+
```bash
|
|
76
|
+
pip install llm-markdownify[docx]
|
|
77
|
+
```
|
|
78
|
+
|
|
79
|
+
### Configure your provider (via LiteLLM)
|
|
80
|
+
Pick one of the following. See the full providers list and details in the LiteLLM docs: [Supported Providers](https://docs.litellm.ai/docs/providers).
|
|
81
|
+
|
|
82
|
+
- **OpenAI**
|
|
83
|
+
- Set your API key:
|
|
84
|
+
```bash
|
|
85
|
+
export OPENAI_API_KEY="sk-..."
|
|
86
|
+
```
|
|
87
|
+
- Example usage:
|
|
88
|
+
```bash
|
|
89
|
+
markdownify run input.pdf -o output.md --model gpt-5-mini
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
- **Google Gemini**
|
|
93
|
+
- Set your API key (Google AI Studio key):
|
|
94
|
+
```bash
|
|
95
|
+
export GOOGLE_API_KEY="..."
|
|
96
|
+
```
|
|
97
|
+
- Example usage (pick a Gemini vision-capable model):
|
|
98
|
+
```bash
|
|
99
|
+
markdownify run input.pdf -o output.md --model gemini/gemini-2.5-flash
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
- **Azure OpenAI**
|
|
103
|
+
- Set these environment variables (values from your Azure OpenAI resource):
|
|
104
|
+
```bash
|
|
105
|
+
export AZURE_API_KEY="..."
|
|
106
|
+
export AZURE_API_BASE="https://<your-resource>.openai.azure.com"
|
|
107
|
+
export AZURE_API_VERSION=""
|
|
108
|
+
```
|
|
109
|
+
- Use your deployment name via the `azure/<deployment_name>` model syntax:
|
|
110
|
+
```bash
|
|
111
|
+
markdownify run input.pdf -o output.md --model azure/<deployment_name>
|
|
112
|
+
```
|
|
113
|
+
- See: [LiteLLM Azure OpenAI](https://docs.litellm.ai/docs/providers/azure_openai)
|
|
114
|
+
|
|
115
|
+
- **OpenAI-compatible APIs**
|
|
116
|
+
- Many providers expose an OpenAI-compatible REST API. Set your API key and base URL:
|
|
117
|
+
```bash
|
|
118
|
+
export OPENAI_API_KEY="..."
|
|
119
|
+
export OPENAI_API_BASE="https://your-openai-compatible-endpoint.com/v1"
|
|
120
|
+
```
|
|
121
|
+
- Use the model name supported by that endpoint:
|
|
122
|
+
```bash
|
|
123
|
+
markdownify run input.pdf -o output.md --model <model-name>
|
|
124
|
+
```
|
|
125
|
+
- Reference: [LiteLLM Providers](https://docs.litellm.ai/docs/providers)
|
|
126
|
+
|
|
127
|
+
For additional providers and advanced configuration (fallbacks, cost tracking, streaming), see the LiteLLM docs: [Getting Started](https://docs.litellm.ai/).
|
|
128
|
+
|
|
129
|
+
### Configuration flags
|
|
130
|
+
- `--model`: LiteLLM model (e.g., `gpt-5-mini`, `azure/<deployment>`, `gemini/gemini-2.5-flash`)
|
|
131
|
+
- `--dpi`: Render DPI (default 72)
|
|
132
|
+
- `--max-group-pages`: Max pages to merge for continued content (default 3)
|
|
133
|
+
- `--no-grouping`: Disable LLM-based grouping
|
|
134
|
+
- `--temperature`, `--max-tokens`: LLM generation params
|
|
135
|
+
|
|
136
|
+
### Dev tooling
|
|
137
|
+
- Install pre-commit and enable the license header hook:
|
|
138
|
+
```bash
|
|
139
|
+
pip install pre-commit
|
|
140
|
+
pre-commit install
|
|
141
|
+
# run on all files once
|
|
142
|
+
pre-commit run --all-files
|
|
143
|
+
```
|
|
144
|
+
This inserts the header:
|
|
145
|
+
```
|
|
146
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
147
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
148
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
### Attribution & License
|
|
152
|
+
This project uses the Apache 2.0 License, which includes an attribution/NOTICE requirement. If you distribute or use this project, please keep the `LICENSE` and `NOTICE` files intact, crediting the original author, Sethu Pavan Venkata Reddy Pastula.
|
|
153
|
+
|
|
154
|
+
- Project repository: https://github.com/sethupavan12/Markdownify
|
|
155
|
+
|
|
156
|
+
### Development
|
|
157
|
+
- Requires Python 3.10+
|
|
158
|
+
- Use `uv` for fast installs: `uv sync`
|
|
159
|
+
- Run tests: `pytest`
|
|
160
|
+
- Lint: `ruff check src tests`
|
|
161
|
+
|
|
162
|
+
### Releasing
|
|
163
|
+
GitHub Actions are configured to:
|
|
164
|
+
- Run tests on PRs/pushes
|
|
165
|
+
- Build & publish to PyPI on tagged releases (requires `PYPI_API_TOKEN` secret)
|
|
@@ -0,0 +1,132 @@
|
|
|
1
|
+
## Markdownify
|
|
2
|
+
|
|
3
|
+
Mardownify is a super easy-to-use PDF to high-quality Markdown converter using Vision LLMs. It supports text, images, signatures, tables, charts, flowcharts and preserves document structure (Headings, numbered lists etc).
|
|
4
|
+
|
|
5
|
+
Tables become Markdown tables, charts become Mermaid diagrams, and images get concise summaries. Use as a CLI or Python library. Works with 100+ LLMs. Recommended to use with `gpt-5-mini` or `gpt-4.1-mini` or even better models for better performance.
|
|
6
|
+
|
|
7
|
+
### Features
|
|
8
|
+
- High-quality complex markdown generation powered by LLMs.
|
|
9
|
+
- Supports Text, Images, Tables, Charts.
|
|
10
|
+
- Built-in prompts tuned for clean Markdown, Mermaid, and structured headings along with ability to customise.
|
|
11
|
+
- Supports multi-page tables, charts and images.
|
|
12
|
+
- High-fidelity page rendering from PDF.
|
|
13
|
+
- Optional DOCX→PDF conversion using MS word installation.
|
|
14
|
+
- Works seamlessly with 100+ LLMs with LiteLLM Intergration.
|
|
15
|
+
|
|
16
|
+
### Install
|
|
17
|
+
```bash
|
|
18
|
+
uv pip install llm-markdownify
|
|
19
|
+
# or
|
|
20
|
+
pip install llm-markdownify
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
### Quickstart (CLI)
|
|
24
|
+
```bash
|
|
25
|
+
markdownify run input.pdf -o output.md --model gpt-5-mini
|
|
26
|
+
```
|
|
27
|
+
|
|
28
|
+
### Python API (one-liner)
|
|
29
|
+
```python
|
|
30
|
+
from llm_markdownify import convert
|
|
31
|
+
|
|
32
|
+
convert(
|
|
33
|
+
"input.pdf",
|
|
34
|
+
"output.md",
|
|
35
|
+
model="gpt-5-mini", # optional; can rely on env/provider defaults
|
|
36
|
+
dpi=72,
|
|
37
|
+
profile="contracts", # or path to JSON profile
|
|
38
|
+
)
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Optional DOCX support (macOS/Windows via Word):
|
|
42
|
+
```bash
|
|
43
|
+
pip install llm-markdownify[docx]
|
|
44
|
+
```
|
|
45
|
+
|
|
46
|
+
### Configure your provider (via LiteLLM)
|
|
47
|
+
Pick one of the following. See the full providers list and details in the LiteLLM docs: [Supported Providers](https://docs.litellm.ai/docs/providers).
|
|
48
|
+
|
|
49
|
+
- **OpenAI**
|
|
50
|
+
- Set your API key:
|
|
51
|
+
```bash
|
|
52
|
+
export OPENAI_API_KEY="sk-..."
|
|
53
|
+
```
|
|
54
|
+
- Example usage:
|
|
55
|
+
```bash
|
|
56
|
+
markdownify run input.pdf -o output.md --model gpt-5-mini
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
- **Google Gemini**
|
|
60
|
+
- Set your API key (Google AI Studio key):
|
|
61
|
+
```bash
|
|
62
|
+
export GOOGLE_API_KEY="..."
|
|
63
|
+
```
|
|
64
|
+
- Example usage (pick a Gemini vision-capable model):
|
|
65
|
+
```bash
|
|
66
|
+
markdownify run input.pdf -o output.md --model gemini/gemini-2.5-flash
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
- **Azure OpenAI**
|
|
70
|
+
- Set these environment variables (values from your Azure OpenAI resource):
|
|
71
|
+
```bash
|
|
72
|
+
export AZURE_API_KEY="..."
|
|
73
|
+
export AZURE_API_BASE="https://<your-resource>.openai.azure.com"
|
|
74
|
+
export AZURE_API_VERSION=""
|
|
75
|
+
```
|
|
76
|
+
- Use your deployment name via the `azure/<deployment_name>` model syntax:
|
|
77
|
+
```bash
|
|
78
|
+
markdownify run input.pdf -o output.md --model azure/<deployment_name>
|
|
79
|
+
```
|
|
80
|
+
- See: [LiteLLM Azure OpenAI](https://docs.litellm.ai/docs/providers/azure_openai)
|
|
81
|
+
|
|
82
|
+
- **OpenAI-compatible APIs**
|
|
83
|
+
- Many providers expose an OpenAI-compatible REST API. Set your API key and base URL:
|
|
84
|
+
```bash
|
|
85
|
+
export OPENAI_API_KEY="..."
|
|
86
|
+
export OPENAI_API_BASE="https://your-openai-compatible-endpoint.com/v1"
|
|
87
|
+
```
|
|
88
|
+
- Use the model name supported by that endpoint:
|
|
89
|
+
```bash
|
|
90
|
+
markdownify run input.pdf -o output.md --model <model-name>
|
|
91
|
+
```
|
|
92
|
+
- Reference: [LiteLLM Providers](https://docs.litellm.ai/docs/providers)
|
|
93
|
+
|
|
94
|
+
For additional providers and advanced configuration (fallbacks, cost tracking, streaming), see the LiteLLM docs: [Getting Started](https://docs.litellm.ai/).
|
|
95
|
+
|
|
96
|
+
### Configuration flags
|
|
97
|
+
- `--model`: LiteLLM model (e.g., `gpt-5-mini`, `azure/<deployment>`, `gemini/gemini-2.5-flash`)
|
|
98
|
+
- `--dpi`: Render DPI (default 72)
|
|
99
|
+
- `--max-group-pages`: Max pages to merge for continued content (default 3)
|
|
100
|
+
- `--no-grouping`: Disable LLM-based grouping
|
|
101
|
+
- `--temperature`, `--max-tokens`: LLM generation params
|
|
102
|
+
|
|
103
|
+
### Dev tooling
|
|
104
|
+
- Install pre-commit and enable the license header hook:
|
|
105
|
+
```bash
|
|
106
|
+
pip install pre-commit
|
|
107
|
+
pre-commit install
|
|
108
|
+
# run on all files once
|
|
109
|
+
pre-commit run --all-files
|
|
110
|
+
```
|
|
111
|
+
This inserts the header:
|
|
112
|
+
```
|
|
113
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
114
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
115
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
### Attribution & License
|
|
119
|
+
This project uses the Apache 2.0 License, which includes an attribution/NOTICE requirement. If you distribute or use this project, please keep the `LICENSE` and `NOTICE` files intact, crediting the original author, Sethu Pavan Venkata Reddy Pastula.
|
|
120
|
+
|
|
121
|
+
- Project repository: https://github.com/sethupavan12/Markdownify
|
|
122
|
+
|
|
123
|
+
### Development
|
|
124
|
+
- Requires Python 3.10+
|
|
125
|
+
- Use `uv` for fast installs: `uv sync`
|
|
126
|
+
- Run tests: `pytest`
|
|
127
|
+
- Lint: `ruff check src tests`
|
|
128
|
+
|
|
129
|
+
### Releasing
|
|
130
|
+
GitHub Actions are configured to:
|
|
131
|
+
- Run tests on PRs/pushes
|
|
132
|
+
- Build & publish to PyPI on tagged releases (requires `PYPI_API_TOKEN` secret)
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
[build-system]
|
|
6
|
+
requires = ["hatchling>=1.18"]
|
|
7
|
+
build-backend = "hatchling.build"
|
|
8
|
+
|
|
9
|
+
[project]
|
|
10
|
+
name = "llm-markdownify"
|
|
11
|
+
version = "0.1.0"
|
|
12
|
+
description = "Convert documents (PDF, DOCX) to high-quality Markdown using Vision LLMs via LiteLLM"
|
|
13
|
+
readme = "README.md"
|
|
14
|
+
requires-python = ">=3.10"
|
|
15
|
+
license = { text = "Apache-2.0" }
|
|
16
|
+
authors = [
|
|
17
|
+
{ name = "Sethu Pavan Venkata Reddy Pastula" }
|
|
18
|
+
]
|
|
19
|
+
keywords = ["markdown", "pdf", "docx", "llm", "vision", "ocr", "litellm"]
|
|
20
|
+
classifiers = [
|
|
21
|
+
"Programming Language :: Python :: 3",
|
|
22
|
+
"License :: OSI Approved :: Apache Software License",
|
|
23
|
+
"Operating System :: OS Independent",
|
|
24
|
+
"Topic :: Text Processing :: Markup",
|
|
25
|
+
"Topic :: Utilities",
|
|
26
|
+
]
|
|
27
|
+
dependencies = [
|
|
28
|
+
"litellm[proxy]>=1.43.0",
|
|
29
|
+
"pypdfium2>=4.30.0",
|
|
30
|
+
"Pillow>=10.3.0",
|
|
31
|
+
"typer>=0.12.3",
|
|
32
|
+
"pydantic>=2.7.0",
|
|
33
|
+
"tqdm>=4.66.0",
|
|
34
|
+
]
|
|
35
|
+
|
|
36
|
+
[project.optional-dependencies]
|
|
37
|
+
docx = [
|
|
38
|
+
"docx2pdf>=0.1.8",
|
|
39
|
+
]
|
|
40
|
+
dev = [
|
|
41
|
+
"pytest>=8.2.0",
|
|
42
|
+
"pytest-cov>=5.0.0",
|
|
43
|
+
"ruff>=0.5.6",
|
|
44
|
+
"pre-commit>=3.6.0",
|
|
45
|
+
]
|
|
46
|
+
|
|
47
|
+
[dependency-groups]
|
|
48
|
+
dev = [
|
|
49
|
+
"pytest>=8.2.0",
|
|
50
|
+
"pytest-cov>=5.0.0",
|
|
51
|
+
"ruff>=0.5.6",
|
|
52
|
+
"pre-commit>=3.6.0",
|
|
53
|
+
]
|
|
54
|
+
|
|
55
|
+
[project.urls]
|
|
56
|
+
Homepage = "https://github.com/sethupavan12/Markdownify"
|
|
57
|
+
Repository = "https://github.com/sethupavan12/Markdownify"
|
|
58
|
+
Issues = "https://github.com/sethupavan12/Markdownify/issues"
|
|
59
|
+
|
|
60
|
+
[project.scripts]
|
|
61
|
+
markdownify = "llm_markdownify.cli:app"
|
|
62
|
+
|
|
63
|
+
[tool.uv]
|
|
64
|
+
default-groups = ["dev"]
|
|
65
|
+
|
|
66
|
+
[tool.pytest.ini_options]
|
|
67
|
+
addopts = "-q"
|
|
68
|
+
pythonpath = ["src"]
|
|
69
|
+
|
|
70
|
+
[tool.ruff]
|
|
71
|
+
line-length = 100
|
|
72
|
+
|
|
73
|
+
[tool.hatch.build.targets.wheel]
|
|
74
|
+
packages = ["src/llm_markdownify"]
|
|
75
|
+
|
|
76
|
+
[tool.hatch.build]
|
|
77
|
+
include = [
|
|
78
|
+
"src/llm_markdownify/**",
|
|
79
|
+
"README.md",
|
|
80
|
+
"LICENSE",
|
|
81
|
+
"NOTICE",
|
|
82
|
+
]
|
|
@@ -0,0 +1,15 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
__all__ = [
|
|
6
|
+
"MarkdownifyConfig",
|
|
7
|
+
"Markdownifier",
|
|
8
|
+
"convert",
|
|
9
|
+
]
|
|
10
|
+
|
|
11
|
+
__version__ = "0.1.0"
|
|
12
|
+
|
|
13
|
+
from .config import MarkdownifyConfig # noqa: E402
|
|
14
|
+
from .markdownifier import Markdownifier # noqa: E402
|
|
15
|
+
from .api import convert # noqa: E402
|
|
@@ -0,0 +1,62 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
from __future__ import annotations
|
|
6
|
+
|
|
7
|
+
from pathlib import Path
|
|
8
|
+
from typing import Optional
|
|
9
|
+
|
|
10
|
+
from .config import MarkdownifyConfig
|
|
11
|
+
from .markdownifier import Markdownifier
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
def convert(
|
|
15
|
+
input_path: str | Path,
|
|
16
|
+
output_path: str | Path,
|
|
17
|
+
*,
|
|
18
|
+
model: Optional[str] = None,
|
|
19
|
+
dpi: int = 200,
|
|
20
|
+
max_group_pages: int = 3,
|
|
21
|
+
enable_grouping: bool = True,
|
|
22
|
+
temperature: float = 0.2,
|
|
23
|
+
max_tokens: Optional[int] = None,
|
|
24
|
+
concurrency: int = 4,
|
|
25
|
+
profile: Optional[str] = None,
|
|
26
|
+
allow_docx: bool = False,
|
|
27
|
+
) -> Path:
|
|
28
|
+
"""Convert a document to Markdown using the configured LLM via LiteLLM.
|
|
29
|
+
|
|
30
|
+
Parameters
|
|
31
|
+
- input_path: PDF path (preferred) or DOCX if `allow_docx=True`
|
|
32
|
+
- output_path: Markdown file destination
|
|
33
|
+
- model: LiteLLM model name (e.g., 'gpt-4.1-mini', 'azure/<deployment>', 'gemini/gemini-2.5-flash')
|
|
34
|
+
- dpi: Render DPI for PDF pages (higher = slower, clearer)
|
|
35
|
+
- max_group_pages: Max pages to merge when a table/chart spans pages
|
|
36
|
+
- enable_grouping: Whether to use LLM to detect cross-page continuations
|
|
37
|
+
- temperature: LLM temperature
|
|
38
|
+
- max_tokens: LLM max tokens; if None, uses config default
|
|
39
|
+
- concurrency: Max parallel LLM calls across page groups
|
|
40
|
+
- profile: Prompt profile name ('contracts', 'generic') or path to a JSON profile
|
|
41
|
+
- allow_docx: Enable DOCX via Word/COM conversion (not recommended; prefer PDFs)
|
|
42
|
+
|
|
43
|
+
Returns
|
|
44
|
+
- Path to the written Markdown file
|
|
45
|
+
"""
|
|
46
|
+
cfg_kwargs = dict(
|
|
47
|
+
input_path=Path(input_path),
|
|
48
|
+
output_path=Path(output_path),
|
|
49
|
+
dpi=dpi,
|
|
50
|
+
max_group_pages=max_group_pages,
|
|
51
|
+
enable_grouping=enable_grouping,
|
|
52
|
+
temperature=temperature,
|
|
53
|
+
concurrency=concurrency,
|
|
54
|
+
allow_docx=allow_docx,
|
|
55
|
+
)
|
|
56
|
+
if model is not None:
|
|
57
|
+
cfg_kwargs["model"] = model
|
|
58
|
+
if max_tokens is not None:
|
|
59
|
+
cfg_kwargs["max_tokens"] = max_tokens
|
|
60
|
+
|
|
61
|
+
cfg = MarkdownifyConfig(**cfg_kwargs)
|
|
62
|
+
return Markdownifier(cfg, profile=profile).run()
|
|
@@ -0,0 +1,74 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
from __future__ import annotations
|
|
6
|
+
|
|
7
|
+
from pathlib import Path
|
|
8
|
+
from typing import Optional
|
|
9
|
+
|
|
10
|
+
import typer
|
|
11
|
+
|
|
12
|
+
from .config import MarkdownifyConfig
|
|
13
|
+
from .markdownifier import Markdownifier
|
|
14
|
+
|
|
15
|
+
app = typer.Typer(help="Convert documents (PDF) to Markdown using Vision LLMs via LiteLLM.")
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
@app.command()
|
|
19
|
+
def run(
|
|
20
|
+
input_path: str = typer.Argument(
|
|
21
|
+
..., help="Path to input .pdf (preferred) or .docx (discouraged)"
|
|
22
|
+
),
|
|
23
|
+
output: str = typer.Option(..., "-o", "--output", help="Output .md path"),
|
|
24
|
+
model: Optional[str] = typer.Option(
|
|
25
|
+
None, help="LiteLLM model, e.g. gpt-4.1-mini, azure/<deployment>, gemini/gemini-2.5-flash"
|
|
26
|
+
),
|
|
27
|
+
dpi: int = typer.Option(
|
|
28
|
+
72,
|
|
29
|
+
help="DPI for rendering PDF pages. Higher DPI may improve OCR accuracy if document is blurry.",
|
|
30
|
+
),
|
|
31
|
+
max_group_pages: int = typer.Option(3, help="Max pages to merge for continued content"),
|
|
32
|
+
grouping: bool = typer.Option(
|
|
33
|
+
True,
|
|
34
|
+
"--grouping/--no-grouping",
|
|
35
|
+
help="Enable LLM-based grouping of continued content",
|
|
36
|
+
),
|
|
37
|
+
temperature: float = typer.Option(
|
|
38
|
+
0.2, help="LLM temperature. Lower makes OCR results more reliable."
|
|
39
|
+
),
|
|
40
|
+
max_tokens: Optional[int] = typer.Option(
|
|
41
|
+
None, help="LLM max tokens (defaults to config default). Change based on model limitations"
|
|
42
|
+
),
|
|
43
|
+
concurrency: int = typer.Option(
|
|
44
|
+
4,
|
|
45
|
+
help="Max concurrent LLM requests for page groups. Higher concurrency means faster processing at the risk of hitting rate limits.",
|
|
46
|
+
),
|
|
47
|
+
profile: Optional[str] = typer.Option(
|
|
48
|
+
None, help="Prompt profile name (e.g., 'contracts', 'generic') or path to JSON profile"
|
|
49
|
+
),
|
|
50
|
+
allow_docx: bool = typer.Option(
|
|
51
|
+
False, help="Allow DOCX via Word/COM conversion (not recommended). Prefer PDFs."
|
|
52
|
+
),
|
|
53
|
+
):
|
|
54
|
+
cfg_kwargs = dict(
|
|
55
|
+
input_path=Path(input_path),
|
|
56
|
+
output_path=Path(output),
|
|
57
|
+
dpi=dpi,
|
|
58
|
+
max_group_pages=max_group_pages,
|
|
59
|
+
enable_grouping=grouping,
|
|
60
|
+
temperature=temperature,
|
|
61
|
+
concurrency=concurrency,
|
|
62
|
+
allow_docx=allow_docx,
|
|
63
|
+
)
|
|
64
|
+
if model:
|
|
65
|
+
cfg_kwargs["model"] = model
|
|
66
|
+
if max_tokens is not None:
|
|
67
|
+
cfg_kwargs["max_tokens"] = max_tokens
|
|
68
|
+
|
|
69
|
+
cfg = MarkdownifyConfig(**cfg_kwargs)
|
|
70
|
+
Markdownifier(cfg, profile=profile).run()
|
|
71
|
+
|
|
72
|
+
|
|
73
|
+
if __name__ == "__main__": # pragma: no cover
|
|
74
|
+
app()
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
from __future__ import annotations
|
|
6
|
+
|
|
7
|
+
import os
|
|
8
|
+
from pathlib import Path
|
|
9
|
+
from typing import Optional
|
|
10
|
+
|
|
11
|
+
from pydantic import BaseModel, Field, field_validator, model_validator
|
|
12
|
+
|
|
13
|
+
|
|
14
|
+
class MarkdownifyConfig(BaseModel):
|
|
15
|
+
"""Configuration for the markdownification process."""
|
|
16
|
+
|
|
17
|
+
input_path: Path = Field(..., description="Path to input PDF/DOCX file")
|
|
18
|
+
output_path: Path = Field(..., description="Path to output Markdown file")
|
|
19
|
+
|
|
20
|
+
dpi: int = Field(72, ge=72, le=600, description="DPI used to render PDF pages")
|
|
21
|
+
max_group_pages: int = Field(3, ge=1, le=10, description="Max pages to group together")
|
|
22
|
+
enable_grouping: bool = Field(True, description="Enable LLM-based grouping")
|
|
23
|
+
|
|
24
|
+
# Prefer PDFs; DOCX allowed only with explicit opt-in
|
|
25
|
+
allow_docx: bool = Field(
|
|
26
|
+
False,
|
|
27
|
+
description="Allow DOCX via Word/COM conversion (not recommended). Prefer PDFs.",
|
|
28
|
+
)
|
|
29
|
+
|
|
30
|
+
model: str = Field(
|
|
31
|
+
default_factory=lambda: os.getenv("LLM_MARKDOWNIFY_MODEL", "gpt-4.1-mini"),
|
|
32
|
+
description="LiteLLM model name (e.g., gpt-4.1-mini, azure/<deployment>, gemini/gemini-2.5-flash)",
|
|
33
|
+
)
|
|
34
|
+
temperature: float = Field(0.1, ge=0.0, le=1)
|
|
35
|
+
max_tokens: int = Field(16000, ge=256, le=128000)
|
|
36
|
+
|
|
37
|
+
concurrency: int = Field(
|
|
38
|
+
4,
|
|
39
|
+
ge=1,
|
|
40
|
+
le=1000,
|
|
41
|
+
description="Max concurrent LLM requests when processing page groups",
|
|
42
|
+
)
|
|
43
|
+
|
|
44
|
+
# Optional path where page images are cached for debugging
|
|
45
|
+
cache_dir: Optional[Path] = Field(None)
|
|
46
|
+
|
|
47
|
+
@field_validator("input_path")
|
|
48
|
+
@classmethod
|
|
49
|
+
def _validate_input(cls, path: Path) -> Path:
|
|
50
|
+
if not path.exists():
|
|
51
|
+
raise ValueError(f"Input file not found: {path}")
|
|
52
|
+
if path.suffix.lower() not in {".pdf", ".docx"}:
|
|
53
|
+
raise ValueError("input_path must be a .pdf or .docx file")
|
|
54
|
+
return path
|
|
55
|
+
|
|
56
|
+
@field_validator("output_path")
|
|
57
|
+
@classmethod
|
|
58
|
+
def _validate_output(cls, path: Path) -> Path:
|
|
59
|
+
parent = path.parent
|
|
60
|
+
if not parent.exists():
|
|
61
|
+
parent.mkdir(parents=True, exist_ok=True)
|
|
62
|
+
if path.suffix.lower() not in {".md", ".markdown"}:
|
|
63
|
+
raise ValueError("output_path must be a .md or .markdown file")
|
|
64
|
+
return path
|
|
65
|
+
|
|
66
|
+
@model_validator(mode="after")
|
|
67
|
+
def _enforce_pdf_preference(self) -> "MarkdownifyConfig":
|
|
68
|
+
if self.input_path.suffix.lower() == ".docx" and not self.allow_docx:
|
|
69
|
+
raise ValueError(
|
|
70
|
+
"DOCX input is not enabled. Prefer exporting to PDF, or rerun with --allow-docx (requires Word/COM)."
|
|
71
|
+
)
|
|
72
|
+
return self
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
from __future__ import annotations
|
|
6
|
+
|
|
7
|
+
from typing import List
|
|
8
|
+
|
|
9
|
+
from .llm import assess_continuation
|
|
10
|
+
from .pager import PageImage
|
|
11
|
+
from .logging import get_logger
|
|
12
|
+
from .prompt_profiles import PromptProfile
|
|
13
|
+
|
|
14
|
+
logger = get_logger("llm_markdownify.grouping")
|
|
15
|
+
|
|
16
|
+
|
|
17
|
+
def group_pages(
|
|
18
|
+
pages: List[PageImage],
|
|
19
|
+
model: str,
|
|
20
|
+
max_group_pages: int,
|
|
21
|
+
enable_grouping: bool,
|
|
22
|
+
profile: PromptProfile,
|
|
23
|
+
) -> List[List[PageImage]]:
|
|
24
|
+
if not pages:
|
|
25
|
+
return []
|
|
26
|
+
|
|
27
|
+
if not enable_grouping:
|
|
28
|
+
return [[p] for p in pages]
|
|
29
|
+
|
|
30
|
+
groups: List[List[PageImage]] = []
|
|
31
|
+
current_group: List[PageImage] = [pages[0]]
|
|
32
|
+
|
|
33
|
+
for i in range(len(pages)):
|
|
34
|
+
if i == len(pages) - 1:
|
|
35
|
+
groups.append(current_group)
|
|
36
|
+
break
|
|
37
|
+
a = pages[i]
|
|
38
|
+
b = pages[i + 1]
|
|
39
|
+
|
|
40
|
+
label = assess_continuation(
|
|
41
|
+
model=model, first_data_url=a.data_url, second_data_url=b.data_url, profile=profile
|
|
42
|
+
)
|
|
43
|
+
logger.info("Continuation assessment for pages %d->%d: %s", a.index + 1, b.index + 1, label)
|
|
44
|
+
|
|
45
|
+
continues = label == "CONTINUE_NEXT"
|
|
46
|
+
if continues and len(current_group) < max_group_pages:
|
|
47
|
+
current_group.append(b)
|
|
48
|
+
else:
|
|
49
|
+
groups.append(current_group)
|
|
50
|
+
current_group = [b]
|
|
51
|
+
|
|
52
|
+
return groups
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
from __future__ import annotations
|
|
6
|
+
|
|
7
|
+
from typing import List
|
|
8
|
+
|
|
9
|
+
import litellm # type: ignore
|
|
10
|
+
from litellm import completion # type: ignore
|
|
11
|
+
|
|
12
|
+
from .prompt_profiles import PromptProfile
|
|
13
|
+
|
|
14
|
+
# Drop unsupported params for strict models (e.g., gpt-5-mini)
|
|
15
|
+
litellm.drop_params = True # type: ignore[attr-defined]
|
|
16
|
+
|
|
17
|
+
|
|
18
|
+
def _message_with_images(text: str, image_data_urls: List[str]) -> dict:
|
|
19
|
+
content = [{"type": "text", "text": text}]
|
|
20
|
+
for url in image_data_urls:
|
|
21
|
+
content.append({"type": "image_url", "image_url": {"url": url}})
|
|
22
|
+
return {"role": "user", "content": content}
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
def assess_continuation(
|
|
26
|
+
model: str,
|
|
27
|
+
first_data_url: str,
|
|
28
|
+
second_data_url: str | None,
|
|
29
|
+
profile: PromptProfile,
|
|
30
|
+
) -> str:
|
|
31
|
+
images = [first_data_url] + ([second_data_url] if second_data_url else [])
|
|
32
|
+
messages = [
|
|
33
|
+
{"role": "system", "content": profile.continuation_system},
|
|
34
|
+
_message_with_images(profile.continuation_user, images),
|
|
35
|
+
]
|
|
36
|
+
resp = completion(model=model, messages=messages, temperature=0.0, max_tokens=4)
|
|
37
|
+
return str(resp["choices"][0]["message"]["content"]).strip().upper()
|
|
38
|
+
|
|
39
|
+
|
|
40
|
+
def generate_markdown(
|
|
41
|
+
model: str,
|
|
42
|
+
image_data_urls: List[str],
|
|
43
|
+
profile: PromptProfile,
|
|
44
|
+
temperature: float = 0.2,
|
|
45
|
+
max_tokens: int = 2000,
|
|
46
|
+
) -> str:
|
|
47
|
+
messages = [
|
|
48
|
+
{"role": "system", "content": profile.markdown_system},
|
|
49
|
+
_message_with_images(profile.markdown_user, image_data_urls),
|
|
50
|
+
]
|
|
51
|
+
resp = completion(
|
|
52
|
+
model=model, messages=messages, temperature=temperature, max_tokens=max_tokens
|
|
53
|
+
)
|
|
54
|
+
return str(resp["choices"][0]["message"]["content"]).strip()
|
|
@@ -0,0 +1,23 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
import logging
|
|
6
|
+
import sys
|
|
7
|
+
|
|
8
|
+
|
|
9
|
+
def get_logger(name: str) -> logging.Logger:
|
|
10
|
+
logger = logging.getLogger(name)
|
|
11
|
+
if logger.handlers:
|
|
12
|
+
return logger
|
|
13
|
+
|
|
14
|
+
logger.setLevel(logging.INFO)
|
|
15
|
+
handler = logging.StreamHandler(sys.stdout)
|
|
16
|
+
formatter = logging.Formatter(
|
|
17
|
+
fmt="%(asctime)s | %(levelname)s | %(name)s | %(message)s",
|
|
18
|
+
datefmt="%Y-%m-%d %H:%M:%S",
|
|
19
|
+
)
|
|
20
|
+
handler.setFormatter(formatter)
|
|
21
|
+
logger.addHandler(handler)
|
|
22
|
+
logger.propagate = False
|
|
23
|
+
return logger
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
from __future__ import annotations
|
|
6
|
+
|
|
7
|
+
from concurrent.futures import ThreadPoolExecutor, as_completed
|
|
8
|
+
from pathlib import Path
|
|
9
|
+
from typing import List, Tuple
|
|
10
|
+
|
|
11
|
+
from tqdm import tqdm
|
|
12
|
+
|
|
13
|
+
from .config import MarkdownifyConfig
|
|
14
|
+
from .grouping import group_pages
|
|
15
|
+
from .llm import generate_markdown
|
|
16
|
+
from .logging import get_logger
|
|
17
|
+
from .pager import PageImage, load_document_pages
|
|
18
|
+
from .prompt_profiles import load_prompt_profile, PromptProfile
|
|
19
|
+
|
|
20
|
+
logger = get_logger("llm_markdownify.core")
|
|
21
|
+
|
|
22
|
+
|
|
23
|
+
class Markdownifier:
|
|
24
|
+
"""Orchestrates the conversion of a document into Markdown using a Vision LLM."""
|
|
25
|
+
|
|
26
|
+
def __init__(self, config: MarkdownifyConfig, profile: str | None = None) -> None:
|
|
27
|
+
self.config = config
|
|
28
|
+
self.profile: PromptProfile = load_prompt_profile(profile or "contracts")
|
|
29
|
+
|
|
30
|
+
def _render_pages(self) -> List[PageImage]:
|
|
31
|
+
return load_document_pages(
|
|
32
|
+
self.config.input_path, dpi=self.config.dpi, allow_docx=self.config.allow_docx
|
|
33
|
+
)
|
|
34
|
+
|
|
35
|
+
def _group_pages(self, pages: List[PageImage]) -> List[List[PageImage]]:
|
|
36
|
+
return group_pages(
|
|
37
|
+
pages=pages,
|
|
38
|
+
model=self.config.model,
|
|
39
|
+
max_group_pages=self.config.max_group_pages,
|
|
40
|
+
enable_grouping=self.config.enable_grouping,
|
|
41
|
+
profile=self.profile,
|
|
42
|
+
)
|
|
43
|
+
|
|
44
|
+
def _markdown_for_group(self, group: List[PageImage]) -> str:
|
|
45
|
+
image_urls = [p.data_url for p in group]
|
|
46
|
+
return generate_markdown(
|
|
47
|
+
model=self.config.model,
|
|
48
|
+
image_data_urls=image_urls,
|
|
49
|
+
profile=self.profile,
|
|
50
|
+
temperature=self.config.temperature,
|
|
51
|
+
max_tokens=self.config.max_tokens,
|
|
52
|
+
)
|
|
53
|
+
|
|
54
|
+
def run(self) -> Path:
|
|
55
|
+
pages = self._render_pages()
|
|
56
|
+
groups = self._group_pages(pages)
|
|
57
|
+
|
|
58
|
+
logger.info(
|
|
59
|
+
"Processing %d groups with concurrency=%d", len(groups), self.config.concurrency
|
|
60
|
+
)
|
|
61
|
+
|
|
62
|
+
# Submit LLM work in parallel, but preserve order by collecting (idx, result)
|
|
63
|
+
results: List[Tuple[int, str]] = []
|
|
64
|
+
with ThreadPoolExecutor(max_workers=self.config.concurrency) as executor:
|
|
65
|
+
future_to_idx = {
|
|
66
|
+
executor.submit(self._markdown_for_group, group): idx
|
|
67
|
+
for idx, group in enumerate(groups)
|
|
68
|
+
}
|
|
69
|
+
for future in tqdm(as_completed(future_to_idx), total=len(groups), desc="LLM groups"):
|
|
70
|
+
idx = future_to_idx[future]
|
|
71
|
+
md = future.result()
|
|
72
|
+
results.append((idx, md))
|
|
73
|
+
|
|
74
|
+
ordered = [text for _, text in sorted(results, key=lambda t: t[0])]
|
|
75
|
+
output = "\n\n".join(ordered).strip() + "\n"
|
|
76
|
+
self.config.output_path.write_text(output, encoding="utf-8")
|
|
77
|
+
logger.info("Wrote Markdown to %s", self.config.output_path)
|
|
78
|
+
return self.config.output_path
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
from __future__ import annotations
|
|
6
|
+
|
|
7
|
+
import base64
|
|
8
|
+
from dataclasses import dataclass
|
|
9
|
+
from io import BytesIO
|
|
10
|
+
from pathlib import Path
|
|
11
|
+
from typing import Iterable, List
|
|
12
|
+
|
|
13
|
+
import pypdfium2 as pdfium
|
|
14
|
+
|
|
15
|
+
from .logging import get_logger
|
|
16
|
+
|
|
17
|
+
try:
|
|
18
|
+
from docx2pdf import convert as docx2pdf_convert # type: ignore
|
|
19
|
+
except Exception: # pragma: no cover - optional
|
|
20
|
+
docx2pdf_convert = None # type: ignore
|
|
21
|
+
|
|
22
|
+
logger = get_logger("llm_markdownify.pager")
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
@dataclass
|
|
26
|
+
class PageImage:
|
|
27
|
+
index: int
|
|
28
|
+
width: int
|
|
29
|
+
height: int
|
|
30
|
+
content: bytes # PNG bytes
|
|
31
|
+
|
|
32
|
+
@property
|
|
33
|
+
def data_url(self) -> str:
|
|
34
|
+
b64 = base64.b64encode(self.content).decode("ascii")
|
|
35
|
+
return f"data:image/png;base64,{b64}"
|
|
36
|
+
|
|
37
|
+
|
|
38
|
+
def _docx_to_pdf(input_path: Path) -> Path:
|
|
39
|
+
if docx2pdf_convert is None:
|
|
40
|
+
raise RuntimeError(
|
|
41
|
+
"DOCX support requires 'docx2pdf' and platform support for Word/COM. Prefer PDFs."
|
|
42
|
+
)
|
|
43
|
+
temp_pdf = input_path.with_suffix(".converted.pdf")
|
|
44
|
+
logger.info("Converting DOCX to PDF: %s -> %s", input_path, temp_pdf)
|
|
45
|
+
docx2pdf_convert(str(input_path), str(temp_pdf))
|
|
46
|
+
return temp_pdf
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
def iter_pdf_pages_as_images(pdf_path: Path, dpi: int) -> Iterable[PageImage]:
|
|
50
|
+
logger.info("Rendering PDF pages to images at %s DPI", dpi)
|
|
51
|
+
pdf = pdfium.PdfDocument(str(pdf_path))
|
|
52
|
+
num_pages = len(pdf)
|
|
53
|
+
scale = dpi / 72.0
|
|
54
|
+
for i in range(num_pages):
|
|
55
|
+
page = pdf[i]
|
|
56
|
+
bitmap = page.render(scale=scale)
|
|
57
|
+
pil_image = bitmap.to_pil()
|
|
58
|
+
with BytesIO() as buf:
|
|
59
|
+
pil_image.save(buf, format="PNG")
|
|
60
|
+
data = buf.getvalue()
|
|
61
|
+
yield PageImage(index=i, width=pil_image.width, height=pil_image.height, content=data)
|
|
62
|
+
|
|
63
|
+
|
|
64
|
+
def load_document_pages(input_path: Path, dpi: int, allow_docx: bool = False) -> List[PageImage]:
|
|
65
|
+
"""Load a PDF (preferred) or DOCX (if allowed) as a list of rendered page images."""
|
|
66
|
+
suffix = input_path.suffix.lower()
|
|
67
|
+
if suffix == ".docx":
|
|
68
|
+
if not allow_docx:
|
|
69
|
+
raise ValueError(
|
|
70
|
+
"DOCX not allowed. Prefer exporting DOCX to PDF, or enable --allow-docx (requires Word/COM)."
|
|
71
|
+
)
|
|
72
|
+
pdf_path = _docx_to_pdf(input_path)
|
|
73
|
+
else:
|
|
74
|
+
pdf_path = input_path
|
|
75
|
+
|
|
76
|
+
pages = list(iter_pdf_pages_as_images(pdf_path, dpi=dpi))
|
|
77
|
+
logger.info("Loaded %d pages", len(pages))
|
|
78
|
+
return pages
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
from __future__ import annotations
|
|
6
|
+
|
|
7
|
+
import json
|
|
8
|
+
from dataclasses import dataclass
|
|
9
|
+
from pathlib import Path
|
|
10
|
+
from typing import Dict
|
|
11
|
+
|
|
12
|
+
from . import prompts as default_prompts
|
|
13
|
+
|
|
14
|
+
|
|
15
|
+
@dataclass(frozen=True)
|
|
16
|
+
class PromptProfile:
|
|
17
|
+
name: str
|
|
18
|
+
continuation_system: str
|
|
19
|
+
continuation_user: str
|
|
20
|
+
markdown_system: str
|
|
21
|
+
markdown_user: str
|
|
22
|
+
|
|
23
|
+
|
|
24
|
+
_BUILTIN_PROFILES: Dict[str, PromptProfile] = {
|
|
25
|
+
"contracts": PromptProfile(
|
|
26
|
+
name="contracts",
|
|
27
|
+
continuation_system=default_prompts.CONTINUATION_SYSTEM_PROMPT,
|
|
28
|
+
continuation_user=default_prompts.CONTINUATION_USER_PROMPT,
|
|
29
|
+
markdown_system=default_prompts.MARKDOWN_SYSTEM_PROMPT,
|
|
30
|
+
markdown_user=default_prompts.MARKDOWN_USER_PROMPT,
|
|
31
|
+
),
|
|
32
|
+
"generic": PromptProfile(
|
|
33
|
+
name="generic",
|
|
34
|
+
continuation_system=(
|
|
35
|
+
"You analyze page images to decide if content visually continues (tables/charts split). "
|
|
36
|
+
"Respond only CONTINUE_NEXT or NONE."
|
|
37
|
+
),
|
|
38
|
+
continuation_user=(
|
|
39
|
+
"Look for split tables/charts at the end of page A and start of page B. If found, CONTINUE_NEXT; else NONE."
|
|
40
|
+
),
|
|
41
|
+
markdown_system=(
|
|
42
|
+
"Convert page images into clean Markdown. Use clear headings, lists, and tables; cover all pages; avoid page numbers."
|
|
43
|
+
),
|
|
44
|
+
markdown_user=("Produce a single coherent Markdown output from the provided page images."),
|
|
45
|
+
),
|
|
46
|
+
}
|
|
47
|
+
|
|
48
|
+
|
|
49
|
+
def load_prompt_profile(name_or_path: str) -> PromptProfile:
|
|
50
|
+
candidate = Path(name_or_path)
|
|
51
|
+
if candidate.exists() and candidate.is_file():
|
|
52
|
+
with candidate.open("r", encoding="utf-8") as f:
|
|
53
|
+
data = json.load(f)
|
|
54
|
+
required = {
|
|
55
|
+
"name",
|
|
56
|
+
"continuation_system",
|
|
57
|
+
"continuation_user",
|
|
58
|
+
"markdown_system",
|
|
59
|
+
"markdown_user",
|
|
60
|
+
}
|
|
61
|
+
missing = required - set(data.keys())
|
|
62
|
+
if missing:
|
|
63
|
+
raise ValueError(
|
|
64
|
+
f"Prompt profile missing required fields {sorted(missing)} in {candidate}"
|
|
65
|
+
)
|
|
66
|
+
return PromptProfile(
|
|
67
|
+
name=str(data["name"]),
|
|
68
|
+
continuation_system=str(data["continuation_system"]),
|
|
69
|
+
continuation_user=str(data["continuation_user"]),
|
|
70
|
+
markdown_system=str(data["markdown_system"]),
|
|
71
|
+
markdown_user=str(data["markdown_user"]),
|
|
72
|
+
)
|
|
73
|
+
|
|
74
|
+
key = name_or_path.strip().lower()
|
|
75
|
+
if key in _BUILTIN_PROFILES:
|
|
76
|
+
return _BUILTIN_PROFILES[key]
|
|
77
|
+
|
|
78
|
+
raise ValueError(
|
|
79
|
+
f"Unknown prompt profile '{name_or_path}'. Provide a built-in name ({', '.join(sorted(_BUILTIN_PROFILES))}) "
|
|
80
|
+
f"or a path to a JSON file with the fields: name, continuation_system, continuation_user, markdown_system, markdown_user."
|
|
81
|
+
)
|
|
@@ -0,0 +1,78 @@
|
|
|
1
|
+
# Copyright (c) 2025 Sethu Pavan Venkata Reddy Pastula
|
|
2
|
+
# Licensed under the Apache License, Version 2.0. See LICENSE file for details.
|
|
3
|
+
# SPDX-License-Identifier: Apache-2.0
|
|
4
|
+
|
|
5
|
+
from __future__ import annotations
|
|
6
|
+
|
|
7
|
+
CONTINUATION_SYSTEM_PROMPT = (
|
|
8
|
+
"You are a document structure analyst. Given one or two consecutive page images, "
|
|
9
|
+
"decide if the FIRST page should be MERGED with the NEXT page. \n"
|
|
10
|
+
"MERGE ONLY IF there is a clear visual continuation across the page boundary of: \n"
|
|
11
|
+
"- a table (gridlines/cell borders/columns cut at the bottom of page A and resuming at the top of page B), or\n"
|
|
12
|
+
"- a boxed layout/panel, or\n"
|
|
13
|
+
"- a chart/figure/diagram clearly split across pages.\n"
|
|
14
|
+
"Plain text paragraphs DO NOT qualify. Headings and body text continuity alone is NOT a reason to merge.\n"
|
|
15
|
+
"If uncertain, respond NONE. Respond with ONLY one token: CONTINUE_NEXT or NONE."
|
|
16
|
+
)
|
|
17
|
+
|
|
18
|
+
CONTINUATION_USER_PROMPT = (
|
|
19
|
+
"Check for split visual structures at the bottom of page A and the top of page B: \n"
|
|
20
|
+
"- table gridlines or cell borders continuing,\n"
|
|
21
|
+
"- boxed panels continuing,\n"
|
|
22
|
+
"- charts/figures cut between pages.\n"
|
|
23
|
+
"If you see such a split visual structure, answer CONTINUE_NEXT. Otherwise answer NONE."
|
|
24
|
+
)
|
|
25
|
+
|
|
26
|
+
MARKDOWN_SYSTEM_PROMPT = (
|
|
27
|
+
"You are a meticulous technical writer converting page images into clean Markdown. Follow this CHECKLIST strictly.\n"
|
|
28
|
+
"\n"
|
|
29
|
+
"1) Table of Contents (TOC) detection:\n"
|
|
30
|
+
" - If the page shows a TOC (label like 'CONTENTS' or dense sequential entries like '1 …', '2 …' with dot leaders/page numbers), render it as a simple list.\n"
|
|
31
|
+
" - DO NOT promote TOC entries to headings. Schedules in TOC are listed plainly.\n"
|
|
32
|
+
"\n"
|
|
33
|
+
"2) Top-level sections ('# '):\n"
|
|
34
|
+
" - A top-level section must be a concise, title-like line (typically ≤ 2 lines and a short phrase), not a long sentence.\n"
|
|
35
|
+
" - Use '# ' when the line begins with a single integer section number, optionally followed by a dot (e.g., '1', '1.', '12', '12.'), AND the remainder looks like a section name (short phrase, not a long sentence).\n"
|
|
36
|
+
" - Examples that SHOULD be headings: '# 2. SCOPE OF ENGAGEMENT', '# 18 ESCROW AGREEMENT'.\n"
|
|
37
|
+
" - Output as '# <full original line>' (preserve numeric prefix and casing).\n"
|
|
38
|
+
" - Even if an overall title exists (e.g., '# TERMS AND CONDITIONS'), subsequent integer sections still become '# 1 …', '# 2 …', etc.\n"
|
|
39
|
+
" - If the content after the number reads like a sentence (e.g., contains verbs such as 'shall', 'agree', 'will', 'must' early on, or clearly reads as a full clause), DO NOT make it a heading; keep it as a numbered line.\n"
|
|
40
|
+
"\n"
|
|
41
|
+
"3) Decimal-numbered items (NEVER headings):\n"
|
|
42
|
+
" - Items like '1.2', '12.1', '12.1.1' are NOT headings. Keep them as numbered lines under their parent section.\n"
|
|
43
|
+
" - If a page/group BEGINS with a decimal-numbered item (e.g., '4.3', '4.2.3'), treat it as CONTINUATION from a previous section; DO NOT fabricate '# 4'.\n"
|
|
44
|
+
" - Uppercase/bold/visual prominence does NOT change this rule. Decimal-numbered items must never become headings.\n"
|
|
45
|
+
"\n"
|
|
46
|
+
"4) Subheadings ('## '):\n"
|
|
47
|
+
" - Use '## ' ONLY for true subheadings that are NOT decimal-numbered list points (e.g., '## Warranty' under '# 3. Charges').\n"
|
|
48
|
+
" - Do not convert ordinary numbered paragraphs/lists into headings.\n"
|
|
49
|
+
"\n"
|
|
50
|
+
"5) Tables, charts, and images:\n"
|
|
51
|
+
" - Tables: render as valid GitHub-Flavored Markdown tables with proper alignment.\n"
|
|
52
|
+
" - Charts/diagrams: use Mermaid when feasible.\n"
|
|
53
|
+
" - Images/figures: include concise alt-text/captions. Dont try to provide a link to the image. You are just going to tell what the image is about in excruciating detail.\n"
|
|
54
|
+
"\n"
|
|
55
|
+
"6) Structure & hygiene:\n"
|
|
56
|
+
" - Cover ALL provided page images in order; do not omit later pages.\n"
|
|
57
|
+
" - Keep content readable; do not include page numbers or scanning artifacts.\n"
|
|
58
|
+
"\n"
|
|
59
|
+
"7) Output constraints (critical):\n"
|
|
60
|
+
" - Output ONLY the document content. DO NOT add any meta commentary, assurances, disclaimers, or notes (e.g., 'Note: Decimal-numbered items …', 'According to the images …', 'Summary of changes …').\n"
|
|
61
|
+
" - Do NOT include this checklist, instructions, or any explanation of your process.\n"
|
|
62
|
+
" - Do NOT add content that is not visibly present in the document.\n"
|
|
63
|
+
"\n"
|
|
64
|
+
"Examples (Do/Don't):\n"
|
|
65
|
+
"- Do: '# TERMS AND CONDITIONS' then '# 1. DEFINITIONS AND INTERPRETATION' then lines '1.1 …', '1.2 …'.\n"
|
|
66
|
+
"- Do: '# 2. SCOPE OF ENGAGEMENT' and '# 12 ESCROW AGREEMENT' (short, title-like).\n"
|
|
67
|
+
"- Do: At page start with '4.3 …', keep it as a numbered line (continuation), DO NOT add '# 4'.\n"
|
|
68
|
+
"- Do: TOC as a list of entries (no '#').\n"
|
|
69
|
+
"- Don't: '# 1.2 …' or '## 12.1 …' or '# 12.4 …'.\n"
|
|
70
|
+
"- Don't: '# 5 The Parties shall co-operate …' (this is a long, sentence-like numbered paragraph; keep as a numbered line).\n"
|
|
71
|
+
"- Don't: Any 'Note:'/'Disclaimer:' or statements about your output or method.\n"
|
|
72
|
+
)
|
|
73
|
+
|
|
74
|
+
MARKDOWN_USER_PROMPT = (
|
|
75
|
+
"Convert the provided page images into a single coherent Markdown segment. \n"
|
|
76
|
+
"Apply the checklist strictly: '# ' only for concise, title-like integer-numbered sections ('1'/'1.', '12'/'12.'); decimal-numbered items never become headings (even if bold/uppercase/first line); if the text after an integer number reads as a long sentence, keep it as a numbered line; TOC entries are lists; '## ' reserved only for true non-numbered subheadings. \n"
|
|
77
|
+
"Return ONLY the document content in Markdown with no notes, disclaimers, or extra commentary."
|
|
78
|
+
)
|