docling 1.6.2__py3-none-any.whl → 1.19.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -0,0 +1,380 @@
1
+ Metadata-Version: 2.1
2
+ Name: docling
3
+ Version: 1.19.0
4
+ Summary: Docling PDF conversion package
5
+ Home-page: https://github.com/DS4SD/docling
6
+ License: MIT
7
+ Keywords: docling,convert,document,pdf,layout model,segmentation,table structure,table former
8
+ Author: Christoph Auer
9
+ Author-email: cau@zurich.ibm.com
10
+ Requires-Python: >=3.10,<4.0
11
+ Classifier: Development Status :: 5 - Production/Stable
12
+ Classifier: Intended Audience :: Developers
13
+ Classifier: Intended Audience :: Science/Research
14
+ Classifier: License :: OSI Approved :: MIT License
15
+ Classifier: Operating System :: MacOS :: MacOS X
16
+ Classifier: Operating System :: POSIX :: Linux
17
+ Classifier: Programming Language :: Python :: 3
18
+ Classifier: Programming Language :: Python :: 3.10
19
+ Classifier: Programming Language :: Python :: 3.11
20
+ Classifier: Programming Language :: Python :: 3.12
21
+ Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
22
+ Provides-Extra: tesserocr
23
+ Requires-Dist: certifi (>=2024.7.4)
24
+ Requires-Dist: deepsearch-glm (>=0.22.0,<0.23.0)
25
+ Requires-Dist: docling-core (>=1.6.2,<2.0.0)
26
+ Requires-Dist: docling-ibm-models (>=2.0.0,<3.0.0)
27
+ Requires-Dist: docling-parse (>=1.4.1,<2.0.0)
28
+ Requires-Dist: easyocr (>=1.7,<2.0)
29
+ Requires-Dist: filetype (>=1.2.0,<2.0.0)
30
+ Requires-Dist: huggingface_hub (>=0.23,<1)
31
+ Requires-Dist: pyarrow (>=16.1.0,<17.0.0)
32
+ Requires-Dist: pydantic (>=2.0.0,<3.0.0)
33
+ Requires-Dist: pydantic-settings (>=2.3.0,<3.0.0)
34
+ Requires-Dist: pypdfium2 (>=4.30.0,<5.0.0)
35
+ Requires-Dist: requests (>=2.32.3,<3.0.0)
36
+ Requires-Dist: rtree (>=1.3.0,<2.0.0)
37
+ Requires-Dist: scipy (>=1.14.1,<2.0.0)
38
+ Requires-Dist: tesserocr (>=2.7.1,<3.0.0) ; extra == "tesserocr"
39
+ Requires-Dist: torch (>=2.2.2,<2.3.0) ; sys_platform == "darwin" and platform_machine == "x86_64"
40
+ Requires-Dist: torch (>=2.2.2,<3.0.0) ; sys_platform != "darwin" or platform_machine != "x86_64"
41
+ Requires-Dist: torchvision (>=0,<1) ; sys_platform != "darwin" or platform_machine != "x86_64"
42
+ Requires-Dist: torchvision (>=0.17.2,<0.18.0) ; sys_platform == "darwin" and platform_machine == "x86_64"
43
+ Requires-Dist: typer (>=0.12.5,<0.13.0)
44
+ Project-URL: Repository, https://github.com/DS4SD/docling
45
+ Description-Content-Type: text/markdown
46
+
47
+ <p align="center">
48
+ <a href="https://github.com/ds4sd/docling">
49
+ <img loading="lazy" alt="Docling" src="https://github.com/DS4SD/docling/raw/main/logo.png" width="150" />
50
+ </a>
51
+ </p>
52
+
53
+ # Docling
54
+
55
+ [![arXiv](https://img.shields.io/badge/arXiv-2408.09869-b31b1b.svg)](https://arxiv.org/abs/2408.09869)
56
+ [![PyPI version](https://img.shields.io/pypi/v/docling)](https://pypi.org/project/docling/)
57
+ ![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue)
58
+ [![Poetry](https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json)](https://python-poetry.org/)
59
+ [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
60
+ [![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)
61
+ [![Pydantic v2](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/pydantic/pydantic/main/docs/badge/v2.json)](https://pydantic.dev)
62
+ [![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)
63
+ [![License MIT](https://img.shields.io/github/license/DS4SD/docling)](https://opensource.org/licenses/MIT)
64
+
65
+ Docling bundles PDF document conversion to JSON and Markdown in an easy, self-contained package.
66
+
67
+ ## Features
68
+ * ⚡ Converts any PDF document to JSON or Markdown format, stable and lightning fast
69
+ * 📑 Understands detailed page layout, reading order and recovers table structures
70
+ * 📝 Extracts metadata from the document, such as title, authors, references and language
71
+ * 🔍 Includes OCR support for scanned PDFs
72
+ * 🤖 Integrates easily with LLM app / RAG frameworks like 🦙 LlamaIndex and 🦜🔗 LangChain
73
+ * 💻 Provides a simple and convenient CLI
74
+
75
+ ## Installation
76
+
77
+ To use Docling, simply install `docling` from your package manager, e.g. pip:
78
+ ```bash
79
+ pip install docling
80
+ ```
81
+
82
+ Works on macOS, Linux and Windows environments. Both x86_64 and arm64 architectures.
83
+
84
+ <details>
85
+ <summary><b>Alternative PyTorch distributions</b></summary>
86
+
87
+ The Docling models depend on the [PyTorch](https://pytorch.org/) library.
88
+ Depending on your architecture, you might want to use a different distribution of `torch`.
89
+ For example, you might want support for different accelerator or for a cpu-only version.
90
+ All the different ways for installing `torch` are listed on their website <https://pytorch.org/>.
91
+
92
+ One common situation is the installation on Linux systems with cpu-only support.
93
+ In this case, we suggest the installation of Docling with the following options
94
+
95
+ ```bash
96
+ # Example for installing on the Linux cpu-only version
97
+ pip install docling --extra-index-url https://download.pytorch.org/whl/cpu
98
+ ```
99
+ </details>
100
+
101
+ <details>
102
+ <summary><b>Alternative OCR engines</b></summary>
103
+
104
+ Docling supports multiple OCR engines for processing scanned documents. The current version provides
105
+ the following engines.
106
+
107
+ | Engine | Installation | Usage |
108
+ | ------ | ------------ | ----- |
109
+ | [EasyOCR](https://github.com/JaidedAI/EasyOCR) | Default in Docling or via `pip install easyocr`. | `EasyOcrOptions` |
110
+ | Tesseract | System dependency. See description for Tesseract and Tesserocr below. | `TesseractOcrOptions` |
111
+ | Tesseract CLI | System dependency. See description below. | `TesseractCliOcrOptions` |
112
+
113
+ The Docling `DocumentConverter` allows to choose the OCR engine with the `ocr_options` settings. For example
114
+
115
+ ```python
116
+ from docling.datamodel.base_models import ConversionStatus, PipelineOptions
117
+ from docling.datamodel.pipeline_options import PipelineOptions, EasyOcrOptions, TesseractOcrOptions
118
+ from docling.document_converter import DocumentConverter
119
+
120
+ pipeline_options = PipelineOptions()
121
+ pipeline_options.do_ocr = True
122
+ pipeline_options.ocr_options = TesseractOcrOptions() # Use Tesseract
123
+
124
+ doc_converter = DocumentConverter(
125
+ pipeline_options=pipeline_options,
126
+ )
127
+ ```
128
+
129
+ #### Tesseract installation
130
+
131
+ [Tesseract](https://github.com/tesseract-ocr/tesseract) is a popular OCR engine which is available
132
+ on most operating systems. For using this engine with Docling, Tesseract must be installed on your
133
+ system, using the packaging tool of your choice. Below we provide example commands.
134
+ After installing Tesseract you are expected to provide the path to its language files using the
135
+ `TESSDATA_PREFIX` environment variable (note that it must terminate with a slash `/`).
136
+
137
+ For macOS, we reccomend using [Homebrew](https://brew.sh/).
138
+
139
+ ```console
140
+ brew install tesseract leptonica pkg-config
141
+ TESSDATA_PREFIX=/opt/homebrew/share/tessdata/
142
+ echo "Set TESSDATA_PREFIX=${TESSDATA_PREFIX}"
143
+ ```
144
+
145
+ For Debian-based systems.
146
+
147
+ ```console
148
+ apt-get install tesseract-ocr tesseract-ocr-eng libtesseract-dev libleptonica-dev pkg-config
149
+ TESSDATA_PREFIX=$(dpkg -L tesseract-ocr-eng | grep tessdata$)
150
+ echo "Set TESSDATA_PREFIX=${TESSDATA_PREFIX}"
151
+ ```
152
+
153
+ For RHEL systems.
154
+
155
+ ```console
156
+ dnf install tesseract tesseract-devel tesseract-langpack-eng leptonica-devel
157
+ TESSDATA_PREFIX=/usr/share/tesseract/tessdata/
158
+ echo "Set TESSDATA_PREFIX=${TESSDATA_PREFIX}"
159
+ ```
160
+
161
+ #### Linking to Tesseract
162
+ The most efficient usage of the Tesseract library is via linking. Docling is using
163
+ the [Tesserocr](https://github.com/sirfz/tesserocr) package for this.
164
+
165
+ If you get into installation issues of Tesserocr, we suggest using the following
166
+ installation options:
167
+
168
+ ```console
169
+ pip uninstall tesserocr
170
+ pip install --no-binary :all: tesserocr
171
+ ```
172
+ </details>
173
+
174
+ <details>
175
+ <summary><b>Docling development setup</b></summary>
176
+
177
+ To develop for Docling (features, bugfixes etc.), install as follows from your local clone's root dir:
178
+ ```bash
179
+ poetry install --all-extras
180
+ ```
181
+ </details>
182
+
183
+ ## Getting started
184
+
185
+ ### Convert a single document
186
+
187
+ To convert invidual PDF documents, use `convert_single()`, for example:
188
+ ```python
189
+ from docling.document_converter import DocumentConverter
190
+
191
+ source = "https://arxiv.org/pdf/2408.09869" # PDF path or URL
192
+ converter = DocumentConverter()
193
+ result = converter.convert_single(source)
194
+ print(result.render_as_markdown()) # output: "## Docling Technical Report[...]"
195
+ print(result.render_as_doctags()) # output: "<document><title><page_1><loc_20>..."
196
+ ```
197
+
198
+ ### Convert a batch of documents
199
+
200
+ For an example of batch-converting documents, see [batch_convert.py](https://github.com/DS4SD/docling/blob/main/examples/batch_convert.py).
201
+
202
+ From a local repo clone, you can run it with:
203
+
204
+ ```
205
+ python examples/batch_convert.py
206
+ ```
207
+ The output of the above command will be written to `./scratch`.
208
+
209
+ ### CLI
210
+
211
+ You can also use Docling directly from your command line to convert individual files —be it local or by URL— or whole directories.
212
+
213
+ A simple example would look like this:
214
+ ```console
215
+ docling https://arxiv.org/pdf/2206.01062
216
+ ```
217
+
218
+ To see all available options (export formats etc.) run `docling --help`.
219
+
220
+ <details>
221
+ <summary><b>CLI reference</b></summary>
222
+
223
+ Here are the available options as of this writing (for an up-to-date listing, run `docling --help`):
224
+
225
+ ```console
226
+ $ docling --help
227
+
228
+ Usage: docling [OPTIONS] source
229
+
230
+ ╭─ Arguments ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
231
+ │ * input_sources source PDF files to convert. Can be local file / directory paths or URL. [default: None] [required] │
232
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
233
+ ╭─ Options ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
234
+ │ --json --no-json If enabled the document is exported as JSON. [default: no-json] │
235
+ │ --md --no-md If enabled the document is exported as Markdown. [default: md] │
236
+ │ --txt --no-txt If enabled the document is exported as Text. [default: no-txt] │
237
+ │ --doctags --no-doctags If enabled the document is exported as Doc Tags. [default: no-doctags] │
238
+ │ --ocr --no-ocr If enabled, the bitmap content will be processed using OCR. [default: ocr] │
239
+ │ --backend [pypdfium2|docling] The PDF backend to use. [default: docling] │
240
+ │ --output PATH Output directory where results are saved. [default: .] │
241
+ │ --version Show version information. │
242
+ │ --help Show this message and exit. │
243
+ ╰──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
244
+ ```
245
+ </details>
246
+
247
+ ### RAG
248
+ Check out the following examples showcasing RAG using Docling with standard LLM application frameworks:
249
+ - [Basic RAG pipeline with 🦙 LlamaIndex](https://github.com/DS4SD/docling/tree/main/examples/rag_llamaindex.ipynb)
250
+ - [Basic RAG pipeline with 🦜🔗 LangChain](https://github.com/DS4SD/docling/tree/main/examples/rag_langchain.ipynb)
251
+
252
+ ## Advanced features
253
+
254
+ ### Adjust pipeline features
255
+
256
+ The example file [custom_convert.py](https://github.com/DS4SD/docling/blob/main/examples/custom_convert.py) contains multiple ways
257
+ one can adjust the conversion pipeline and features.
258
+
259
+
260
+ #### Control pipeline options
261
+
262
+ You can control if table structure recognition or OCR should be performed by arguments passed to `DocumentConverter`:
263
+ ```python
264
+ doc_converter = DocumentConverter(
265
+ artifacts_path=artifacts_path,
266
+ pipeline_options=PipelineOptions(
267
+ do_table_structure=False, # controls if table structure is recovered
268
+ do_ocr=True, # controls if OCR is applied (ignores programmatic content)
269
+ ),
270
+ )
271
+ ```
272
+
273
+ #### Control table extraction options
274
+
275
+ You can control if table structure recognition should map the recognized structure back to PDF cells (default) or use text cells from the structure prediction itself.
276
+ This can improve output quality if you find that multiple columns in extracted tables are erroneously merged into one.
277
+
278
+
279
+ ```python
280
+ from docling.datamodel.pipeline_options import PipelineOptions
281
+
282
+ pipeline_options = PipelineOptions(do_table_structure=True)
283
+ pipeline_options.table_structure_options.do_cell_matching = False # uses text cells predicted from table structure model
284
+
285
+ doc_converter = DocumentConverter(
286
+ artifacts_path=artifacts_path,
287
+ pipeline_options=pipeline_options,
288
+ )
289
+ ```
290
+
291
+ Since docling 1.16.0: You can control which TableFormer mode you want to use. Choose between `TableFormerMode.FAST` (default) and `TableFormerMode.ACCURATE` (better, but slower) to receive better quality with difficult table structures.
292
+
293
+ ```python
294
+ from docling.datamodel.pipeline_options import PipelineOptions, TableFormerMode
295
+
296
+ pipeline_options = PipelineOptions(do_table_structure=True)
297
+ pipeline_options.table_structure_options.mode = TableFormerMode.ACCURATE # use more accurate TableFormer model
298
+
299
+ doc_converter = DocumentConverter(
300
+ artifacts_path=artifacts_path,
301
+ pipeline_options=pipeline_options,
302
+ )
303
+ ```
304
+
305
+ ### Impose limits on the document size
306
+
307
+ You can limit the file size and number of pages which should be allowed to process per document:
308
+ ```python
309
+ conv_input = DocumentConversionInput.from_paths(
310
+ paths=[Path("./test/data/2206.01062.pdf")],
311
+ limits=DocumentLimits(max_num_pages=100, max_file_size=20971520)
312
+ )
313
+ ```
314
+
315
+ ### Convert from binary PDF streams
316
+
317
+ You can convert PDFs from a binary stream instead of from the filesystem as follows:
318
+ ```python
319
+ buf = BytesIO(your_binary_stream)
320
+ docs = [DocumentStream(filename="my_doc.pdf", stream=buf)]
321
+ conv_input = DocumentConversionInput.from_streams(docs)
322
+ results = doc_converter.convert(conv_input)
323
+ ```
324
+ ### Limit resource usage
325
+
326
+ You can limit the CPU threads used by Docling by setting the environment variable `OMP_NUM_THREADS` accordingly. The default setting is using 4 CPU threads.
327
+
328
+ ### Chunking
329
+
330
+ You can perform a hierarchy-aware chunking of a Docling document as follows:
331
+
332
+ ```python
333
+ from docling.document_converter import DocumentConverter
334
+ from docling_core.transforms.chunker import HierarchicalChunker
335
+
336
+ doc = DocumentConverter().convert_single("https://arxiv.org/pdf/2206.01062").output
337
+ chunks = list(HierarchicalChunker().chunk(doc))
338
+ # > [
339
+ # > ChunkWithMetadata(
340
+ # > path='$.main-text[0]',
341
+ # > text='DocLayNet: A Large Human-Annotated Dataset [...]',
342
+ # > page=1,
343
+ # > bbox=[107.30, 672.38, 505.19, 709.08]
344
+ # > ),
345
+ # > [...]
346
+ # > ]
347
+ ```
348
+
349
+
350
+ ## Technical report
351
+
352
+ For more details on Docling's inner workings, check out the [Docling Technical Report](https://arxiv.org/abs/2408.09869).
353
+
354
+ ## Contributing
355
+
356
+ Please read [Contributing to Docling](https://github.com/DS4SD/docling/blob/main/CONTRIBUTING.md) for details.
357
+
358
+
359
+ ## References
360
+
361
+ If you use Docling in your projects, please consider citing the following:
362
+
363
+ ```bib
364
+ @techreport{Docling,
365
+ author = {Deep Search Team},
366
+ month = {8},
367
+ title = {Docling Technical Report},
368
+ url = {https://arxiv.org/abs/2408.09869},
369
+ eprint = {2408.09869},
370
+ doi = {10.48550/arXiv.2408.09869},
371
+ version = {1.0.0},
372
+ year = {2024}
373
+ }
374
+ ```
375
+
376
+ ## License
377
+
378
+ The Docling codebase is under MIT license.
379
+ For individual model usage, please refer to the model licenses found in the original packages.
380
+
@@ -0,0 +1,34 @@
1
+ docling/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
2
+ docling/backend/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
3
+ docling/backend/abstract_backend.py,sha256=clJtGxLedpLriEhpx7oyxjmlwMLPorkv-1tdfZm9GdA,1546
4
+ docling/backend/docling_parse_backend.py,sha256=RUWWZbx2cUotZeeTkc-Lbg2k8MVFXFxaDjM4sPfaFZE,7475
5
+ docling/backend/pypdfium2_backend.py,sha256=bIIImVM73wmcVcKMqjl4JF8CD-Qj2W5rZbI4G7clU4s,8877
6
+ docling/cli/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
7
+ docling/cli/main.py,sha256=Q_5HTL2O20bMlF-U8Ac8ev4iqgLPyrwlHEyLAq6rezg,7913
8
+ docling/datamodel/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
9
+ docling/datamodel/base_models.py,sha256=ihBfPvVhxnEGmrzBMma_2RlOyObpaBAb7O5oBFBlIks,8833
10
+ docling/datamodel/document.py,sha256=8iHylLaty-ZVYGzYmP61dZPtduzEyiRM79Tx3gS9UEU,16195
11
+ docling/datamodel/pipeline_options.py,sha256=KTjos_AheIUDlMm4xsV9V1cvM5cI3ZG43z-WXAP_FXU,1931
12
+ docling/datamodel/settings.py,sha256=t5g6wrEJnPa9gBzMMl8ppgBRUYz-8xgopEtfMS0ZH28,733
13
+ docling/document_converter.py,sha256=V0Q5hCRyTbmavhQBN48biBttxpvPyoAPmFRhSePv2vE,11149
14
+ docling/models/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
15
+ docling/models/base_ocr_model.py,sha256=a89dLaT2vV5lTSNeetlCY9OnUd4LP4My_ZRky_fpVgo,4530
16
+ docling/models/ds_glm_model.py,sha256=VXGmj8cW0WKMz1He4tp1lZhXHkS8Z39U1G-ujkc7deU,3368
17
+ docling/models/easyocr_model.py,sha256=fmfTvOfqo97n_xYQDPRMKlvMOs9QqgLgSTqwBDrjK_8,3010
18
+ docling/models/layout_model.py,sha256=5wCohyzGK1p5F5cTRY5QWbW2AI4eevXobJDJdSLhX7k,11272
19
+ docling/models/page_assemble_model.py,sha256=8eoG2WiFxPxq9TPvM-wkngb2gkr0tdtCRVXg1JcTETo,5550
20
+ docling/models/table_structure_model.py,sha256=iHJjWdKCpTcH3l_ElMWnC5pt6tkUpIuByed304Fdq9w,6009
21
+ docling/models/tesseract_ocr_cli_model.py,sha256=s2uzcjTPqbfqPCgtz8nvfK8QVtuvQBR8OpTR3H_XdR8,5504
22
+ docling/models/tesseract_ocr_model.py,sha256=PqQv1Hv3GC1FByjegWvjNFw15Jcw-mT25_MvFr3hAHQ,4575
23
+ docling/pipeline/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
24
+ docling/pipeline/base_model_pipeline.py,sha256=rrMog3EuiR5Gx9OWtfMj24rQvHCrWkxZ3g9OIr7LPSQ,607
25
+ docling/pipeline/standard_model_pipeline.py,sha256=SVyMYmP1YLzYqDqagjn-pWugnjS7bp9YbmlV-VcenpI,2589
26
+ docling/utils/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
27
+ docling/utils/export.py,sha256=bKLdbeUcR-rQsGPV1IqJkCHKMCv7X2QOHyxmjNuH3HE,4655
28
+ docling/utils/layout_utils.py,sha256=FOFbL0hKzUoWXdZaeUvEtFqKv0IkPifIr4sdGW4suKs,31804
29
+ docling/utils/utils.py,sha256=llhXSbIDNZ1MHOwBEfLHBAoJIAYI7QlPIonlI1jLUJ0,1208
30
+ docling-1.19.0.dist-info/LICENSE,sha256=mBb7ErEcM8VS9OhiGHnQ2kk75HwPhr54W1Oiz3965MY,1088
31
+ docling-1.19.0.dist-info/METADATA,sha256=ZJ0aAmgxVDYaEKkBnm8yexI4zIcG2iqGC-NddLdabmM,16795
32
+ docling-1.19.0.dist-info/WHEEL,sha256=sP946D7jFCHeNz5Iq4fL4Lu-PrWrFsgfLXbbkciIZwg,88
33
+ docling-1.19.0.dist-info/entry_points.txt,sha256=VOSzV77znM52dz5ysaDuJ0ijl1cnfrh1ZPg8od5OcTs,48
34
+ docling-1.19.0.dist-info/RECORD,,
@@ -0,0 +1,3 @@
1
+ [console_scripts]
2
+ docling=docling.cli.main:app
3
+
@@ -1,192 +0,0 @@
1
- Metadata-Version: 2.1
2
- Name: docling
3
- Version: 1.6.2
4
- Summary: Docling PDF conversion package
5
- Home-page: https://github.com/DS4SD/docling
6
- License: MIT
7
- Keywords: docling,convert,document,pdf,layout model,segmentation,table structure,table former
8
- Author: Christoph Auer
9
- Author-email: cau@zurich.ibm.com
10
- Requires-Python: >=3.10,<4.0
11
- Classifier: Development Status :: 5 - Production/Stable
12
- Classifier: Intended Audience :: Developers
13
- Classifier: Intended Audience :: Science/Research
14
- Classifier: License :: OSI Approved :: MIT License
15
- Classifier: Operating System :: MacOS :: MacOS X
16
- Classifier: Operating System :: POSIX :: Linux
17
- Classifier: Programming Language :: Python :: 3
18
- Classifier: Programming Language :: Python :: 3.10
19
- Classifier: Programming Language :: Python :: 3.11
20
- Classifier: Programming Language :: Python :: 3.12
21
- Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
22
- Requires-Dist: certifi (>=2024.7.4)
23
- Requires-Dist: deepsearch-glm (>=0.19.0,<1)
24
- Requires-Dist: docling-core (>=1.1.2,<2.0.0)
25
- Requires-Dist: docling-ibm-models (>=1.1.2,<2.0.0)
26
- Requires-Dist: docling-parse (>=0.2.0,<0.3.0)
27
- Requires-Dist: easyocr (>=1.7,<2.0)
28
- Requires-Dist: filetype (>=1.2.0,<2.0.0)
29
- Requires-Dist: huggingface_hub (>=0.23,<1)
30
- Requires-Dist: pydantic (>=2.0.0,<3.0.0)
31
- Requires-Dist: pydantic-settings (>=2.3.0,<3.0.0)
32
- Requires-Dist: pypdfium2 (>=4.30.0,<5.0.0)
33
- Requires-Dist: requests (>=2.32.3,<3.0.0)
34
- Requires-Dist: rtree (>=1.3.0,<2.0.0)
35
- Requires-Dist: scipy (>=1.14.1,<2.0.0)
36
- Project-URL: Repository, https://github.com/DS4SD/docling
37
- Description-Content-Type: text/markdown
38
-
39
- <p align="center">
40
- <a href="https://github.com/ds4sd/docling">
41
- <img loading="lazy" alt="Docling" src="https://github.com/DS4SD/docling/raw/main/logo.png" width="150" />
42
- </a>
43
- </p>
44
-
45
- # Docling
46
-
47
- [![arXiv](https://img.shields.io/badge/arXiv-2408.09869-b31b1b.svg)](https://arxiv.org/abs/2408.09869)
48
- [![PyPI version](https://img.shields.io/pypi/v/docling)](https://pypi.org/project/docling/)
49
- ![Python](https://img.shields.io/badge/python-3.10%20%7C%203.11%20%7C%203.12-blue)
50
- [![Poetry](https://img.shields.io/endpoint?url=https://python-poetry.org/badge/v0.json)](https://python-poetry.org/)
51
- [![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
52
- [![Imports: isort](https://img.shields.io/badge/%20imports-isort-%231674b1?style=flat&labelColor=ef8336)](https://pycqa.github.io/isort/)
53
- [![Pydantic v2](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/pydantic/pydantic/main/docs/badge/v2.json)](https://pydantic.dev)
54
- [![pre-commit](https://img.shields.io/badge/pre--commit-enabled-brightgreen?logo=pre-commit&logoColor=white)](https://github.com/pre-commit/pre-commit)
55
- [![License MIT](https://img.shields.io/github/license/DS4SD/docling)](https://opensource.org/licenses/MIT)
56
-
57
- Docling bundles PDF document conversion to JSON and Markdown in an easy, self-contained package.
58
-
59
- ## Features
60
- * ⚡ Converts any PDF document to JSON or Markdown format, stable and lightning fast
61
- * 📑 Understands detailed page layout, reading order and recovers table structures
62
- * 📝 Extracts metadata from the document, such as title, authors, references and language
63
- * 🔍 Optionally applies OCR (use with scanned PDFs)
64
-
65
- ## Installation
66
-
67
- To use Docling, simply install `docling` from your package manager, e.g. pip:
68
- ```bash
69
- pip install docling
70
- ```
71
-
72
- > [!NOTE]
73
- > Works on macOS and Linux environments. Windows platforms are currently not tested.
74
-
75
- ### Development setup
76
-
77
- To develop for Docling, you need Python 3.10 / 3.11 / 3.12 and Poetry. You can then install from your local clone's root dir:
78
- ```bash
79
- poetry install --all-extras
80
- ```
81
-
82
- ## Usage
83
-
84
- ### Convert a single document
85
-
86
- To convert invidual PDF documents, use `convert_single()`, for example:
87
- ```python
88
- from docling.document_converter import DocumentConverter
89
-
90
- source = "https://arxiv.org/pdf/2206.01062" # PDF path or URL
91
- converter = DocumentConverter()
92
- doc = converter.convert_single(source)
93
- print(doc.export_to_markdown()) # output: "## DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis [...]"
94
- ```
95
-
96
- ### Convert a batch of documents
97
-
98
- For an example of batch-converting documents, see [batch_convert.py](https://github.com/DS4SD/docling/blob/main/examples/batch_convert.py).
99
-
100
- From a local repo clone, you can run it with:
101
-
102
- ```
103
- python examples/batch_convert.py
104
- ```
105
- The output of the above command will be written to `./scratch`.
106
-
107
- ### Adjust pipeline features
108
-
109
- The example file [custom_convert.py](https://github.com/DS4SD/docling/blob/main/examples/custom_convert.py) contains multiple ways
110
- one can adjust the conversion pipeline and features.
111
-
112
-
113
- #### Control pipeline options
114
-
115
- You can control if table structure recognition or OCR should be performed by arguments passed to `DocumentConverter`:
116
- ```python
117
- doc_converter = DocumentConverter(
118
- artifacts_path=artifacts_path,
119
- pipeline_options=PipelineOptions(
120
- do_table_structure=False, # controls if table structure is recovered
121
- do_ocr=True, # controls if OCR is applied (ignores programmatic content)
122
- ),
123
- )
124
- ```
125
-
126
- #### Control table extraction options
127
-
128
- You can control if table structure recognition should map the recognized structure back to PDF cells (default) or use text cells from the structure prediction itself.
129
- This can improve output quality if you find that multiple columns in extracted tables are erroneously merged into one.
130
-
131
-
132
- ```python
133
- pipeline_options = PipelineOptions(do_table_structure=True)
134
- pipeline_options.table_structure_options.do_cell_matching = False # uses text cells predicted from table structure model
135
-
136
- doc_converter = DocumentConverter(
137
- artifacts_path=artifacts_path,
138
- pipeline_options=pipeline_options,
139
- )
140
- ```
141
-
142
- ### Impose limits on the document size
143
-
144
- You can limit the file size and number of pages which should be allowed to process per document:
145
- ```python
146
- conv_input = DocumentConversionInput.from_paths(
147
- paths=[Path("./test/data/2206.01062.pdf")],
148
- limits=DocumentLimits(max_num_pages=100, max_file_size=20971520)
149
- )
150
- ```
151
-
152
- ### Convert from binary PDF streams
153
-
154
- You can convert PDFs from a binary stream instead of from the filesystem as follows:
155
- ```python
156
- buf = BytesIO(your_binary_stream)
157
- docs = [DocumentStream(filename="my_doc.pdf", stream=buf)]
158
- conv_input = DocumentConversionInput.from_streams(docs)
159
- converted_docs = doc_converter.convert(conv_input)
160
- ```
161
- ### Limit resource usage
162
-
163
- You can limit the CPU threads used by Docling by setting the environment variable `OMP_NUM_THREADS` accordingly. The default setting is using 4 CPU threads.
164
-
165
-
166
- ## Contributing
167
-
168
- Please read [Contributing to Docling](https://github.com/DS4SD/docling/blob/main/CONTRIBUTING.md) for details.
169
-
170
-
171
- ## References
172
-
173
- If you use Docling in your projects, please consider citing the following:
174
-
175
- ```bib
176
- @techreport{Docling,
177
- author = {Deep Search Team},
178
- month = {8},
179
- title = {{Docling Technical Report}},
180
- url={https://arxiv.org/abs/2408.09869},
181
- eprint={2408.09869},
182
- doi = "10.48550/arXiv.2408.09869",
183
- version = {1.0.0},
184
- year = {2024}
185
- }
186
- ```
187
-
188
- ## License
189
-
190
- The Docling codebase is under MIT license.
191
- For individual model usage, please refer to the model licenses found in the original packages.
192
-
@@ -1,27 +0,0 @@
1
- docling/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
2
- docling/backend/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
3
- docling/backend/abstract_backend.py,sha256=ZfEHaBPGM1cmqrhaEoU3MHhnHU11NhOnhtFEIbVMYDo,1221
4
- docling/backend/docling_parse_backend.py,sha256=TN7Ln3Lkc8k0v6HzxA2iUGc8f2iqMw0I-3eryLQkpdw,6924
5
- docling/backend/pypdfium2_backend.py,sha256=xUiIYgd7i22YDx4-W2hfPUaQFszW0gcT6pavG5qZ8LE,8062
6
- docling/datamodel/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
7
- docling/datamodel/base_models.py,sha256=5VHit5h7OleKnbhvy-sWDxQLizEdNrGUBrypyzwHyAE,8604
8
- docling/datamodel/document.py,sha256=Dgi9pSwXCgIoR26MKiRDiVMyMaFKdvGSKq2Fm5Lef9M,13173
9
- docling/datamodel/settings.py,sha256=t5g6wrEJnPa9gBzMMl8ppgBRUYz-8xgopEtfMS0ZH28,733
10
- docling/document_converter.py,sha256=UFSELvUSWsr8s0VByu4lNuzu7bn7zZauJTL3FTSLSBg,10371
11
- docling/models/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
12
- docling/models/base_ocr_model.py,sha256=Ipl82a3AV2OsgMQSMEMpnWJ6MXcmyIQzmp52PmTaB0g,4465
13
- docling/models/ds_glm_model.py,sha256=wmb--2JKFQby-kvidw6PyM8wURPXYPQ_Z_eKKCBAdYQ,3192
14
- docling/models/easyocr_model.py,sha256=ABIqALvtNNrDQ47fXaZ0lDFhOwKsYGUUlAPnIsFZgZA,2232
15
- docling/models/layout_model.py,sha256=ZFmaLXlRWUfsT1pJCiYVxhQFrBBsiz6Aw0m9GM3UvVM,11249
16
- docling/models/page_assemble_model.py,sha256=8eoG2WiFxPxq9TPvM-wkngb2gkr0tdtCRVXg1JcTETo,5550
17
- docling/models/table_structure_model.py,sha256=5jzTlpM-GdCSq4l0vD1W6aSPTJXeTcXEnNuPxnw-DlA,5437
18
- docling/pipeline/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
19
- docling/pipeline/base_model_pipeline.py,sha256=AC5NTR0xLy5JIZqsTINkKEHeCPqpyvJpuE_bcnZhyvI,529
20
- docling/pipeline/standard_model_pipeline.py,sha256=UTjyaEXvz9htYZz-IMTkn11cZwNjgvo_Fl2dfBVnRQs,1442
21
- docling/utils/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
22
- docling/utils/layout_utils.py,sha256=FOFbL0hKzUoWXdZaeUvEtFqKv0IkPifIr4sdGW4suKs,31804
23
- docling/utils/utils.py,sha256=llhXSbIDNZ1MHOwBEfLHBAoJIAYI7QlPIonlI1jLUJ0,1208
24
- docling-1.6.2.dist-info/LICENSE,sha256=ACwmltkrXIz5VsEQcrqljq-fat6ZXAMepjXGoe40KtE,1069
25
- docling-1.6.2.dist-info/METADATA,sha256=JF_IcxQ0hSuqcEhHU8qf5UlB3udVHRmRWiozPkLAcP4,7229
26
- docling-1.6.2.dist-info/WHEEL,sha256=sP946D7jFCHeNz5Iq4fL4Lu-PrWrFsgfLXbbkciIZwg,88
27
- docling-1.6.2.dist-info/RECORD,,