table-proof-py 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- table_proof_py-0.1.0/.gitignore +18 -0
- table_proof_py-0.1.0/CHANGELOG.md +13 -0
- table_proof_py-0.1.0/CONTRIBUTING.md +52 -0
- table_proof_py-0.1.0/LICENSE +21 -0
- table_proof_py-0.1.0/PKG-INFO +177 -0
- table_proof_py-0.1.0/PUBLISHING.md +85 -0
- table_proof_py-0.1.0/README.md +140 -0
- table_proof_py-0.1.0/THIRD_PARTY.md +13 -0
- table_proof_py-0.1.0/benchmarks/DOCLING_LICENSE +21 -0
- table_proof_py-0.1.0/benchmarks/README.md +172 -0
- table_proof_py-0.1.0/benchmarks/THIRD_PARTY.md +32 -0
- table_proof_py-0.1.0/benchmarks/__init__.py +1 -0
- table_proof_py-0.1.0/benchmarks/common.py +116 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/gold.json +86 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/page6-7.24.2.json +5253 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/page6-7.24.2.md +34 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/page6-7.24.2.runtime.json +50 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/page6.pdf +0 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/parse-7.24.2.json +5253 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/parse-7.24.2.md +34 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/source.json +29 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/source.pdf +0 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/table2-7.20.0.json +2658 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/table2-7.20.0.manifest.json +71 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/table2-7.20.0.md +9 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/table2-7.20.0.runtime.json +50 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/table2-7.24.2.json +2658 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/table2-7.24.2.manifest.json +111 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/table2-7.24.2.md +9 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/table2-7.24.2.runtime.json +50 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5136/table2.pdf +0 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5198/affected.json +471 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5198/affected.manifest.json +61 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5198/affected.md +6 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5198/applied_fix.json +9 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5198/fixed.json +471 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5198/fixed.manifest.json +53 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5198/fixed.md +6 -0
- table_proof_py-0.1.0/benchmarks/fixtures/docling_5198/source.pdf +105 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/clean-empty-cell.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/clean-empty-cell.manifest.json +19 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/clean-empty-cell.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/clean-repeated-value.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/clean-repeated-value.manifest.json +19 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/clean-repeated-value.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/clean.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/clean.manifest.json +19 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/clean.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/column-value-swap.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/column-value-swap.manifest.json +36 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/column-value-swap.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/decimal-loss.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/decimal-loss.manifest.json +28 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/decimal-loss.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/empty-cell.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/empty-cell.manifest.json +28 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/empty-cell.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/header-swap.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/header-swap.manifest.json +36 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/header-swap.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/missing-geometry.json +165 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/missing-geometry.manifest.json +19 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/missing-geometry.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/missing-table.json +17 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/missing-table.manifest.json +24 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/missing-table.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/row-label-swap.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/row-label-swap.manifest.json +36 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/row-label-swap.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/row-value-swap.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/row-value-swap.manifest.json +36 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/row-value-swap.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/scan-only.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/scan-only.manifest.json +19 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/scan-only.pdf +85 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/spanning-header.json +238 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/spanning-header.manifest.json +19 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/spanning-header.pdf +93 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/unit-change.json +256 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/unit-change.manifest.json +28 -0
- table_proof_py-0.1.0/benchmarks/fixtures/synthetic/unit-change.pdf +95 -0
- table_proof_py-0.1.0/benchmarks/make_synthetic.py +124 -0
- table_proof_py-0.1.0/benchmarks/patch_docling.py +67 -0
- table_proof_py-0.1.0/benchmarks/prepare_public_pdf.py +83 -0
- table_proof_py-0.1.0/benchmarks/reproduce_docling.py +134 -0
- table_proof_py-0.1.0/benchmarks/reproduce_public_pdf.py +185 -0
- table_proof_py-0.1.0/benchmarks/requirements-extraction-lock.txt +103 -0
- table_proof_py-0.1.0/benchmarks/requirements-lock.txt +52 -0
- table_proof_py-0.1.0/benchmarks/results.json +2860 -0
- table_proof_py-0.1.0/benchmarks/results.md +32 -0
- table_proof_py-0.1.0/benchmarks/run.py +348 -0
- table_proof_py-0.1.0/docs/API.md +73 -0
- table_proof_py-0.1.0/docs/ARCHITECTURE.md +79 -0
- table_proof_py-0.1.0/docs/DATASETS.md +85 -0
- table_proof_py-0.1.0/docs/MVP_PLAN.md +82 -0
- table_proof_py-0.1.0/docs/VISION.md +174 -0
- table_proof_py-0.1.0/docs/assets/vision-docling-5136.json +386 -0
- table_proof_py-0.1.0/docs/assets/vision-smoke.json +164 -0
- table_proof_py-0.1.0/pyproject.toml +97 -0
- table_proof_py-0.1.0/scripts/examine_distribution.py +197 -0
- table_proof_py-0.1.0/scripts/vision_smoke.py +113 -0
- table_proof_py-0.1.0/scripts/wheel_smoke.py +14 -0
- table_proof_py-0.1.0/spec/report-v1.schema.json +225 -0
- table_proof_py-0.1.0/src/table_proof/__init__.py +24 -0
- table_proof_py-0.1.0/src/table_proof/__main__.py +3 -0
- table_proof_py-0.1.0/src/table_proof/api.py +65 -0
- table_proof_py-0.1.0/src/table_proof/auditor.py +521 -0
- table_proof_py-0.1.0/src/table_proof/cli.py +68 -0
- table_proof_py-0.1.0/src/table_proof/docling.py +168 -0
- table_proof_py-0.1.0/src/table_proof/models.py +111 -0
- table_proof_py-0.1.0/src/table_proof/pdf.py +28 -0
- table_proof_py-0.1.0/src/table_proof/py.typed +0 -0
- table_proof_py-0.1.0/src/table_proof/report.py +217 -0
- table_proof_py-0.1.0/src/table_proof/vision.py +381 -0
- table_proof_py-0.1.0/tests/conftest.py +63 -0
- table_proof_py-0.1.0/tests/test_api_cli_report.py +136 -0
- table_proof_py-0.1.0/tests/test_auditor.py +504 -0
- table_proof_py-0.1.0/tests/test_benchmark.py +110 -0
- table_proof_py-0.1.0/tests/test_distribution.py +103 -0
- table_proof_py-0.1.0/tests/test_docling.py +111 -0
- table_proof_py-0.1.0/tests/test_pdf.py +35 -0
- table_proof_py-0.1.0/tests/test_properties.py +95 -0
- table_proof_py-0.1.0/tests/test_report_schema.py +17 -0
- table_proof_py-0.1.0/tests/test_review_cases.py +43 -0
- table_proof_py-0.1.0/tests/test_vision.py +240 -0
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.1.0
|
|
4
|
+
|
|
5
|
+
- Add source-PDF audits for extracted table cells.
|
|
6
|
+
- Add Docling JSON and object adapters with rich-reference resolution.
|
|
7
|
+
- Add JSON and HTML evidence reports.
|
|
8
|
+
- Add an optional local Table Transformer path.
|
|
9
|
+
- Add unit, property, integration, and distribution tests.
|
|
10
|
+
- Add pinned regression fixtures and a comparative benchmark.
|
|
11
|
+
- Add the MVP plan and dataset acceptance criteria.
|
|
12
|
+
- Add a PyPI release workflow and archive content examination.
|
|
13
|
+
- Use the `table-proof-py` distribution name on PyPI.
|
|
@@ -0,0 +1,52 @@
|
|
|
1
|
+
# Contribute
|
|
2
|
+
|
|
3
|
+
## Install
|
|
4
|
+
|
|
5
|
+
Use Python 3.12 for development. From the repository root, run:
|
|
6
|
+
|
|
7
|
+
```sh
|
|
8
|
+
python3.12 -m venv .venv
|
|
9
|
+
.venv/bin/python -m pip install -e '.[dev]'
|
|
10
|
+
```
|
|
11
|
+
|
|
12
|
+
## Before a commit
|
|
13
|
+
|
|
14
|
+
```sh
|
|
15
|
+
.venv/bin/ruff check .
|
|
16
|
+
.venv/bin/ruff format --check .
|
|
17
|
+
.venv/bin/mypy src/table_proof
|
|
18
|
+
.venv/bin/pytest --cov=table_proof --cov-report=term-missing
|
|
19
|
+
.venv/bin/python -m benchmarks.run \
|
|
20
|
+
--methods tableproof token_multiset_control structural_validity_control \
|
|
21
|
+
--output artifacts/local-benchmark.json
|
|
22
|
+
.venv/bin/python -m build
|
|
23
|
+
.venv/bin/python -m twine check dist/*
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
The default tests do not download model weights. Read the [vision guide](docs/VISION.md) for a real-model run.
|
|
27
|
+
|
|
28
|
+
## Add a defect case
|
|
29
|
+
|
|
30
|
+
1. Reduce the source to the smallest useful public example.
|
|
31
|
+
2. Record its license and attribution.
|
|
32
|
+
3. Record the extractor version and exact command.
|
|
33
|
+
4. Preserve the actual extracted output.
|
|
34
|
+
5. Label the wrong cells against the original source.
|
|
35
|
+
6. Add a correct control from the same layout.
|
|
36
|
+
7. Record unsupported evidence explicitly.
|
|
37
|
+
|
|
38
|
+
Do not submit confidential documents. Use an authorized public source or an independent synthetic reproduction.
|
|
39
|
+
|
|
40
|
+
## Change a detector
|
|
41
|
+
|
|
42
|
+
Add a regression test that fails before the change. Include a correct table that exercises the same rule.
|
|
43
|
+
|
|
44
|
+
Keep uncertain evidence separate from an error finding. A broader detector must not silently convert uncertainty to success.
|
|
45
|
+
|
|
46
|
+
Update the architecture and report documents when behavior changes. Write short sentences with one meaning per term.
|
|
47
|
+
|
|
48
|
+
## Dependencies
|
|
49
|
+
|
|
50
|
+
Keep heavy model packages in the vision extra. Keep model downloads explicit.
|
|
51
|
+
|
|
52
|
+
For the measured competitor comparison, use the isolated benchmark lock file. Do not replace a competitor with a local approximation.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Eduard Smirnov
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,177 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: table-proof-py
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Audit PDF table cell associations against source evidence
|
|
5
|
+
Project-URL: Homepage, https://github.com/eduard-s/table-proof
|
|
6
|
+
Project-URL: Repository, https://github.com/eduard-s/table-proof
|
|
7
|
+
Project-URL: Issues, https://github.com/eduard-s/table-proof/issues
|
|
8
|
+
Author: Eduard Smirnov
|
|
9
|
+
License-Expression: MIT
|
|
10
|
+
License-File: LICENSE
|
|
11
|
+
Classifier: Development Status :: 3 - Alpha
|
|
12
|
+
Classifier: Programming Language :: Python :: 3
|
|
13
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
14
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
17
|
+
Classifier: Typing :: Typed
|
|
18
|
+
Requires-Python: >=3.11
|
|
19
|
+
Requires-Dist: pdfplumber<0.12,>=0.11.4
|
|
20
|
+
Requires-Dist: pydantic<3,>=2.11
|
|
21
|
+
Provides-Extra: dev
|
|
22
|
+
Requires-Dist: build<2,>=1; extra == 'dev'
|
|
23
|
+
Requires-Dist: docling-core<3,>=2.48; extra == 'dev'
|
|
24
|
+
Requires-Dist: hypothesis<7,>=6; extra == 'dev'
|
|
25
|
+
Requires-Dist: jsonschema<5,>=4; extra == 'dev'
|
|
26
|
+
Requires-Dist: mypy<2,>=1.15; extra == 'dev'
|
|
27
|
+
Requires-Dist: pytest-cov<8,>=6; extra == 'dev'
|
|
28
|
+
Requires-Dist: pytest<10,>=8; extra == 'dev'
|
|
29
|
+
Requires-Dist: reportlab<5,>=4; extra == 'dev'
|
|
30
|
+
Requires-Dist: ruff<1,>=0.14; extra == 'dev'
|
|
31
|
+
Requires-Dist: twine<7,>=6; extra == 'dev'
|
|
32
|
+
Provides-Extra: vision
|
|
33
|
+
Requires-Dist: timm<2,>=1; extra == 'vision'
|
|
34
|
+
Requires-Dist: torch>=2.6; extra == 'vision'
|
|
35
|
+
Requires-Dist: transformers<5,>=4.50; extra == 'vision'
|
|
36
|
+
Description-Content-Type: text/markdown
|
|
37
|
+
|
|
38
|
+
# TableProof
|
|
39
|
+
|
|
40
|
+
[](https://github.com/eduard-s/table-proof/actions/workflows/ci.yml)
|
|
41
|
+
|
|
42
|
+
TableProof finds suspected errors in extracted PDF tables. It compares cell text and row or column positions with source evidence.
|
|
43
|
+
|
|
44
|
+
Every number can survive extraction while its row label changes. TableProof reports the disputed cell and its original PDF region.
|
|
45
|
+
|
|
46
|
+
This is an alpha Python library. It needs Python 3.11 or later.
|
|
47
|
+
|
|
48
|
+
## Installation
|
|
49
|
+
|
|
50
|
+
```sh
|
|
51
|
+
python -m pip install table-proof-py
|
|
52
|
+
table-proof source.pdf docling-export.json --html report.html --json report.json
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
`docling-export.json` is a Docling document export with table cells and source coordinates.
|
|
56
|
+
|
|
57
|
+
For the optional local vision model, install the extra dependencies:
|
|
58
|
+
|
|
59
|
+
```sh
|
|
60
|
+
python -m pip install 'table-proof-py[vision]'
|
|
61
|
+
```
|
|
62
|
+
|
|
63
|
+
## Try the real regression case
|
|
64
|
+
|
|
65
|
+
The recorded regression pair contains this error:
|
|
66
|
+
|
|
67
|
+
| Source row | Affected extracted row | Result |
|
|
68
|
+
| --- | --- | --- |
|
|
69
|
+
| `BBB → 20.0 mg` | `CCC → 20.0 mg` | Row label differs from the source region. |
|
|
70
|
+
| `CCC → 30.0 mg` | `BBB → 30.0 mg` | Row label differs from the source region. |
|
|
71
|
+
|
|
72
|
+
The regression fixtures are in the source repository. Run these commands:
|
|
73
|
+
|
|
74
|
+
```sh
|
|
75
|
+
git clone https://github.com/eduard-s/table-proof.git
|
|
76
|
+
cd table-proof
|
|
77
|
+
python3 -m venv .venv
|
|
78
|
+
.venv/bin/python -m pip install -e .
|
|
79
|
+
.venv/bin/table-proof \
|
|
80
|
+
benchmarks/fixtures/docling_5198/source.pdf \
|
|
81
|
+
benchmarks/fixtures/docling_5198/affected.json \
|
|
82
|
+
--html report.html --json report.json
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
The command exits with code `1`. It reports two swapped row labels in the affected fixture.
|
|
86
|
+
|
|
87
|
+
Replace `affected.json` with `fixed.json` to examine the paired control. That command exits with code `0`.
|
|
88
|
+
|
|
89
|
+
The fixture reproduces a real Docling defect on a generated PDF. It does not represent an independent production document.
|
|
90
|
+
|
|
91
|
+
Read the [benchmark results](https://github.com/eduard-s/table-proof/blob/main/benchmarks/results.md) and [fixture provenance](https://github.com/eduard-s/table-proof/blob/main/benchmarks/README.md).
|
|
92
|
+
|
|
93
|
+
## Python API
|
|
94
|
+
|
|
95
|
+
```python
|
|
96
|
+
import json
|
|
97
|
+
from pathlib import Path
|
|
98
|
+
|
|
99
|
+
from table_proof import audit_tables
|
|
100
|
+
|
|
101
|
+
fixture = Path("benchmarks/fixtures/docling_5198")
|
|
102
|
+
document = json.loads((fixture / "affected.json").read_text())
|
|
103
|
+
report = audit_tables(fixture / "source.pdf", document)
|
|
104
|
+
|
|
105
|
+
print(report.status)
|
|
106
|
+
report.export_json("report.json")
|
|
107
|
+
report.export_html("report.html", source_pdf=str(fixture / "source.pdf"))
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
The adapter accepts Docling JSON, a `DoclingDocument`, or a conversion result with a `document` field. Docling is optional at runtime.
|
|
111
|
+
|
|
112
|
+
Rich cells can contain text references. The adapter resolves the references that control the rendered table. It does not trust stale cell text.
|
|
113
|
+
|
|
114
|
+
## What the MVP does
|
|
115
|
+
|
|
116
|
+
- Compares cell tokens with PDF words in the cell region.
|
|
117
|
+
- Detects reversed row or column positions and invalid cell spans.
|
|
118
|
+
- Reports missing source text, incomplete geometry, and ambiguous word assignments.
|
|
119
|
+
- Produces JSON findings and local HTML evidence.
|
|
120
|
+
- Preserves the source PDF and extraction.
|
|
121
|
+
- Supports an optional, pinned Table Transformer model for independent structure evidence.
|
|
122
|
+
|
|
123
|
+
The default path uses PDF text and geometry. It makes no network requests.
|
|
124
|
+
|
|
125
|
+
The optional model runs locally. It needs an explicit download option before its first use. Read the [vision guide](https://github.com/eduard-s/table-proof/blob/main/docs/VISION.md).
|
|
126
|
+
|
|
127
|
+
## Results and limits
|
|
128
|
+
|
|
129
|
+
| Status | Meaning | CLI exit code |
|
|
130
|
+
| --- | --- | --- |
|
|
131
|
+
| `no_findings` | The selected checks found no error in the supplied evidence. | `0` |
|
|
132
|
+
| `review` | At least one check found a suspected error. | `1` |
|
|
133
|
+
| `uncertain` | Evidence is missing, ambiguous, or unsupported. | `3` |
|
|
134
|
+
| `error` | The command did not process the input. | `2` |
|
|
135
|
+
|
|
136
|
+
`no_findings` does not prove semantic accuracy. An empty table list returns `uncertain`.
|
|
137
|
+
|
|
138
|
+
Start with digital PDFs and tables that occupy one page. Scans and multi-page tables need other methods.
|
|
139
|
+
|
|
140
|
+
The tool does not certify units or source facts. It does not repair data automatically. Detector scores are not calibrated error probabilities.
|
|
141
|
+
|
|
142
|
+
The [small development benchmark](https://github.com/eduard-s/table-proof/blob/main/benchmarks/results.md) compares TableProof with local controls and pinned upstream pdfmux code.
|
|
143
|
+
Its cases establish a reproducible regression result. They do not establish broad superiority on real documents.
|
|
144
|
+
|
|
145
|
+
On the included public paper, TableProof flags six missing headers. It returns uncertainty for a clipped superscript in another parser output.
|
|
146
|
+
|
|
147
|
+
The optional vision model recovers the table grid but adds no error detection on that public example.
|
|
148
|
+
Read the [recorded model results](https://github.com/eduard-s/table-proof/blob/main/docs/VISION.md) before a model accuracy claim.
|
|
149
|
+
|
|
150
|
+
## Development
|
|
151
|
+
|
|
152
|
+
```sh
|
|
153
|
+
.venv/bin/python -m pip install -e '.[dev]'
|
|
154
|
+
.venv/bin/ruff check .
|
|
155
|
+
.venv/bin/ruff format --check .
|
|
156
|
+
.venv/bin/mypy src/table_proof
|
|
157
|
+
.venv/bin/pytest --cov=table_proof --cov-report=term-missing
|
|
158
|
+
.venv/bin/python -m build
|
|
159
|
+
.venv/bin/python -m twine check dist/*
|
|
160
|
+
```
|
|
161
|
+
|
|
162
|
+
Use Python 3.12 for the type analysis. CI runs tests on Python 3.11 through 3.14.
|
|
163
|
+
|
|
164
|
+
## Documents
|
|
165
|
+
|
|
166
|
+
- [MVP plan and acceptance criteria](https://github.com/eduard-s/table-proof/blob/main/docs/MVP_PLAN.md)
|
|
167
|
+
- [Architecture and Python stack](https://github.com/eduard-s/table-proof/blob/main/docs/ARCHITECTURE.md)
|
|
168
|
+
- [API and report contract](https://github.com/eduard-s/table-proof/blob/main/docs/API.md)
|
|
169
|
+
- [Datasets and evaluation plan](https://github.com/eduard-s/table-proof/blob/main/docs/DATASETS.md)
|
|
170
|
+
- [Benchmark reproduction](https://github.com/eduard-s/table-proof/blob/main/benchmarks/README.md)
|
|
171
|
+
- [Optional vision model](https://github.com/eduard-s/table-proof/blob/main/docs/VISION.md)
|
|
172
|
+
- [Development and contribution](https://github.com/eduard-s/table-proof/blob/main/CONTRIBUTING.md)
|
|
173
|
+
- [Release instructions](https://github.com/eduard-s/table-proof/blob/main/PUBLISHING.md)
|
|
174
|
+
|
|
175
|
+
## License
|
|
176
|
+
|
|
177
|
+
MIT. See [LICENSE](https://github.com/eduard-s/table-proof/blob/main/LICENSE) and [third-party notices](https://github.com/eduard-s/table-proof/blob/main/THIRD_PARTY.md).
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
# Publish TableProof
|
|
2
|
+
|
|
3
|
+
The distribution name is `table-proof-py`. The Python import is `table_proof`. The command is `table-proof`.
|
|
4
|
+
|
|
5
|
+
PyPI rejected `table-proof` because it was too similar to an existing project. Use `table-proof-py` for installation and publisher configuration.
|
|
6
|
+
|
|
7
|
+
Version `0.1.0` is an alpha release. The current benchmark does not establish broad accuracy.
|
|
8
|
+
|
|
9
|
+
The first release needs a pending trusted publisher for this repository. Later releases use that same publisher.
|
|
10
|
+
|
|
11
|
+
## Local distribution
|
|
12
|
+
|
|
13
|
+
```sh
|
|
14
|
+
.venv/bin/python -m build
|
|
15
|
+
.venv/bin/python -m twine check dist/*
|
|
16
|
+
.venv/bin/python scripts/examine_distribution.py
|
|
17
|
+
python3 -m venv .venv-wheel
|
|
18
|
+
.venv-wheel/bin/python -m pip install dist/*.whl
|
|
19
|
+
.venv-wheel/bin/python -m table_proof --version
|
|
20
|
+
.venv-wheel/bin/python scripts/wheel_smoke.py
|
|
21
|
+
```
|
|
22
|
+
|
|
23
|
+
The wheel smoke test uses the recorded regression pair. It needs only the runtime dependencies.
|
|
24
|
+
|
|
25
|
+
The archive examination rejects cache files, model weights, symbolic links, and unexpected paths. The source archive contains only selected project files.
|
|
26
|
+
|
|
27
|
+
## Configure the first publication
|
|
28
|
+
|
|
29
|
+
1. Sign in to the PyPI account that will own `table-proof-py`.
|
|
30
|
+
2. Open [PyPI Publishing](https://pypi.org/manage/account/publishing/).
|
|
31
|
+
3. Add a pending GitHub publisher with these values:
|
|
32
|
+
|
|
33
|
+
| Field | Value |
|
|
34
|
+
| --- | --- |
|
|
35
|
+
| PyPI project name | `table-proof-py` |
|
|
36
|
+
| GitHub owner | `eduard-s` |
|
|
37
|
+
| GitHub repository | `table-proof` |
|
|
38
|
+
| Workflow filename | `publish-python.yml` |
|
|
39
|
+
| Environment | `pypi` |
|
|
40
|
+
|
|
41
|
+
4. Create the `pypi` environment in this repository's GitHub settings.
|
|
42
|
+
5. Examine the environment rules for the release account.
|
|
43
|
+
|
|
44
|
+
The ToolBoundary publisher does not authorize this repository. This workflow uses GitHub OIDC and needs no stored PyPI token.
|
|
45
|
+
|
|
46
|
+
A pending publisher does not reserve the package name. Read the [PyPI guide](https://docs.pypi.org/trusted-publishers/creating-a-project-through-oidc/).
|
|
47
|
+
|
|
48
|
+
## Rehearse the release
|
|
49
|
+
|
|
50
|
+
1. Open the repository's **Actions** page.
|
|
51
|
+
2. Select **Publish Python**.
|
|
52
|
+
3. Select **Run workflow** on `main`.
|
|
53
|
+
4. Examine the distribution job and its artifacts.
|
|
54
|
+
|
|
55
|
+
A manual run makes and examines the distributions. It does not publish them. Publication starts only from a pushed version tag.
|
|
56
|
+
|
|
57
|
+
## Publish version 0.1.0
|
|
58
|
+
|
|
59
|
+
After the publisher configuration and release examination, run these commands:
|
|
60
|
+
|
|
61
|
+
```sh
|
|
62
|
+
git switch main
|
|
63
|
+
git pull --ff-only
|
|
64
|
+
git tag v0.1.0
|
|
65
|
+
git push origin v0.1.0
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
The tag must match the version in `pyproject.toml`. The workflow runs tests before publication.
|
|
69
|
+
|
|
70
|
+
The publish job uses the exact distribution artifacts from the successful preparation job. It runs separately from the package tests.
|
|
71
|
+
|
|
72
|
+
Read the [PyPI publication guide](https://docs.pypi.org/trusted-publishers/using-a-publisher/) for the authentication process.
|
|
73
|
+
|
|
74
|
+
## Examine the public package
|
|
75
|
+
|
|
76
|
+
After publication, run these commands from the repository root:
|
|
77
|
+
|
|
78
|
+
```sh
|
|
79
|
+
python3 -m venv .venv-published
|
|
80
|
+
.venv-published/bin/python -m pip install --index-url https://pypi.org/simple 'table-proof-py==0.1.0'
|
|
81
|
+
.venv-published/bin/python -m table_proof --version
|
|
82
|
+
.venv-published/bin/python scripts/wheel_smoke.py
|
|
83
|
+
```
|
|
84
|
+
|
|
85
|
+
After publication, examine the PyPI page. Add its link to the README, profile, and website. Keep benchmark limitations visible.
|
|
@@ -0,0 +1,140 @@
|
|
|
1
|
+
# TableProof
|
|
2
|
+
|
|
3
|
+
[](https://github.com/eduard-s/table-proof/actions/workflows/ci.yml)
|
|
4
|
+
|
|
5
|
+
TableProof finds suspected errors in extracted PDF tables. It compares cell text and row or column positions with source evidence.
|
|
6
|
+
|
|
7
|
+
Every number can survive extraction while its row label changes. TableProof reports the disputed cell and its original PDF region.
|
|
8
|
+
|
|
9
|
+
This is an alpha Python library. It needs Python 3.11 or later.
|
|
10
|
+
|
|
11
|
+
## Installation
|
|
12
|
+
|
|
13
|
+
```sh
|
|
14
|
+
python -m pip install table-proof-py
|
|
15
|
+
table-proof source.pdf docling-export.json --html report.html --json report.json
|
|
16
|
+
```
|
|
17
|
+
|
|
18
|
+
`docling-export.json` is a Docling document export with table cells and source coordinates.
|
|
19
|
+
|
|
20
|
+
For the optional local vision model, install the extra dependencies:
|
|
21
|
+
|
|
22
|
+
```sh
|
|
23
|
+
python -m pip install 'table-proof-py[vision]'
|
|
24
|
+
```
|
|
25
|
+
|
|
26
|
+
## Try the real regression case
|
|
27
|
+
|
|
28
|
+
The recorded regression pair contains this error:
|
|
29
|
+
|
|
30
|
+
| Source row | Affected extracted row | Result |
|
|
31
|
+
| --- | --- | --- |
|
|
32
|
+
| `BBB → 20.0 mg` | `CCC → 20.0 mg` | Row label differs from the source region. |
|
|
33
|
+
| `CCC → 30.0 mg` | `BBB → 30.0 mg` | Row label differs from the source region. |
|
|
34
|
+
|
|
35
|
+
The regression fixtures are in the source repository. Run these commands:
|
|
36
|
+
|
|
37
|
+
```sh
|
|
38
|
+
git clone https://github.com/eduard-s/table-proof.git
|
|
39
|
+
cd table-proof
|
|
40
|
+
python3 -m venv .venv
|
|
41
|
+
.venv/bin/python -m pip install -e .
|
|
42
|
+
.venv/bin/table-proof \
|
|
43
|
+
benchmarks/fixtures/docling_5198/source.pdf \
|
|
44
|
+
benchmarks/fixtures/docling_5198/affected.json \
|
|
45
|
+
--html report.html --json report.json
|
|
46
|
+
```
|
|
47
|
+
|
|
48
|
+
The command exits with code `1`. It reports two swapped row labels in the affected fixture.
|
|
49
|
+
|
|
50
|
+
Replace `affected.json` with `fixed.json` to examine the paired control. That command exits with code `0`.
|
|
51
|
+
|
|
52
|
+
The fixture reproduces a real Docling defect on a generated PDF. It does not represent an independent production document.
|
|
53
|
+
|
|
54
|
+
Read the [benchmark results](https://github.com/eduard-s/table-proof/blob/main/benchmarks/results.md) and [fixture provenance](https://github.com/eduard-s/table-proof/blob/main/benchmarks/README.md).
|
|
55
|
+
|
|
56
|
+
## Python API
|
|
57
|
+
|
|
58
|
+
```python
|
|
59
|
+
import json
|
|
60
|
+
from pathlib import Path
|
|
61
|
+
|
|
62
|
+
from table_proof import audit_tables
|
|
63
|
+
|
|
64
|
+
fixture = Path("benchmarks/fixtures/docling_5198")
|
|
65
|
+
document = json.loads((fixture / "affected.json").read_text())
|
|
66
|
+
report = audit_tables(fixture / "source.pdf", document)
|
|
67
|
+
|
|
68
|
+
print(report.status)
|
|
69
|
+
report.export_json("report.json")
|
|
70
|
+
report.export_html("report.html", source_pdf=str(fixture / "source.pdf"))
|
|
71
|
+
```
|
|
72
|
+
|
|
73
|
+
The adapter accepts Docling JSON, a `DoclingDocument`, or a conversion result with a `document` field. Docling is optional at runtime.
|
|
74
|
+
|
|
75
|
+
Rich cells can contain text references. The adapter resolves the references that control the rendered table. It does not trust stale cell text.
|
|
76
|
+
|
|
77
|
+
## What the MVP does
|
|
78
|
+
|
|
79
|
+
- Compares cell tokens with PDF words in the cell region.
|
|
80
|
+
- Detects reversed row or column positions and invalid cell spans.
|
|
81
|
+
- Reports missing source text, incomplete geometry, and ambiguous word assignments.
|
|
82
|
+
- Produces JSON findings and local HTML evidence.
|
|
83
|
+
- Preserves the source PDF and extraction.
|
|
84
|
+
- Supports an optional, pinned Table Transformer model for independent structure evidence.
|
|
85
|
+
|
|
86
|
+
The default path uses PDF text and geometry. It makes no network requests.
|
|
87
|
+
|
|
88
|
+
The optional model runs locally. It needs an explicit download option before its first use. Read the [vision guide](https://github.com/eduard-s/table-proof/blob/main/docs/VISION.md).
|
|
89
|
+
|
|
90
|
+
## Results and limits
|
|
91
|
+
|
|
92
|
+
| Status | Meaning | CLI exit code |
|
|
93
|
+
| --- | --- | --- |
|
|
94
|
+
| `no_findings` | The selected checks found no error in the supplied evidence. | `0` |
|
|
95
|
+
| `review` | At least one check found a suspected error. | `1` |
|
|
96
|
+
| `uncertain` | Evidence is missing, ambiguous, or unsupported. | `3` |
|
|
97
|
+
| `error` | The command did not process the input. | `2` |
|
|
98
|
+
|
|
99
|
+
`no_findings` does not prove semantic accuracy. An empty table list returns `uncertain`.
|
|
100
|
+
|
|
101
|
+
Start with digital PDFs and tables that occupy one page. Scans and multi-page tables need other methods.
|
|
102
|
+
|
|
103
|
+
The tool does not certify units or source facts. It does not repair data automatically. Detector scores are not calibrated error probabilities.
|
|
104
|
+
|
|
105
|
+
The [small development benchmark](https://github.com/eduard-s/table-proof/blob/main/benchmarks/results.md) compares TableProof with local controls and pinned upstream pdfmux code.
|
|
106
|
+
Its cases establish a reproducible regression result. They do not establish broad superiority on real documents.
|
|
107
|
+
|
|
108
|
+
On the included public paper, TableProof flags six missing headers. It returns uncertainty for a clipped superscript in another parser output.
|
|
109
|
+
|
|
110
|
+
The optional vision model recovers the table grid but adds no error detection on that public example.
|
|
111
|
+
Read the [recorded model results](https://github.com/eduard-s/table-proof/blob/main/docs/VISION.md) before a model accuracy claim.
|
|
112
|
+
|
|
113
|
+
## Development
|
|
114
|
+
|
|
115
|
+
```sh
|
|
116
|
+
.venv/bin/python -m pip install -e '.[dev]'
|
|
117
|
+
.venv/bin/ruff check .
|
|
118
|
+
.venv/bin/ruff format --check .
|
|
119
|
+
.venv/bin/mypy src/table_proof
|
|
120
|
+
.venv/bin/pytest --cov=table_proof --cov-report=term-missing
|
|
121
|
+
.venv/bin/python -m build
|
|
122
|
+
.venv/bin/python -m twine check dist/*
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
Use Python 3.12 for the type analysis. CI runs tests on Python 3.11 through 3.14.
|
|
126
|
+
|
|
127
|
+
## Documents
|
|
128
|
+
|
|
129
|
+
- [MVP plan and acceptance criteria](https://github.com/eduard-s/table-proof/blob/main/docs/MVP_PLAN.md)
|
|
130
|
+
- [Architecture and Python stack](https://github.com/eduard-s/table-proof/blob/main/docs/ARCHITECTURE.md)
|
|
131
|
+
- [API and report contract](https://github.com/eduard-s/table-proof/blob/main/docs/API.md)
|
|
132
|
+
- [Datasets and evaluation plan](https://github.com/eduard-s/table-proof/blob/main/docs/DATASETS.md)
|
|
133
|
+
- [Benchmark reproduction](https://github.com/eduard-s/table-proof/blob/main/benchmarks/README.md)
|
|
134
|
+
- [Optional vision model](https://github.com/eduard-s/table-proof/blob/main/docs/VISION.md)
|
|
135
|
+
- [Development and contribution](https://github.com/eduard-s/table-proof/blob/main/CONTRIBUTING.md)
|
|
136
|
+
- [Release instructions](https://github.com/eduard-s/table-proof/blob/main/PUBLISHING.md)
|
|
137
|
+
|
|
138
|
+
## License
|
|
139
|
+
|
|
140
|
+
MIT. See [LICENSE](https://github.com/eduard-s/table-proof/blob/main/LICENSE) and [third-party notices](https://github.com/eduard-s/table-proof/blob/main/THIRD_PARTY.md).
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# Third-party materials
|
|
2
|
+
|
|
3
|
+
TableProof uses the MIT license for its original code and generated fixtures.
|
|
4
|
+
|
|
5
|
+
The benchmark adapts a public Docling-core regression reproducer and its proposed fix. Read [the benchmark notices](benchmarks/THIRD_PARTY.md).
|
|
6
|
+
|
|
7
|
+
The optional Table Transformer checkpoint remains a separate download. Its [model card](https://huggingface.co/microsoft/table-transformer-structure-recognition-v1.1-all) supplies its metadata.
|
|
8
|
+
|
|
9
|
+
The [upstream project](https://github.com/microsoft/table-transformer) provides model code, datasets, and evaluation resources. Dataset terms remain separate from TableProof's license.
|
|
10
|
+
|
|
11
|
+
pdfmux is an optional benchmark dependency. Its upstream code runs in an isolated environment. TableProof does not distribute a copied pdfmux implementation.
|
|
12
|
+
|
|
13
|
+
Each dependency distribution retains its own license files. The benchmark manifests record licenses for included source materials.
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2024 International Business Machines
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|