pymupdf4llm-rs 0.4.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- pymupdf4llm_rs-0.4.0/.gitattributes +19 -0
- pymupdf4llm_rs-0.4.0/.github/ISSUE_TEMPLATE/bug_report.yml +48 -0
- pymupdf4llm_rs-0.4.0/.github/ISSUE_TEMPLATE/config.yml +5 -0
- pymupdf4llm_rs-0.4.0/.github/ISSUE_TEMPLATE/feature_request.yml +24 -0
- pymupdf4llm_rs-0.4.0/.github/PULL_REQUEST_TEMPLATE.md +19 -0
- pymupdf4llm_rs-0.4.0/.github/workflows/ci.yml +72 -0
- pymupdf4llm_rs-0.4.0/.github/workflows/release-wheels.yml +88 -0
- pymupdf4llm_rs-0.4.0/.gitignore +33 -0
- pymupdf4llm_rs-0.4.0/CHANGELOG.md +54 -0
- pymupdf4llm_rs-0.4.0/CONTRIBUTING.md +46 -0
- pymupdf4llm_rs-0.4.0/Cargo.lock +902 -0
- pymupdf4llm_rs-0.4.0/Cargo.toml +61 -0
- pymupdf4llm_rs-0.4.0/LICENSE +188 -0
- pymupdf4llm_rs-0.4.0/PKG-INFO +202 -0
- pymupdf4llm_rs-0.4.0/README.md +173 -0
- pymupdf4llm_rs-0.4.0/README.zh-CN.md +173 -0
- pymupdf4llm_rs-0.4.0/SECURITY.md +34 -0
- pymupdf4llm_rs-0.4.0/pyproject.toml +56 -0
- pymupdf4llm_rs-0.4.0/python/pymupdf4llm_rs/__init__.py +67 -0
- pymupdf4llm_rs-0.4.0/python/pymupdf4llm_rs/__init__.pyi +174 -0
- pymupdf4llm_rs-0.4.0/python/pymupdf4llm_rs/py.typed +0 -0
- pymupdf4llm_rs-0.4.0/src/async_engine.rs +345 -0
- pymupdf4llm_rs-0.4.0/src/bin/pymupdf4llm.rs +138 -0
- pymupdf4llm_rs-0.4.0/src/config.rs +548 -0
- pymupdf4llm_rs-0.4.0/src/error.rs +36 -0
- pymupdf4llm_rs-0.4.0/src/layout/columns.rs +231 -0
- pymupdf4llm_rs-0.4.0/src/layout/headers.rs +168 -0
- pymupdf4llm_rs-0.4.0/src/layout/mod.rs +198 -0
- pymupdf4llm_rs-0.4.0/src/layout/spatial.rs +136 -0
- pymupdf4llm_rs-0.4.0/src/layout/tables.rs +448 -0
- pymupdf4llm_rs-0.4.0/src/lib.rs +317 -0
- pymupdf4llm_rs-0.4.0/src/llama_reader.rs +69 -0
- pymupdf4llm_rs-0.4.0/src/markdown/chunk.rs +64 -0
- pymupdf4llm_rs-0.4.0/src/markdown/mod.rs +18 -0
- pymupdf4llm_rs-0.4.0/src/markdown/renderer.rs +512 -0
- pymupdf4llm_rs-0.4.0/src/pdf/content_stream.rs +1236 -0
- pymupdf4llm_rs-0.4.0/src/pdf/encoding.rs +1020 -0
- pymupdf4llm_rs-0.4.0/src/pdf/font.rs +465 -0
- pymupdf4llm_rs-0.4.0/src/pdf/mod.rs +152 -0
- pymupdf4llm_rs-0.4.0/src/pdf/objects.rs +136 -0
- pymupdf4llm_rs-0.4.0/src/pdf/parser.rs +1248 -0
- pymupdf4llm_rs-0.4.0/src/python.rs +1262 -0
- pymupdf4llm_rs-0.4.0/src/text_cleaner.rs +155 -0
- pymupdf4llm_rs-0.4.0/tests/compatibility_tests.rs +594 -0
- pymupdf4llm_rs-0.4.0/tests/test_python_api.py +162 -0
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
# Normalize text line endings to LF
|
|
2
|
+
* text=auto eol=lf
|
|
3
|
+
|
|
4
|
+
# Rust Sources & Configs
|
|
5
|
+
*.rs text eol=lf
|
|
6
|
+
*.toml text eol=lf
|
|
7
|
+
*.lock text eol=lf
|
|
8
|
+
*.md text eol=lf
|
|
9
|
+
*.yml text eol=lf
|
|
10
|
+
*.yaml text eol=lf
|
|
11
|
+
*.sh text eol=lf
|
|
12
|
+
|
|
13
|
+
# Binary Formats
|
|
14
|
+
*.pdf binary
|
|
15
|
+
*.png binary
|
|
16
|
+
*.jpg binary
|
|
17
|
+
*.jpeg binary
|
|
18
|
+
*.webp binary
|
|
19
|
+
*.gz binary
|
|
@@ -0,0 +1,48 @@
|
|
|
1
|
+
name: Bug Report / 缺陷反馈
|
|
2
|
+
description: Report a PDF parsing, Markdown formatting, or mojibake hygiene issue / 提交 PDF 解析、排版或乱码清洗缺陷
|
|
3
|
+
title: "[Bug]: "
|
|
4
|
+
labels: ["bug", "triage"]
|
|
5
|
+
body:
|
|
6
|
+
- type: markdown
|
|
7
|
+
attributes:
|
|
8
|
+
value: |
|
|
9
|
+
Thank you for reporting an issue! Both **English** and **简体中文** descriptions are welcome.
|
|
10
|
+
感谢您的反馈!支持使用**英文**或**简体中文**填写以下信息。
|
|
11
|
+
|
|
12
|
+
- type: dropdown
|
|
13
|
+
id: component
|
|
14
|
+
attributes:
|
|
15
|
+
label: Affected Module / 受影响模块
|
|
16
|
+
options:
|
|
17
|
+
- PDF Binary / Stream Parser (`src/pdf`)
|
|
18
|
+
- Layout / Multi-Column / Tables (`src/layout`)
|
|
19
|
+
- Text Hygiene / Mojibake (`src/text_cleaner.rs`)
|
|
20
|
+
- Async Engine / CLI (`src/async_engine.rs`, `src/bin`)
|
|
21
|
+
validations:
|
|
22
|
+
required: true
|
|
23
|
+
|
|
24
|
+
- type: textarea
|
|
25
|
+
id: description
|
|
26
|
+
attributes:
|
|
27
|
+
label: Issue Description / 问题描述
|
|
28
|
+
description: Describe expected PyMuPDF4LLM output vs. actual output / 描述期望的 Markdown 输出与实际差异
|
|
29
|
+
placeholder: e.g. Multi-table page merges two tables, or Type0 CJK characters render as mojibake...
|
|
30
|
+
validations:
|
|
31
|
+
required: true
|
|
32
|
+
|
|
33
|
+
- type: textarea
|
|
34
|
+
id: options
|
|
35
|
+
attributes:
|
|
36
|
+
label: Options & Rust Toolchain / 运行参数与环境
|
|
37
|
+
description: Table strategy, header mode, hygiene flags, rustc version / 表格策略、标题模式、清洗开关与 Rust 版本
|
|
38
|
+
placeholder: |
|
|
39
|
+
- table_strategy: TableStrategy::LinesStrict
|
|
40
|
+
- hdr_info: HeaderMode::IdentifyHeaders
|
|
41
|
+
- hygiene: clean_mojibake=true, dehyphenate=true, dedup_running_headers=false
|
|
42
|
+
- rustc --version:
|
|
43
|
+
|
|
44
|
+
- type: textarea
|
|
45
|
+
id: reproduction
|
|
46
|
+
attributes:
|
|
47
|
+
label: Minimal PDF Sample or Steps / 最小复现步骤或 PDF 特征
|
|
48
|
+
description: Please share PDF producer, font encoding (Type0/TrueType/Type1), or sanitized snippet / 请说明 PDF 生成器、字体编码或脱敏样本
|
|
@@ -0,0 +1,24 @@
|
|
|
1
|
+
name: Feature or Parity Request / 功能与兼容需求
|
|
2
|
+
description: Request PyMuPDF4LLM API parity, PDF stream filter, or performance optimization / 提交新特性、PDF 编码兼容或性能优化建议
|
|
3
|
+
title: "[Feat]: "
|
|
4
|
+
labels: ["enhancement"]
|
|
5
|
+
body:
|
|
6
|
+
- type: markdown
|
|
7
|
+
attributes:
|
|
8
|
+
value: |
|
|
9
|
+
Both **English** and **简体中文** proposals are welcome.
|
|
10
|
+
欢迎使用**英文**或**简体中文**提交功能建议与 `pymupdf4llm` 兼容性需求。
|
|
11
|
+
|
|
12
|
+
- type: textarea
|
|
13
|
+
id: summary
|
|
14
|
+
attributes:
|
|
15
|
+
label: Feature Summary / 需求概述
|
|
16
|
+
description: What capability or PyMuPDF4LLM parameter would you like added? / 希望增加什么能力或兼容参数?
|
|
17
|
+
validations:
|
|
18
|
+
required: true
|
|
19
|
+
|
|
20
|
+
- type: textarea
|
|
21
|
+
id: rationale
|
|
22
|
+
attributes:
|
|
23
|
+
label: Use Case & Algorithms / 使用场景与实现思路
|
|
24
|
+
description: Explain the PDF structure, encoding, or concurrency bottleneck / 描述对应 PDF 结构特征、编码规范或并发瓶颈
|
|
@@ -0,0 +1,19 @@
|
|
|
1
|
+
## Summary / 变更说明
|
|
2
|
+
|
|
3
|
+
<!-- Summarize what changed in English and/or 简体中文 -->
|
|
4
|
+
-
|
|
5
|
+
|
|
6
|
+
## Type of Change / 变更类型
|
|
7
|
+
|
|
8
|
+
- [ ] PDF Format / Encoding Compatibility (`XRef`, `LZWDecode`, `ToUnicode`, `Form XObject`, etc.)
|
|
9
|
+
- [ ] Text Hygiene & Noise Cleanup (`mojibake`, `3 Tr OCR`, watermarks, de-hyphenation, running headers)
|
|
10
|
+
- [ ] Layout & Algorithms (`SpatialGrid`, `UnionFind`, X-Y Cut, header clustering)
|
|
11
|
+
- [ ] Async & Concurrency / CLI (`Tokio`, `Rayon`, `DashMap`, `pymupdf4llm` CLI)
|
|
12
|
+
- [ ] Documentation / CI (`README`, `CHANGELOG`, `.github`)
|
|
13
|
+
|
|
14
|
+
## Verification Checklist / 验证清单
|
|
15
|
+
|
|
16
|
+
- [ ] `cargo fmt --all -- --check`
|
|
17
|
+
- [ ] `cargo clippy --all-targets --all-features -- -D warnings`
|
|
18
|
+
- [ ] `cargo test --all-features`
|
|
19
|
+
- [ ] Updated bilingual docs (`README.md` / `README.zh-CN.md` / `CHANGELOG.md`) if public API changed
|
|
@@ -0,0 +1,72 @@
|
|
|
1
|
+
name: CI (Pure-Rust Crate)
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
branches: [main, master]
|
|
6
|
+
pull_request:
|
|
7
|
+
branches: [main, master]
|
|
8
|
+
|
|
9
|
+
env:
|
|
10
|
+
CARGO_TERM_COLOR: always
|
|
11
|
+
|
|
12
|
+
jobs:
|
|
13
|
+
rust:
|
|
14
|
+
name: Build, Clippy & Test (pymupdf4llm-rs)
|
|
15
|
+
runs-on: ubuntu-latest
|
|
16
|
+
steps:
|
|
17
|
+
- name: Checkout repository
|
|
18
|
+
uses: actions/checkout@v4
|
|
19
|
+
|
|
20
|
+
- name: Install stable Rust toolchain
|
|
21
|
+
uses: dtolnay/rust-toolchain@stable
|
|
22
|
+
with:
|
|
23
|
+
components: rustfmt, clippy
|
|
24
|
+
|
|
25
|
+
- name: Cache Cargo artifacts
|
|
26
|
+
uses: actions/cache@v4
|
|
27
|
+
with:
|
|
28
|
+
path: |
|
|
29
|
+
~/.cargo/bin/
|
|
30
|
+
~/.cargo/registry/index/
|
|
31
|
+
~/.cargo/registry/cache/
|
|
32
|
+
~/.cargo/git/db/
|
|
33
|
+
target/
|
|
34
|
+
key: ${{ runner.os }}-cargo-${{ hashFiles('Cargo.toml') }}
|
|
35
|
+
|
|
36
|
+
- name: Check formatting (rustfmt)
|
|
37
|
+
run: cargo fmt --all -- --check
|
|
38
|
+
|
|
39
|
+
- name: Run Clippy lints
|
|
40
|
+
run: cargo clippy --all-targets --features "async,cli,images" -- -D warnings
|
|
41
|
+
|
|
42
|
+
- name: Run unit & compatibility tests
|
|
43
|
+
run: cargo test --features "async,cli,images"
|
|
44
|
+
|
|
45
|
+
- name: Check feature combinations
|
|
46
|
+
run: |
|
|
47
|
+
cargo check --no-default-features
|
|
48
|
+
cargo check --no-default-features --features async
|
|
49
|
+
cargo check --features "async,cli,images"
|
|
50
|
+
|
|
51
|
+
python-pyo3:
|
|
52
|
+
name: Build & Test Python Package (PyO3 + maturin)
|
|
53
|
+
runs-on: ubuntu-latest
|
|
54
|
+
steps:
|
|
55
|
+
- name: Checkout repository
|
|
56
|
+
uses: actions/checkout@v4
|
|
57
|
+
|
|
58
|
+
- name: Set up Python
|
|
59
|
+
uses: actions/setup-python@v5
|
|
60
|
+
with:
|
|
61
|
+
python-version: "3.11"
|
|
62
|
+
|
|
63
|
+
- name: Install stable Rust toolchain
|
|
64
|
+
uses: dtolnay/rust-toolchain@stable
|
|
65
|
+
|
|
66
|
+
- name: Install maturin and pytest
|
|
67
|
+
run: pip install --upgrade pip maturin pytest
|
|
68
|
+
|
|
69
|
+
- name: Build extension module & run pytest
|
|
70
|
+
run: |
|
|
71
|
+
maturin develop --release
|
|
72
|
+
pytest -v tests/test_python_api.py
|
|
@@ -0,0 +1,88 @@
|
|
|
1
|
+
name: Release Python Wheels
|
|
2
|
+
|
|
3
|
+
on:
|
|
4
|
+
push:
|
|
5
|
+
tags:
|
|
6
|
+
- "v*"
|
|
7
|
+
workflow_dispatch:
|
|
8
|
+
|
|
9
|
+
permissions:
|
|
10
|
+
contents: write
|
|
11
|
+
|
|
12
|
+
jobs:
|
|
13
|
+
linux:
|
|
14
|
+
runs-on: ${{ matrix.runner }}
|
|
15
|
+
strategy:
|
|
16
|
+
matrix:
|
|
17
|
+
include:
|
|
18
|
+
- runner: ubuntu-latest
|
|
19
|
+
target: x86_64
|
|
20
|
+
- runner: ubuntu-24.04-arm
|
|
21
|
+
target: aarch64
|
|
22
|
+
steps:
|
|
23
|
+
- uses: actions/checkout@v4
|
|
24
|
+
- uses: actions/setup-python@v5
|
|
25
|
+
with:
|
|
26
|
+
python-version: "3.11"
|
|
27
|
+
- name: Build abi3 wheels (manylinux)
|
|
28
|
+
uses: PyO3/maturin-action@v1
|
|
29
|
+
with:
|
|
30
|
+
target: ${{ matrix.target }}
|
|
31
|
+
args: --release --out dist --find-interpreter
|
|
32
|
+
manylinux: auto
|
|
33
|
+
- uses: actions/upload-artifact@v4
|
|
34
|
+
with:
|
|
35
|
+
name: wheels-linux-${{ matrix.target }}
|
|
36
|
+
path: dist
|
|
37
|
+
|
|
38
|
+
macos:
|
|
39
|
+
runs-on: macos-14
|
|
40
|
+
strategy:
|
|
41
|
+
matrix:
|
|
42
|
+
target: [aarch64-apple-darwin, x86_64-apple-darwin]
|
|
43
|
+
steps:
|
|
44
|
+
- uses: actions/checkout@v4
|
|
45
|
+
- uses: actions/setup-python@v5
|
|
46
|
+
with:
|
|
47
|
+
python-version: "3.11"
|
|
48
|
+
- name: Build abi3 wheels (macOS)
|
|
49
|
+
uses: PyO3/maturin-action@v1
|
|
50
|
+
with:
|
|
51
|
+
target: ${{ matrix.target }}
|
|
52
|
+
args: --release --out dist --find-interpreter
|
|
53
|
+
- uses: actions/upload-artifact@v4
|
|
54
|
+
with:
|
|
55
|
+
name: wheels-macos-${{ matrix.target }}
|
|
56
|
+
path: dist
|
|
57
|
+
|
|
58
|
+
windows:
|
|
59
|
+
runs-on: windows-latest
|
|
60
|
+
steps:
|
|
61
|
+
- uses: actions/checkout@v4
|
|
62
|
+
- uses: actions/setup-python@v5
|
|
63
|
+
with:
|
|
64
|
+
python-version: "3.11"
|
|
65
|
+
- name: Build abi3 wheels (Windows x64)
|
|
66
|
+
uses: PyO3/maturin-action@v1
|
|
67
|
+
with:
|
|
68
|
+
target: x64
|
|
69
|
+
args: --release --out dist --find-interpreter
|
|
70
|
+
- uses: actions/upload-artifact@v4
|
|
71
|
+
with:
|
|
72
|
+
name: wheels-windows-x64
|
|
73
|
+
path: dist
|
|
74
|
+
|
|
75
|
+
release:
|
|
76
|
+
needs: [linux, macos, windows]
|
|
77
|
+
runs-on: ubuntu-latest
|
|
78
|
+
if: startsWith(github.ref, 'refs/tags/v')
|
|
79
|
+
steps:
|
|
80
|
+
- uses: actions/download-artifact@v4
|
|
81
|
+
with:
|
|
82
|
+
pattern: wheels-*
|
|
83
|
+
path: dist
|
|
84
|
+
merge-multiple: true
|
|
85
|
+
- name: Publish wheels to GitHub Release
|
|
86
|
+
uses: softprops/action-gh-release@v2
|
|
87
|
+
with:
|
|
88
|
+
files: dist/*.whl
|
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
# Rust / Cargo
|
|
2
|
+
/target/
|
|
3
|
+
**/*.rs.bk
|
|
4
|
+
*.pdb
|
|
5
|
+
|
|
6
|
+
# Python / Maturin
|
|
7
|
+
*.so
|
|
8
|
+
*.pyd
|
|
9
|
+
*.whl
|
|
10
|
+
__pycache__/
|
|
11
|
+
.pytest_cache/
|
|
12
|
+
.venv/
|
|
13
|
+
*.egg-info/
|
|
14
|
+
python/pymupdf4llm_rs/*.so
|
|
15
|
+
python/pymupdf4llm_rs/*.pyd
|
|
16
|
+
|
|
17
|
+
# AI Studio Runtime Bridge (not required when consuming Rust crate)
|
|
18
|
+
node_modules/
|
|
19
|
+
dist/
|
|
20
|
+
.playwright-cli/
|
|
21
|
+
|
|
22
|
+
# Environment
|
|
23
|
+
.env
|
|
24
|
+
.env.*
|
|
25
|
+
|
|
26
|
+
# Editor & OS Artifacts
|
|
27
|
+
.DS_Store
|
|
28
|
+
Thumbs.db
|
|
29
|
+
*.log
|
|
30
|
+
.vscode/*
|
|
31
|
+
!.vscode/extensions.json
|
|
32
|
+
.idea/
|
|
33
|
+
.aistudio/
|
|
@@ -0,0 +1,54 @@
|
|
|
1
|
+
# Changelog / 更新日志
|
|
2
|
+
|
|
3
|
+
All notable changes to `pymupdf4llm-rs` are documented in this file.
|
|
4
|
+
本文档记录 `pymupdf4llm-rs` 纯 Rust 核心库的重要版本变更。
|
|
5
|
+
|
|
6
|
+
---
|
|
7
|
+
|
|
8
|
+
## [0.4.0] - 2026-11-16
|
|
9
|
+
|
|
10
|
+
### English
|
|
11
|
+
- **PDF Format & Encoding Compatibility**:
|
|
12
|
+
- Added `/Pages` tree inheritance for `/MediaBox`, `/CropBox`, `/Rotate`, and `/Resources`.
|
|
13
|
+
- Added `/Subtype /Form` XObject recursive stream execution with `/Matrix` & font stack isolation.
|
|
14
|
+
- Added pure-Rust `LZWDecode` decompression (9–12 bit variable codes with `EarlyChange`).
|
|
15
|
+
- Added `Type0`/CID `/W` width parsing, array-form `beginbfrange` `ToUnicode` CMap decoding, `MacRomanEncoding`, and 70+ Adobe Glyph names.
|
|
16
|
+
- Added `/Rotate` (0°/90°/180°/270°) page coordinate normalization and `DashMap` lock-free font caching.
|
|
17
|
+
- **Text Hygiene & Noise Cleanup**:
|
|
18
|
+
- Added `TextHygieneConfig` & `HygieneStats` diagnostics (`clean_mojibake`, `dehyphenate`, `dedup_faux_bold`, `ignore_invisible_text`, `strip_watermarks`, `dedup_running_headers`).
|
|
19
|
+
- Stripped `3 Tr` invisible OCR overlays, diagonal/low-alpha watermarks, unmapped PUA/control mojibake, and faux-bold overprint duplicates.
|
|
20
|
+
- Added ligature expansion (`ff`, `fi`, `fl`, `ffi`, `ffl`, `ſt`), PUA bullet normalization, line-wrap de-hyphenation, and adjacent same-URI hyperlink merging.
|
|
21
|
+
- **Algorithms & Performance**:
|
|
22
|
+
- Added 2D `SpatialGrid` for $O(1)$ average-case link attachment and table cell span assignment.
|
|
23
|
+
- Added `UnionFind` (DSU) connected components to isolate multiple independent tables per page and cluster vector graphics.
|
|
24
|
+
- **Repository Structure & Python Package (`PyO3` + `maturin`)**:
|
|
25
|
+
- Promoted `pymupdf4llm-rs` to repository root as a standalone pure-Rust library + CLI crate + Python wheel (`abi3-py39`).
|
|
26
|
+
- Added reusable `PdfExtractor` and parse-once `ParsedPdf` Python classes holding shared `Arc<PdfEngine>`, `Arc<ToMarkdownOptions>`, Tokio runtime, global `ToUnicode` CMap cache, and `max_pdf_bytes` / `max_pages` safety guards with GIL release (`py.allow_threads`).
|
|
27
|
+
- Enforced 64 MB decompression-bomb limits across `FlateDecode`, `LZWDecode`, `ASCII85Decode`, and `RunLengthDecode`.
|
|
28
|
+
|
|
29
|
+
### 简体中文
|
|
30
|
+
- **PDF 格式与文字编码兼容性扩展**:
|
|
31
|
+
- 支持 `/Pages` 树向子页面继承 `/MediaBox`、`/CropBox`、`/Rotate` 与 `/Resources`。
|
|
32
|
+
- 支持 `/Subtype /Form` 表单 XObject 递归解释执行及局部变换矩阵与字体栈叠加。
|
|
33
|
+
- 实现纯 Rust 版 `LZWDecode` 解压器(支持 9–12 位可变码长与 `EarlyChange`)。
|
|
34
|
+
- 完善 `Type0`/CID `/W` 宽度解析、数组型 `beginbfrange` `ToUnicode` CMap、`MacRomanEncoding` 与 70+ Adobe Glyph 字符名映射。
|
|
35
|
+
- 增加 `/Rotate`(0°/90°/180°/270°)页面旋转坐标归一化与无锁 `DashMap` 字体缓存。
|
|
36
|
+
- **乱码治理与噪声过滤**:
|
|
37
|
+
- 新增 `TextHygieneConfig` 与 `HygieneStats` 诊断统计(`clean_mojibake`、`dehyphenate`、`dedup_faux_bold`、`ignore_invisible_text`、`strip_watermarks`、`dedup_running_headers`)。
|
|
38
|
+
- 自动过滤 `3 Tr` 隐形 OCR 层、斜向/低透明度水印、不可映射 PUA/控制符乱码以及伪粗体(Faux-Bold)重影。
|
|
39
|
+
- 支持连字展开、PUA 项目符号归一化、换行断词拼接及相邻同 URI 超链接 Span 合并。
|
|
40
|
+
- **空间算法与性能优化**:
|
|
41
|
+
- 引入二维 `SpatialGrid` 空间网格索引,将链接命中与表格单元匹配降至均摊 $O(1)$。
|
|
42
|
+
- 引入并查集 `UnionFind` 连通分量算法,精准分离单页多张独立表格并高效聚类矢量图形。
|
|
43
|
+
- **仓库结构精简**:
|
|
44
|
+
- 移除前端展示代码,将纯 Rust 工具包与 CLI 主程序置于仓库根目录。
|
|
45
|
+
|
|
46
|
+
---
|
|
47
|
+
|
|
48
|
+
## [0.1.0] - 2026-11-16
|
|
49
|
+
|
|
50
|
+
### English
|
|
51
|
+
- Initial pure-Rust `pymupdf4llm-rs` crate with PDF 1.0–2.0 parser, `FlateDecode`, recursive X-Y Cut multi-column ordering, `IdentifyHeaders` / `TocHeaders`, 4 table strategies, Tokio + Rayon concurrency, and CLI binary.
|
|
52
|
+
|
|
53
|
+
### 简体中文
|
|
54
|
+
- 初始版本发布:100% 纯 Rust `pymupdf4llm-rs` 核心库,支持 PDF 1.0–2.0 解析、`FlateDecode`、递归 X-Y Cut 多栏切分、`IdentifyHeaders`/`TocHeaders`、4 种表格策略、Tokio + Rayon 并发调度与 CLI 工具。
|
|
@@ -0,0 +1,46 @@
|
|
|
1
|
+
# Contributing to pymupdf4llm-rs / 参与贡献指南
|
|
2
|
+
|
|
3
|
+
Thank you for contributing to `pymupdf4llm-rs`! Bug reports, PDF compatibility fixtures, performance optimizations, and documentation improvements are welcome in both **English** and **简体中文**.
|
|
4
|
+
|
|
5
|
+
感谢您关注并参与 `pymupdf4llm-rs`!欢迎提交 Bug 反馈、疑难 PDF 兼容性测试用例、性能优化以及文档完善(支持**英文**或**简体中文**)。
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## 1. Development Setup / 本地开发环境
|
|
10
|
+
|
|
11
|
+
- Rust `1.78+` (Edition 2021)
|
|
12
|
+
- 100% pure-Rust dependencies only (`no C/C++ FFI`) / 仅允许纯 Rust 依赖,严禁引入 C/C++ FFI
|
|
13
|
+
|
|
14
|
+
```bash
|
|
15
|
+
cargo fmt --all -- --check
|
|
16
|
+
cargo clippy --all-targets --all-features -- -D warnings
|
|
17
|
+
cargo test --all-features
|
|
18
|
+
```
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## 2. Coding Guidelines / 开发规范
|
|
23
|
+
|
|
24
|
+
1. **Zero C/C++ FFI / 零外部原生依赖**:Keep all parsing, decompression, and layout code in safe pure Rust.
|
|
25
|
+
2. **Algorithmic Complexity / 算法复杂度控制**:Use `SpatialGrid` and `UnionFind` to avoid $O(N^2)$ scans on dense PDF pages.
|
|
26
|
+
3. **Tests / 补充测试**:Add unit or integration tests in `tests/compatibility_tests.rs` for any new PDF syntax, font encoding, or hygiene rule.
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## 3. Commit Convention / 提交信息规范
|
|
31
|
+
|
|
32
|
+
We follow [Conventional Commits](https://www.conventionalcommits.org/):
|
|
33
|
+
|
|
34
|
+
- `feat(pdf): add RunLengthDecode stream filter / 新增 RunLengthDecode 流解压`
|
|
35
|
+
- `fix(hygiene): preserve compound hyphenated terms across line wraps / 修复跨行复合词断字拼接误伤`
|
|
36
|
+
- `perf(layout): optimize SpatialGrid cell sizing / 优化 SpatialGrid 网格粒度`
|
|
37
|
+
- `docs: update README.zh-CN.md / 更新中文文档`
|
|
38
|
+
|
|
39
|
+
---
|
|
40
|
+
|
|
41
|
+
## 4. Pull Request Checklist / PR 自检清单
|
|
42
|
+
|
|
43
|
+
- [ ] `cargo fmt --all -- --check` passes.
|
|
44
|
+
- [ ] `cargo clippy --all-targets --all-features -- -D warnings` passes.
|
|
45
|
+
- [ ] `cargo test --all-features` passes.
|
|
46
|
+
- [ ] Bilingual docs (`README.md` / `README.zh-CN.md` / `CHANGELOG.md`) updated if public API changed.
|