@effortlessmotion/pdf-to-md 0.2.1 → 0.2.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/package.json +2 -3
- package/skills/pdf-to-md/SKILL.md +0 -75
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@effortlessmotion/pdf-to-md",
|
|
3
|
-
"version": "0.2.
|
|
3
|
+
"version": "0.2.2",
|
|
4
4
|
"description": "Extract PDFs to structured markdown: text layer via pdf.js, OCR fallback for image-only pages, pure npm and cross-platform",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pdf",
|
|
@@ -32,8 +32,7 @@
|
|
|
32
32
|
"pdf-to-md": "bin/cli.js"
|
|
33
33
|
},
|
|
34
34
|
"files": [
|
|
35
|
-
"dist"
|
|
36
|
-
"skills"
|
|
35
|
+
"dist"
|
|
37
36
|
],
|
|
38
37
|
"scripts": {
|
|
39
38
|
"build": "tsds build",
|
|
@@ -1,75 +0,0 @@
|
|
|
1
|
-
---
|
|
2
|
-
name: pdf-to-md
|
|
3
|
-
description: "Convert PDFs to Markdown with pdf-to-md, investigate missing or misordered text, and verify conversions against source pages and extraction evidence. Use for conversion, targeted re-extraction, or quality checks of pdf-to-md output."
|
|
4
|
-
---
|
|
5
|
-
|
|
6
|
-
# pdf-to-md
|
|
7
|
-
|
|
8
|
-
pdf-to-md reads embedded text and uses local OCR where needed. A preservation audit checks extracted text against Markdown. Recognition accuracy, reading order, tables and diagrams require comparison with rendered source pages.
|
|
9
|
-
|
|
10
|
-
## Convert with evidence
|
|
11
|
-
|
|
12
|
-
```sh
|
|
13
|
-
pdf-to-md file.pdf out.md --json --page-markers --debug-words=words.jsonl
|
|
14
|
-
```
|
|
15
|
-
|
|
16
|
-
Page markers connect the output to its source pages. `--debug-words` captures text-layer items or recognized OCR words before layout reconstruction. Text items can contain multiple words. Parse the dump as JSON Lines; key order is not fixed.
|
|
17
|
-
|
|
18
|
-
`--ocr` forces recognition and `--no-ocr` retains the text path. `--pages N[-M]` selects source pages. A mixed document can use different extraction paths per page. Keep the input identity, tool version, effective options, report and dump with the run's working artifacts so later corrections use matching evidence.
|
|
19
|
-
|
|
20
|
-
## Find pages to inspect
|
|
21
|
-
|
|
22
|
-
Read `report.pageStats` and warnings:
|
|
23
|
-
|
|
24
|
-
- `danglingLong` identifies prose that may have broken at the wrong place.
|
|
25
|
-
- `reviewBlocks` counts OCR blocks whose structure could not be recovered. The text path does not provide this detector; zero is not a layout guarantee.
|
|
26
|
-
- `lowConfidenceWords` and `lowConfidenceSample` identify uncertain OCR tokens. Missing confidence data is not a high-confidence result.
|
|
27
|
-
- `junkHeadings` identifies possible font-size classification errors.
|
|
28
|
-
- `largeImages` on text pages identifies images that may contain additional text. Compare a targeted OCR pass before replacing usable embedded text.
|
|
29
|
-
|
|
30
|
-
Also inspect sparse or empty pages and any layout the detectors do not cover. An unflagged page can still have incorrect columns, character mappings, tables or missing visual material.
|
|
31
|
-
|
|
32
|
-
## Compare against the page
|
|
33
|
-
|
|
34
|
-
```sh
|
|
35
|
-
pdf-to-md render file.pdf N
|
|
36
|
-
```
|
|
37
|
-
|
|
38
|
-
The default render is 288 DPI; use `--out DIR` for its destination. Compare the rendered page with the output, including item numbers, captions, symbols and reading order. Word presence alone does not validate these relationships.
|
|
39
|
-
|
|
40
|
-
For an OCR comparison or tuning experiment:
|
|
41
|
-
|
|
42
|
-
```sh
|
|
43
|
-
pdf-to-md file.pdf page-N.md --pages N --ocr --dpi 400 --debug-words=page-N.words.jsonl
|
|
44
|
-
```
|
|
45
|
-
|
|
46
|
-
OCR can recover image text and also damage correct embedded text. Higher DPI can improve small labels and worsen ordinary prose. Compare the result before accepting it. Read the package's `QUALITY.md` for tuning controls; their effects apply to OCR geometry.
|
|
47
|
-
|
|
48
|
-
If a page needs manual repair, retain its source dump and page number. An image can preserve a diagram visually but does not recover editable text. Record unresolved character mappings or flattened tables alongside the affected content.
|
|
49
|
-
|
|
50
|
-
## Audit the final Markdown
|
|
51
|
-
|
|
52
|
-
```sh
|
|
53
|
-
pdf-to-md audit words.jsonl out.md
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
A successful audit with `MISSING=0` means the supplied extracted text was accounted for under the auditor's matching rules. Empty or malformed evidence is an error. It does not prove that the dump captured every visible word or that the result is an accurate transcription.
|
|
57
|
-
|
|
58
|
-
The auditor accepts `### pN` headings and `<!-- pN -->` comments. Audit the final file after edits and marker changes. Use the matching extraction dump for any pages replaced with a different extraction. Keep editorial additions separate while checking preservation, because added text can satisfy a missing occurrence.
|
|
59
|
-
|
|
60
|
-
Report preservation results, visual/structural checks, and unresolved issues separately. Avoid a blanket "lossless" claim based on a word audit.
|
|
61
|
-
|
|
62
|
-
## Programmatic repair
|
|
63
|
-
|
|
64
|
-
`--format raw` returns an `Analysis` with per-page source items, blocks and a report. Text pages contain PDF text runs; OCR pages contain recognized words. Boxes use page-normalized coordinates, origin bottom-left, y increasing upward. Block `wordIndexes` link to the page's source list. Preserve that evidence when reordering blocks; rebuilding the source list from corrected output would make the audit circular.
|
|
65
|
-
|
|
66
|
-
OCR uncertainty markers remain visible in Markdown:
|
|
67
|
-
|
|
68
|
-
```text
|
|
69
|
-
> [floats] ...
|
|
70
|
-
> [review: reason] ...
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
Moving or deleting these blocks can remove real content. Check their source pages before changing them.
|
|
74
|
-
|
|
75
|
-
PDF compression is a separate operation. If requested, retain an identifiable original and verify text and visual material independently before replacing it. Lossy image compression can affect subsequent OCR and diagram checks even when extracted text stays identical.
|