@effortlessmotion/pdf-to-md 0.1.7 → 0.1.8
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +69 -76
- package/package.json +2 -8
package/README.md
CHANGED
|
@@ -1,116 +1,109 @@
|
|
|
1
|
-
# pdf-to-md
|
|
1
|
+
# @effortlessmotion/pdf-to-md
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
> contract: recognized text is never silently lost, unrecoverable structure is marked rather
|
|
5
|
-
> than guessed, and every claim the output makes is mechanically checkable. When the defaults
|
|
6
|
-
> misread your document, you get the measured knobs, the detectors that point at exactly what to
|
|
7
|
-
> distrust, and the raw analysis to converge on a lossless conversion yourself.
|
|
3
|
+
Convert born-digital and scanned PDFs to Markdown. `pdf-to-md` reads an existing text layer first and falls back to local OCR for image-only pages. It returns a QA report and marks text it cannot place reliably instead of silently dropping it.
|
|
8
4
|
|
|
9
|
-
|
|
5
|
+
Use it when you need editable Markdown plus evidence about conversion quality. OCR runs locally and makes no cloud calls.
|
|
6
|
+
|
|
7
|
+
## Requirements
|
|
8
|
+
|
|
9
|
+
- Node.js 22.13 or newer
|
|
10
|
+
- English text for the bundled OCR model. Text-layer extraction can preserve other languages.
|
|
11
|
+
|
|
12
|
+
## Install
|
|
10
13
|
|
|
11
14
|
```bash
|
|
12
|
-
npm install pdf-to-md
|
|
15
|
+
npm install @effortlessmotion/pdf-to-md
|
|
13
16
|
```
|
|
14
17
|
|
|
15
|
-
|
|
18
|
+
## Convert a PDF
|
|
16
19
|
|
|
17
|
-
|
|
20
|
+
The package installs the `pdf-to-md` executable:
|
|
18
21
|
|
|
19
22
|
```bash
|
|
20
|
-
npx pdf-to-md book.pdf
|
|
21
|
-
npx pdf-to-md scan.pdf out.md --ocr # force the OCR path
|
|
23
|
+
npx pdf-to-md book.pdf
|
|
22
24
|
```
|
|
23
25
|
|
|
24
|
-
|
|
25
|
-
|
|
26
|
-
|
|
27
|
-
|
|
28
|
-
|
|
26
|
+
This writes `book.md`. If the PDF has too little embedded text, the command automatically uses OCR and reports that choice on stderr.
|
|
27
|
+
|
|
28
|
+
Specify an output path or force OCR when needed:
|
|
29
|
+
|
|
30
|
+
```bash
|
|
31
|
+
npx pdf-to-md scan.pdf notes.md --ocr
|
|
29
32
|
```
|
|
30
33
|
|
|
31
|
-
|
|
34
|
+
A conversion can produce ordinary Markdown plus visible review markers:
|
|
32
35
|
|
|
33
|
-
|
|
36
|
+
```markdown
|
|
37
|
+
## DAILY TALLIES
|
|
34
38
|
|
|
35
|
-
|
|
36
|
-
import { pdfToMarkdown } from 'pdf-to-md';
|
|
39
|
+
Every skiff was counted at the north pier.
|
|
37
40
|
|
|
38
|
-
|
|
41
|
+
> [review: unstructured labels] 12 47 380 5 91
|
|
39
42
|
```
|
|
40
43
|
|
|
41
|
-
|
|
44
|
+
The excerpt is illustrative. Review markers preserve text whose structure could not be recovered reliably.
|
|
42
45
|
|
|
43
|
-
|
|
46
|
+
## Library use
|
|
44
47
|
|
|
45
|
-
|
|
48
|
+
```js
|
|
49
|
+
import { pdfToMarkdown } from '@effortlessmotion/pdf-to-md';
|
|
46
50
|
|
|
47
|
-
|
|
51
|
+
const { markdown, report, warnings } = await pdfToMarkdown('book.pdf');
|
|
48
52
|
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
and exits non-zero on any loss (`MISSING > 0`) — a first-class part of the product, usable as a CI
|
|
54
|
-
gate, not a path into the source tree.
|
|
55
|
-
2. **Invisible corruption is minimized and made detectable.** Right-words-wrong-order failures
|
|
56
|
-
(column braiding, mid-sentence splits) are what the geometry works hardest to prevent; a second
|
|
57
|
-
class — right order, WRONG VALUE (an OCR misread spliced into otherwise word-like prose) — is
|
|
58
|
-
what confidence scoring catches. **Enforced by:** `report.danglingLong` (the order-failure
|
|
59
|
-
detector no output-only check can prove), `report.reviewBlocks` (blocks the geometry couldn't
|
|
60
|
-
order, labeled rather than guessed at), and `report.lowConfidenceWords` /
|
|
61
|
-
`pageStats[].lowConfidenceSample` (words the engine itself scored below confidence 60 — see
|
|
62
|
-
`LOW_CONFIDENCE_THRESHOLD`'s test in `test/unit/review-blocks.test.ts` for the measured derivation).
|
|
63
|
-
3. **Visible misplacement is acceptable.** A fragment in the floats aside, a `> [review: ...]`
|
|
64
|
-
block, or an odd paragraph break is something a caller can see and edit; the two classes above
|
|
65
|
-
are not. **Enforced by:** the `> [floats]` / `> [review: reason]` markers themselves — regular
|
|
66
|
-
enough (`^> \[(floats|review: [^\]]+)\] (.*)$`) for a downstream agent to find and handle
|
|
67
|
-
without guessing — surfaced in `--json` as `report.floats` / `report.reviewBlocks`.
|
|
53
|
+
console.log(markdown);
|
|
54
|
+
console.log(report.path); // "text" or "ocr"
|
|
55
|
+
console.warn(warnings);
|
|
56
|
+
```
|
|
68
57
|
|
|
69
|
-
|
|
58
|
+
`pdfToMarkdown()` uses the same text-first, automatic-OCR behavior as the CLI. It returns data and does not write files or exit the process. Conversion failures use `PdfToMdError` where the library assigns a package error code; native filesystem errors can also surface when inputs cannot be read.
|
|
70
59
|
|
|
71
|
-
|
|
60
|
+
Use `extractText()` or `extractOcr()` when you need to choose the path yourself. The [API documentation](https://kmalakoff.github.io/pdf-to-md/) lists the public functions, options and return types.
|
|
72
61
|
|
|
73
|
-
|
|
62
|
+
## Know which path ran
|
|
74
63
|
|
|
75
|
-
|
|
64
|
+
The QA report identifies the extraction path and highlights output that deserves review. Important cases:
|
|
76
65
|
|
|
77
|
-
|
|
66
|
+
- The bundled OCR model recognizes English. Use text-layer extraction for PDFs whose embedded text is in another language.
|
|
67
|
+
- A page can contain both embedded text and a large image with additional text. The text path warns about these hybrid pages so you can compare a targeted OCR pass.
|
|
68
|
+
- `> [floats]` and `> [review: ...]` blocks contain recognized text that the converter could not place confidently. They are intentionally visible.
|
|
69
|
+
- OCR quality depends on the scan. Check low-confidence words and rendered pages before relying on the result.
|
|
78
70
|
|
|
79
|
-
|
|
80
|
-
|---|---|---|
|
|
81
|
-
| `--float-max-words` | fragments with more words than this are body text, not decoration | `3` |
|
|
82
|
-
| `--float-max-width` | fraction of page width above which a fragment is too wide to be decoration | `0.15` |
|
|
83
|
-
| `--float-margin` | how far a fragment's left edge must sit from its column edge to be decoration | `0.008` |
|
|
84
|
-
| `--para-gap` | a vertical gap this many times the line pitch is a paragraph break | `1.6` |
|
|
85
|
-
| `--heading-scale` | lines taller than this times body height are headings | `2.0` |
|
|
86
|
-
| `--col-max-width` | boxes wider than this are excluded from gutter detection | `0.55` |
|
|
87
|
-
| `--col-split` | fractional x of the two-column midpoint | `0.5` |
|
|
88
|
-
| `--line-y-tol` | vertical distance within which two boxes are the same line | `0.006` |
|
|
89
|
-
| `--line-height-ratio` | same-line boxes with a larger height ratio don't merge | `2.4` |
|
|
90
|
-
| `--dpi` | OCR page-render resolution; raise for tiny print or chart/label pages, `>~400` measured worse on ordinary prose pages (`test/unit/sparse-retry.test.ts`) | `288` |
|
|
71
|
+
Use `--json` to print the report:
|
|
91
72
|
|
|
92
|
-
|
|
73
|
+
```bash
|
|
74
|
+
npx pdf-to-md book.pdf --json
|
|
75
|
+
```
|
|
93
76
|
|
|
94
|
-
|
|
77
|
+
See [quality checks and tuning](https://github.com/kmalakoff/pdf-to-md/blob/master/QUALITY.md) for the audit workflow, report fields and OCR controls.
|
|
95
78
|
|
|
96
|
-
##
|
|
79
|
+
## Common CLI options
|
|
97
80
|
|
|
98
|
-
```
|
|
99
|
-
|
|
81
|
+
```text
|
|
82
|
+
pdf-to-md <file.pdf> [out.md] [flags]
|
|
83
|
+
pdf-to-md render <file.pdf> <page>[-<page>]
|
|
84
|
+
pdf-to-md audit <words.jsonl> <file.md>
|
|
100
85
|
```
|
|
101
86
|
|
|
102
|
-
|
|
87
|
+
Common flags:
|
|
88
|
+
|
|
89
|
+
- `--ocr` forces OCR; `--no-ocr` disables automatic OCR fallback.
|
|
90
|
+
- `--pages N[-M]` converts a page range.
|
|
91
|
+
- `--stdout` writes the converted content to stdout.
|
|
92
|
+
- `--json` prints the QA report.
|
|
93
|
+
- `--format md|txt|raw` selects Markdown, plain text or analysis JSON.
|
|
94
|
+
- `--debug-words=FILE` writes recognized OCR words for `pdf-to-md audit`.
|
|
95
|
+
- `--dpi=N` changes OCR render resolution. The default is 288; higher values are not always more accurate.
|
|
96
|
+
|
|
97
|
+
Run `npx pdf-to-md --help` for the complete option list.
|
|
98
|
+
|
|
99
|
+
## Agent skill
|
|
103
100
|
|
|
104
|
-
|
|
101
|
+
An optional skill teaches compatible coding agents how to inspect the report, render suspect pages and run targeted conversions:
|
|
105
102
|
|
|
106
103
|
```bash
|
|
107
|
-
|
|
108
|
-
npm test # synthetic fixtures (ground truth by construction), a fetched+sha256-pinned
|
|
109
|
-
# public corpus (skips offline), real engine runs, plus self-skipping
|
|
110
|
-
# private-document extras when available locally
|
|
111
|
-
npm run prepublishOnly # tsds validate: build + biome + depcheck + docs
|
|
104
|
+
npx skills add kmalakoff/pdf-to-md
|
|
112
105
|
```
|
|
113
106
|
|
|
114
|
-
|
|
107
|
+
## Support and license
|
|
115
108
|
|
|
116
|
-
|
|
109
|
+
Report problems through the [GitHub issue tracker](https://github.com/kmalakoff/pdf-to-md/issues). See [CONTRIBUTING.md](https://github.com/kmalakoff/pdf-to-md/blob/master/CONTRIBUTING.md) to work on the package. The package is licensed under the [MIT License](https://github.com/kmalakoff/pdf-to-md/blob/master/LICENSE).
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@effortlessmotion/pdf-to-md",
|
|
3
|
-
"version": "0.1.
|
|
3
|
+
"version": "0.1.8",
|
|
4
4
|
"description": "Extract PDFs to structured markdown: text layer via pdf.js, OCR fallback for image-only pages, pure npm and cross-platform",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"pdf",
|
|
@@ -67,12 +67,6 @@
|
|
|
67
67
|
"access": "public"
|
|
68
68
|
},
|
|
69
69
|
"allowScripts": {
|
|
70
|
-
"tesseract.js": true
|
|
71
|
-
"core-js-pure": true,
|
|
72
|
-
"@swc/core": true,
|
|
73
|
-
"node-filename-to-dist-paths": true,
|
|
74
|
-
"node-semvers": true,
|
|
75
|
-
"node-version-use": true,
|
|
76
|
-
"thread-sleep-compat": true
|
|
70
|
+
"tesseract.js": true
|
|
77
71
|
}
|
|
78
72
|
}
|