mdka 2.1.0 → 2.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +175 -0
- package/package.json +1 -1
package/README.md
ADDED
|
@@ -0,0 +1,175 @@
|
|
|
1
|
+
# mdka
|
|
2
|
+
|
|
3
|
+
**A HTML to Markdown converter written in Rust.**
|
|
4
|
+
|
|
5
|
+
[](https://crates.io/crates/mdka)
|
|
6
|
+
[](https://www.npmjs.com/package/mdka)
|
|
7
|
+
[](https://www.pypi.org/project/mdka)
|
|
8
|
+
[](https://github.com/nabbisen/mdka-rs/blob/main/LICENSE)
|
|
9
|
+
|
|
10
|
+
[](https://docs.rs/mdka)
|
|
11
|
+
[](https://deps.rs/crate/mdka)
|
|
12
|
+
[](https://github.com/nabbisen/mdka-rs/actions/workflows/release-executable.yaml)
|
|
13
|
+
[](https://github.com/nabbisen/mdka-rs/actions/workflows/release-npm.yaml)
|
|
14
|
+
[](https://github.com/nabbisen/mdka-rs/actions/workflows/release-pypi.yaml)
|
|
15
|
+
|
|
16
|
+
mdka balances conversion quality with runtime efficiency —
|
|
17
|
+
readable output from real-world HTML, without sacrificing speed or memory.
|
|
18
|
+
"ka" means "化 (か)" pointing to conversion.
|
|
19
|
+
|
|
20
|
+
---
|
|
21
|
+
|
|
22
|
+
## Why mdka?
|
|
23
|
+
|
|
24
|
+
There are several good HTML-to-Markdown converters in the Rust ecosystem.
|
|
25
|
+
mdka's specific focus is:
|
|
26
|
+
|
|
27
|
+
- **Reliable output from diverse HTML sources.**
|
|
28
|
+
It is built on [scraper](https://crates.io/crates/scraper), which uses
|
|
29
|
+
[html5ever](https://github.com/servo/html5ever) — the HTML5 parser from
|
|
30
|
+
the Servo browser engine. html5ever applies the same parsing algorithm that
|
|
31
|
+
web browsers use, so it handles malformed tags, deeply nested structures,
|
|
32
|
+
CMS output, and SPA-rendered DOM without special-casing.
|
|
33
|
+
- **Crash resistance.**
|
|
34
|
+
Conversion uses non-recursive DFS throughout. There is no stack overflow,
|
|
35
|
+
no matter the nesting depth.
|
|
36
|
+
- **Configurable pre-processing.**
|
|
37
|
+
Five [conversion modes](#conversion-modes) let you tune what gets kept or
|
|
38
|
+
stripped — from noise-free LLM input to lossless archiving.
|
|
39
|
+
- **Multi-language.**
|
|
40
|
+
The same Rust implementation is accessible from Node.js (napi-rs) and
|
|
41
|
+
Python (PyO3).
|
|
42
|
+
|
|
43
|
+
---
|
|
44
|
+
|
|
45
|
+
## Quick Start
|
|
46
|
+
|
|
47
|
+
### Try it from the command line
|
|
48
|
+
|
|
49
|
+
`cargo` (Rust language) installed is required.
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
cargo install mdka-cli
|
|
53
|
+
|
|
54
|
+
echo '<h1>Hello</h1><p><strong>world</strong></p>' | mdka
|
|
55
|
+
# # Hello
|
|
56
|
+
#
|
|
57
|
+
# **world**
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
```bash
|
|
61
|
+
mdka page.html # → page.md (same directory)
|
|
62
|
+
mdka --mode minimal --drop-shell *.html # strip nav/header/footer
|
|
63
|
+
mdka --help # full option list
|
|
64
|
+
```
|
|
65
|
+
|
|
66
|
+
### Add to a Rust project
|
|
67
|
+
|
|
68
|
+
```toml
|
|
69
|
+
# Cargo.toml
|
|
70
|
+
[dependencies]
|
|
71
|
+
mdka = "2"
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
```rust
|
|
75
|
+
use mdka::html_to_markdown;
|
|
76
|
+
|
|
77
|
+
let md = html_to_markdown("<h1>Hello</h1><p><em>world</em></p>");
|
|
78
|
+
// "# Hello\n\n*world*\n"
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
With options:
|
|
82
|
+
|
|
83
|
+
```rust
|
|
84
|
+
use mdka::{html_to_markdown_with};
|
|
85
|
+
use mdka::options::{ConversionMode, ConversionOptions};
|
|
86
|
+
|
|
87
|
+
let mut opts = ConversionOptions::for_mode(ConversionMode::Minimal);
|
|
88
|
+
opts.drop_interactive_shell = true;
|
|
89
|
+
let md = html_to_markdown_with(html, &opts);
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### Add to a Node.js project
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
npm install mdka
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
```js
|
|
99
|
+
const { htmlToMarkdown, htmlToMarkdownWith } = require('mdka')
|
|
100
|
+
|
|
101
|
+
const md = htmlToMarkdown('<h1>Hello</h1>')
|
|
102
|
+
|
|
103
|
+
const md = await htmlToMarkdownWithAsync(html, {
|
|
104
|
+
mode: 'minimal',
|
|
105
|
+
dropInteractiveShell: true,
|
|
106
|
+
})
|
|
107
|
+
```
|
|
108
|
+
|
|
109
|
+
### Add to a Python project
|
|
110
|
+
|
|
111
|
+
```bash
|
|
112
|
+
pip install mdka
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
```python
|
|
116
|
+
import mdka
|
|
117
|
+
|
|
118
|
+
md = mdka.html_to_markdown('<h1>Hello</h1>')
|
|
119
|
+
|
|
120
|
+
md = mdka.html_to_markdown_with(
|
|
121
|
+
html,
|
|
122
|
+
mode=mdka.ConversionMode.Minimal,
|
|
123
|
+
drop_interactive_shell=True,
|
|
124
|
+
)
|
|
125
|
+
```
|
|
126
|
+
|
|
127
|
+
---
|
|
128
|
+
|
|
129
|
+
## Conversion Modes
|
|
130
|
+
|
|
131
|
+
| Mode | Use when |
|
|
132
|
+
|---|---|
|
|
133
|
+
| `Balanced` | General use — default |
|
|
134
|
+
| `Strict` | Debugging, diff comparison |
|
|
135
|
+
| `Minimal` | LLM input, text extraction |
|
|
136
|
+
| `Semantic` | SPA content, ARIA-aware pipelines |
|
|
137
|
+
| `Preserve` | Archiving, audit trails |
|
|
138
|
+
|
|
139
|
+
---
|
|
140
|
+
|
|
141
|
+
## Learn More
|
|
142
|
+
|
|
143
|
+
Full documentation lives in the [`docs/`](./docs/) folder, published as GitHub Pages.
|
|
144
|
+
|
|
145
|
+
https://nabbisen.github.io/mdka-rs/
|
|
146
|
+
|
|
147
|
+
| Topic | Link |
|
|
148
|
+
|---|---|
|
|
149
|
+
| Installation | [/getting-started/installation](https://nabbisen.github.io/mdka-rs/getting-started/installation) |
|
|
150
|
+
| Rust Usage & Examples | [/getting-started/usage-rust](https://nabbisen.github.io/mdka-rs/getting-started/usage-rust) |
|
|
151
|
+
| Node.js Usage | [/getting-started/usage-nodejs](https://nabbisen.github.io/mdka-rs/getting-started/usage-nodejs) |
|
|
152
|
+
| Python Usage | [/getting-started/usage-python](https://nabbisen.github.io/mdka-rs/getting-started/usage-python) |
|
|
153
|
+
| CLI Reference | [/getting-started/usage-cli](https://nabbisen.github.io/mdka-rs/getting-started/usage-cli) |
|
|
154
|
+
| API Reference | [/api/index](https://nabbisen.github.io/mdka-rs/api/index) |
|
|
155
|
+
| Conversion Modes | [/api/modes](https://nabbisen.github.io/mdka-rs/api/modes) |
|
|
156
|
+
| ConversionOptions | [/api/options](https://nabbisen.github.io/mdka-rs/api/options) |
|
|
157
|
+
| Supported Elements | [/api/elements](https://nabbisen.github.io/mdka-rs/api/elements) |
|
|
158
|
+
| Design Philosophy | [/design/philosophy](https://nabbisen.github.io/mdka-rs/design/philosophy) |
|
|
159
|
+
| Performance Characteristics | [/design/performance-characteristics](https://nabbisen.github.io/mdka-rs/design/performance-characteristics) |
|
|
160
|
+
| Architecture | [/design/architecture](https://nabbisen.github.io/mdka-rs/design/architecture) |
|
|
161
|
+
| Features | [/design/features](https://nabbisen.github.io/mdka-rs/design/features) |
|
|
162
|
+
|
|
163
|
+
---
|
|
164
|
+
|
|
165
|
+
## Open-source, with care
|
|
166
|
+
|
|
167
|
+
This project is lovingly built and maintained by volunteers.
|
|
168
|
+
We hope it helps streamline your work.
|
|
169
|
+
Please understand that the project has its own direction — while we welcome feedback, it might not fit every edge case 🌱
|
|
170
|
+
|
|
171
|
+
## Acknowledgements
|
|
172
|
+
|
|
173
|
+
Depends on [scraper](https://crates.io/crates/scraper) (+ [html5ever](https://github.com/servo/html5ever)), [ego-tree](https://crates.io/crates/ego-tree), [rayon](https://crates.io/crates/rayon), [tikv-jemallocator](https://crates.io/crates/tikv-jemallocator) / [tikv-jemalloc-ctl](https://crates.io/crates/tikv-jemalloc-ctl), [thiserror](https://crates.io/crates/thiserror).
|
|
174
|
+
|
|
175
|
+
Also, [napi-rs](https://github.com/napi-rs/napi-rs) on binding for Node.js and PyO3's [pyo3](https://github.com/PyO3/pyo3) / [maturin](https://github.com/PyO3/maturin) on bindings for Python.
|
package/package.json
CHANGED