pi-smart-fetch 0.2.34 → 0.2.36
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +2 -0
- package/dist/index.js +1046 -90
- package/dist/index.js.map +1 -1
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -11,6 +11,7 @@
|
|
|
11
11
|
- 🧠 **Useful metadata** — title, author, site, language, published date when available
|
|
12
12
|
- 📦 **Downloads + large file support** — stream attachments and binaries to temp files
|
|
13
13
|
- 🔁 **Client-side `<meta>` redirects** — follows sane meta refresh redirects with loop limits
|
|
14
|
+
- 🔗 **Alternate content fallback** — when extraction produces no/thin content, follows qualified `<link rel="alternate" type="...">` entries in `<head>` that match the requested output format
|
|
14
15
|
- ⚡ **Batch fetch** — fetch many URLs with bounded concurrency
|
|
15
16
|
- 📝 **Multiple output formats** — `markdown`, `html`, `text`, `json`
|
|
16
17
|
|
|
@@ -29,6 +30,7 @@ This package works on general web pages, but some site types benefit especially
|
|
|
29
30
|
Notes:
|
|
30
31
|
- Defuddle is the cleanup layer: it strips common page chrome like nav, sidebars, related links, share widgets, and footers
|
|
31
32
|
- It does **not** execute JavaScript or solve interactive anti-bot/login flows
|
|
33
|
+
- If an HTML shell advertises alternate content in `<head>`, smart-fetch can follow matching alternates such as `text/markdown`, `text/plain`, `text/html`, or JSON media types according to the requested `format`
|
|
32
34
|
|
|
33
35
|
## Install
|
|
34
36
|
|