minifetch-api 1.0.0 → 1.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +17 -15
- package/package.json +4 -3
package/README.md
CHANGED
|
@@ -1,9 +1,11 @@
|
|
|
1
1
|
# Minifetch API
|
|
2
2
|
<div align=center>
|
|
3
|
-
<
|
|
3
|
+
<a href="https://minifetch.com">
|
|
4
|
+
<img src="https://minifetch.com/minifetch-dog-logo--whitebg.png" width="70" />
|
|
5
|
+
</a>
|
|
4
6
|
</div>
|
|
5
7
|
|
|
6
|
-
**Fetch & extract data from web pages.** [Minifetch.com](https://minifetch.com) provides composable extraction APIs that humans and AI Agents can autonomously discover, orchestrate, and pay for— making web pages simple to access.
|
|
8
|
+
**Fetch & extract data from web pages.** [Minifetch.com](https://minifetch.com) provides composable extraction APIs that humans and AI Agents can autonomously discover, orchestrate, and pay for— making web pages simple to access. Pay-as-you-go at competitive prices.
|
|
7
9
|
|
|
8
10
|
**Full [API docs are here](https://minifetch.com/docs/api)**, including example response data & prices. For AI Agents, read the [LLMs.txt](https://minifetch.com/llms.txt).
|
|
9
11
|
|
|
@@ -53,30 +55,30 @@ After you initialize the client (above), you have the following methods availabl
|
|
|
53
55
|
|
|
54
56
|
Requests exceeding rate limits return `503` errors. This is intentional — back off and retry, max 5-10 queries per second. We'll add bulk fetch and extract in the future. Sign up for the [waitlist here](https://forms.gle/rkMi7T23bHJc8XFw9).
|
|
55
57
|
|
|
56
|
-
-
|
|
58
|
+
- **`client.checkAndExtractUrlPreview(url)`**
|
|
57
59
|
- Extracts a light, token-efficient preview of a URL: title, description, and image (only).
|
|
58
60
|
|
|
59
|
-
-
|
|
60
|
-
- Options: `includeMediaUrls: true`, defaults to false
|
|
61
|
+
- **`client.checkAndExtractUrlContent(url, options)`**
|
|
61
62
|
- Extracts a clean, token-efficient content summary as markdown from a URL. Removes ads, nav, scripts. Much more efficient than raw HTML fetches for LLMs.
|
|
63
|
+
- Options: `{ includeMediaUrls: true }`, defaults to false
|
|
62
64
|
|
|
63
|
-
-
|
|
65
|
+
- **`client.checkAndExtractUrlLinks(url)`**
|
|
64
66
|
- Extracts all links from a URL categorized by type (internal/external/anchor) with SEO metadata. Detects image links, nofollow attributes, and analyzes external domain distribution.
|
|
65
67
|
|
|
66
|
-
-
|
|
67
|
-
- Options: `includeResponseBody: true`, defaults to false
|
|
68
|
+
- **`client.checkAndExtractUrlMetadata(url, options)`**
|
|
68
69
|
- Extracts rich structured metadata from a URL: title, description, og/twitter tags, json-ld, images, headings, response headers, and more. Perfect for SEO research or metadata indexing & analysis. Largest response size of the client methods available.
|
|
70
|
+
- Options: `{ includeResponseBody: true }`, defaults to false
|
|
69
71
|
|
|
70
72
|
For maximum control, you can also use the following methods directly:
|
|
71
73
|
|
|
72
|
-
-
|
|
74
|
+
- **`client.preflightCheck(url)`**
|
|
73
75
|
- Standalone call to check if a URL is allowed to be fetched according to the website's robots.txt file. Use this free endpoint before using our other paid endpoints to avoid spending extra on un-fetchable URLs
|
|
74
|
-
-
|
|
75
|
-
-
|
|
76
|
-
- Options: `includeMediaUrls: true`, defaults to false
|
|
77
|
-
-
|
|
78
|
-
-
|
|
79
|
-
- Options: `includeResponseBody: true`, defaults to false
|
|
76
|
+
- **`client.extractUrlPreview(url)`**
|
|
77
|
+
- **`client.extractUrlContent(url, options)`**
|
|
78
|
+
- Options: `{ includeMediaUrls: true }`, defaults to false
|
|
79
|
+
- **`client.extractUrlLinks(url)`**
|
|
80
|
+
- **`client.extractUrlMetadata(url, options)`**
|
|
81
|
+
- Options: `{ includeResponseBody: true }`, defaults to false
|
|
80
82
|
|
|
81
83
|
### Error Types
|
|
82
84
|
When you wrap the functions above in a try/catch, here are the errors you may encounter:
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "minifetch-api",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.1",
|
|
4
4
|
"description": "Minifetch.com API Client. Fetch & extract data from web pages. For AI Agents & SEO research.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/esm/index.js",
|
|
@@ -34,7 +34,7 @@
|
|
|
34
34
|
"bugs": {
|
|
35
35
|
"url": "https://github.com/Niche-Networks/minifetch-api/issues"
|
|
36
36
|
},
|
|
37
|
-
"homepage": "https://minifetch.com
|
|
37
|
+
"homepage": "https://minifetch.com",
|
|
38
38
|
"repository": {
|
|
39
39
|
"type": "git",
|
|
40
40
|
"url": "git@github.com:Niche-Networks/minifetch-api.git"
|
|
@@ -53,7 +53,8 @@
|
|
|
53
53
|
"test:debug": "vitest run test/debug.test.ts",
|
|
54
54
|
"test:coverage": "vitest run --coverage",
|
|
55
55
|
"example:ts": "cd example-typescript && pnpm install && pnpm start",
|
|
56
|
-
"example:js": "cd example-javascript && pnpm install && pnpm start"
|
|
56
|
+
"example:js": "cd example-javascript && pnpm install && pnpm start",
|
|
57
|
+
"prepublishOnly": "pnpm run build"
|
|
57
58
|
},
|
|
58
59
|
"dependencies": {
|
|
59
60
|
"@solana/kit": "^5.4.0",
|