scavio 0.18.0 → 0.18.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +14 -10
- package/dist/index.cjs +14 -1
- package/dist/index.cjs.map +1 -1
- package/dist/index.js +14 -1
- package/dist/index.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# Scavio
|
|
2
2
|
|
|
3
|
-
TypeScript SDK for the [Scavio API](https://scavio.dev) — real-time web scraping
|
|
3
|
+
TypeScript SDK for the [Scavio API](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — real-time web scraping
|
|
4
4
|
and data extraction across 32 platforms on one API key, plus `extract()` to read
|
|
5
5
|
any URL as clean Markdown.
|
|
6
6
|
|
|
@@ -66,6 +66,10 @@ limit on simultaneous in-flight requests: 1 on free and pay-as-you-go, 5 on
|
|
|
66
66
|
Project, 10 on Bootstrap, 15 on Startup, 50 on Growth, and unlimited on
|
|
67
67
|
Enterprise.
|
|
68
68
|
|
|
69
|
+
Every request carries `X-Client-Source: scavio-js`. From Node.js (and other
|
|
70
|
+
server runtimes) it also sends `User-Agent: scavio-js/<version>` (since 0.18.1);
|
|
71
|
+
browsers do not allow a script to set `User-Agent`, so it is omitted there.
|
|
72
|
+
|
|
69
73
|
## API Reference
|
|
70
74
|
|
|
71
75
|
### Extract (any URL)
|
|
@@ -1036,24 +1040,24 @@ MIT
|
|
|
1036
1040
|
|
|
1037
1041
|
## About Scavio
|
|
1038
1042
|
|
|
1039
|
-
[Scavio](https://scavio.dev) is a unified web data and
|
|
1040
|
-
[search API for AI agents](https://scavio.dev/search-api-for-ai-agents) — one API
|
|
1043
|
+
[Scavio](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) is a unified web data and
|
|
1044
|
+
[search API for AI agents](https://scavio.dev/search-api-for-ai-agents?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — one API
|
|
1041
1045
|
key, structured JSON, no proxies or browser farms to run. It is a real-time
|
|
1042
|
-
[Tavily alternative](https://scavio.dev/alternatives/tavily) and
|
|
1043
|
-
[SerpAPI alternative](https://scavio.dev/alternatives/serpapi), and with
|
|
1046
|
+
[Tavily alternative](https://scavio.dev/alternatives/tavily?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) and
|
|
1047
|
+
[SerpAPI alternative](https://scavio.dev/alternatives/serpapi?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), and with
|
|
1044
1048
|
`extract()` it also covers the read-any-URL job people reach for Firecrawl to do.
|
|
1045
1049
|
|
|
1046
1050
|
What teams build on it:
|
|
1047
1051
|
|
|
1048
|
-
- **SERP and answer engines** — [Google Search API](https://scavio.dev/google-search-api) for organic results, news, images, maps and the knowledge graph
|
|
1049
|
-
- **Price and catalog monitoring** — [Amazon Product API](https://scavio.dev/amazon-product-api), [Walmart Product API](https://scavio.dev/walmart-product-api), eBay, Target and Home Depot product, review and seller data
|
|
1052
|
+
- **SERP and answer engines** — [Google Search API](https://scavio.dev/google-search-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) for organic results, news, images, maps and the knowledge graph
|
|
1053
|
+
- **Price and catalog monitoring** — [Amazon Product API](https://scavio.dev/amazon-product-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [Walmart Product API](https://scavio.dev/walmart-product-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), eBay, Target and Home Depot product, review and seller data
|
|
1050
1054
|
- **Real estate and travel pipelines** — Zillow and Redfin listings and market stats, Booking, Airbnb and Tripadvisor rates and reviews
|
|
1051
1055
|
- **Review mining and competitive research** — Yelp, G2, Capterra, Glassdoor, App Store and Google Play reviews on one shape
|
|
1052
1056
|
- **Recruiting and company intelligence** — Indeed jobs, Glassdoor salaries, SEC EDGAR filings and XBRL facts, UK Companies House officers and filing history
|
|
1053
1057
|
- **Ad and creative intelligence** — Google Ads Transparency Center and Meta Ad Library creatives
|
|
1054
|
-
- **Social listening** — [YouTube API](https://scavio.dev/youtube-transcript-api), [TikTok API](https://scavio.dev/tiktok-api), [Instagram API](https://scavio.dev/instagram-api), [Reddit API](https://scavio.dev/reddit-api), [X API](https://scavio.dev/docs/x-search), [LinkedIn API](https://scavio.dev/docs/linkedin-person), Threads and Kuaishou
|
|
1058
|
+
- **Social listening** — [YouTube API](https://scavio.dev/youtube-transcript-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [TikTok API](https://scavio.dev/tiktok-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [Instagram API](https://scavio.dev/instagram-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [Reddit API](https://scavio.dev/reddit-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [X API](https://scavio.dev/docs/x-search?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [LinkedIn API](https://scavio.dev/docs/linkedin-person?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), Threads and Kuaishou
|
|
1055
1059
|
- **RAG ingestion** — `extract()` reads any URL and hands back readability Markdown ready to chunk and embed
|
|
1056
1060
|
|
|
1057
|
-
Teams choosing between providers can [compare Scavio vs alternatives](https://scavio.dev/compare) side by side.
|
|
1061
|
+
Teams choosing between providers can [compare Scavio vs alternatives](https://scavio.dev/compare?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) side by side.
|
|
1058
1062
|
|
|
1059
|
-
Get a free [API key](https://dashboard.scavio.dev/sign-up) and explore the [documentation](https://scavio.dev/docs/introduction).
|
|
1063
|
+
Get a free [API key](https://dashboard.scavio.dev/sign-up?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) and explore the [documentation](https://scavio.dev/docs/introduction?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme).
|
package/dist/index.cjs
CHANGED
|
@@ -199,12 +199,25 @@ function parseRetryAfter(header) {
|
|
|
199
199
|
var BASE_URL = "https://api.scavio.dev";
|
|
200
200
|
var DEFAULT_TIMEOUT = 3e4;
|
|
201
201
|
var DEFAULT_MAX_RETRIES = 2;
|
|
202
|
+
var SDK_VERSION = true ? "0.18.1" : "0.0.0";
|
|
203
|
+
var USER_AGENT = `scavio-js/${SDK_VERSION}`;
|
|
204
|
+
function isServerRuntime() {
|
|
205
|
+
const g = globalThis;
|
|
206
|
+
if (typeof g.window !== "undefined" && typeof g.document !== "undefined") {
|
|
207
|
+
return false;
|
|
208
|
+
}
|
|
209
|
+
return typeof g.process?.versions?.node === "string";
|
|
210
|
+
}
|
|
202
211
|
function buildHeaders(apiKey) {
|
|
203
|
-
|
|
212
|
+
const headers = {
|
|
204
213
|
Authorization: `Bearer ${apiKey}`,
|
|
205
214
|
"Content-Type": "application/json",
|
|
206
215
|
"X-Client-Source": "scavio-js"
|
|
207
216
|
};
|
|
217
|
+
if (isServerRuntime()) {
|
|
218
|
+
headers["User-Agent"] = USER_AGENT;
|
|
219
|
+
}
|
|
220
|
+
return headers;
|
|
208
221
|
}
|
|
209
222
|
function stripUndefined(obj) {
|
|
210
223
|
const result = {};
|