scavio 0.18.0 → 0.19.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +63 -13
- package/dist/index.cjs +91 -1
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +143 -1
- package/dist/index.d.ts +143 -1
- package/dist/index.js +90 -1
- package/dist/index.js.map +1 -1
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# Scavio
|
|
2
2
|
|
|
3
|
-
TypeScript SDK for the [Scavio API](https://scavio.dev) — real-time web scraping
|
|
4
|
-
and data extraction across
|
|
3
|
+
TypeScript SDK for the [Scavio API](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — real-time web scraping
|
|
4
|
+
and data extraction across 33 platforms on one API key, plus `extract()` to read
|
|
5
5
|
any URL as clean Markdown.
|
|
6
6
|
|
|
7
7
|
Structured JSON in, structured JSON out. No proxies, no headless browsers, no
|
|
@@ -10,7 +10,7 @@ per-site parsers to maintain.
|
|
|
10
10
|
- **Search and SERP** — Google (organic, news, maps, shopping, flights, hotels, trends, AI Mode)
|
|
11
11
|
- **E-commerce** — Amazon, Walmart, Costco, eBay, Target, Home Depot, TikTok Shop
|
|
12
12
|
- **Real estate and travel** — Zillow, Redfin, Booking, Airbnb, Tripadvisor
|
|
13
|
-
- **Reviews and local** — Yelp, G2, Capterra, Glassdoor, App Store, Google Play
|
|
13
|
+
- **Reviews and local** — Trustpilot, Yelp, G2, Capterra, Glassdoor, App Store, Google Play
|
|
14
14
|
- **Jobs and companies** — Indeed, Glassdoor, SEC EDGAR, Companies House
|
|
15
15
|
- **Ads transparency** — Google Ads Transparency Center, Meta Ad Library
|
|
16
16
|
- **Social and video** — YouTube, TikTok, Instagram, Threads, X, LinkedIn, Reddit, Kuaishou
|
|
@@ -66,6 +66,10 @@ limit on simultaneous in-flight requests: 1 on free and pay-as-you-go, 5 on
|
|
|
66
66
|
Project, 10 on Bootstrap, 15 on Startup, 50 on Growth, and unlimited on
|
|
67
67
|
Enterprise.
|
|
68
68
|
|
|
69
|
+
Every request carries `X-Client-Source: scavio-js`. From Node.js (and other
|
|
70
|
+
server runtimes) it also sends `User-Agent: scavio-js/<version>` (since 0.18.1);
|
|
71
|
+
browsers do not allow a script to set `User-Agent`, so it is omitted there.
|
|
72
|
+
|
|
69
73
|
## API Reference
|
|
70
74
|
|
|
71
75
|
### Extract (any URL)
|
|
@@ -894,6 +898,52 @@ under a billed 200, so pass back the `slug` or `reviews_url` that `search` or
|
|
|
894
898
|
already ships inside `product`. `vendor` is null on the product profile —
|
|
895
899
|
Capterra does not publish it there.
|
|
896
900
|
|
|
901
|
+
### Trustpilot
|
|
902
|
+
|
|
903
|
+
Six endpoints, new in 0.19.0, 2 credits each.
|
|
904
|
+
|
|
905
|
+
```typescript
|
|
906
|
+
// Businesses matching a keyword, up to 100 per page.
|
|
907
|
+
const found = await client.trustpilot.search({ query: "vpn", country: "US", page_size: 10 });
|
|
908
|
+
|
|
909
|
+
// Full profile: TrustScore, rating distribution, reply rate and days to reply on
|
|
910
|
+
// negative reviews, AI summary and topics, similar businesses, 20 newest reviews.
|
|
911
|
+
const profile = await client.trustpilot.business({ domain: "nordvpn.com" });
|
|
912
|
+
|
|
913
|
+
// Filtered reviews, 20 per page.
|
|
914
|
+
const bad = await client.trustpilot.reviews({
|
|
915
|
+
domain: "nordvpn.com",
|
|
916
|
+
stars: [1, 2],
|
|
917
|
+
language: "en",
|
|
918
|
+
date_range: "last30days",
|
|
919
|
+
});
|
|
920
|
+
|
|
921
|
+
// The category tree, then ranked businesses in one category.
|
|
922
|
+
const tree = await client.trustpilot.categories();
|
|
923
|
+
const ranked = await client.trustpilot.category({
|
|
924
|
+
category_id: "vpn_service",
|
|
925
|
+
sort: "reviews_count",
|
|
926
|
+
min_trust_score: 4,
|
|
927
|
+
claimed_only: true,
|
|
928
|
+
});
|
|
929
|
+
|
|
930
|
+
// One review by id, with the business it is about.
|
|
931
|
+
const one = await client.trustpilot.review({ review_id: "6aca2504b52cc4b518bf4b75" });
|
|
932
|
+
```
|
|
933
|
+
|
|
934
|
+
`business` and `reviews` take exactly one of `domain` or `url` (any
|
|
935
|
+
trustpilot.com/review/ page). Reviews are capped at 10 pages of 20 (200) per
|
|
936
|
+
filter combination; `max_page` says how far one goes, and slicing by `stars`,
|
|
937
|
+
`language`, `date_range`, `topics` or `search` reaches more. `total_filtered`
|
|
938
|
+
counts every match, so a date range doubles as review velocity. Topic ids for
|
|
939
|
+
the `topics` filter come from `business`'s `topics`. Search and category rows
|
|
940
|
+
carry `email` and `phone` only when the business published them; the AI summary
|
|
941
|
+
and topics appear only when Trustpilot has them for the requested `language`.
|
|
942
|
+
`min_trust_score` is a minimum star rating (3, 4 or 4.5): Trustpilot rounds
|
|
943
|
+
TrustScore to stars, so 4 includes TrustScore 3.8 and up. An unknown business,
|
|
944
|
+
review or category returns a 404 (`NotFoundError`) and is billed. No TrustScore
|
|
945
|
+
history is returned.
|
|
946
|
+
|
|
897
947
|
### SEC EDGAR
|
|
898
948
|
|
|
899
949
|
**Start with `lookup()`.** Callers hold a ticker; EDGAR is keyed by CIK.
|
|
@@ -1036,24 +1086,24 @@ MIT
|
|
|
1036
1086
|
|
|
1037
1087
|
## About Scavio
|
|
1038
1088
|
|
|
1039
|
-
[Scavio](https://scavio.dev) is a unified web data and
|
|
1040
|
-
[search API for AI agents](https://scavio.dev/search-api-for-ai-agents) — one API
|
|
1089
|
+
[Scavio](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) is a unified web data and
|
|
1090
|
+
[search API for AI agents](https://scavio.dev/search-api-for-ai-agents?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — one API
|
|
1041
1091
|
key, structured JSON, no proxies or browser farms to run. It is a real-time
|
|
1042
|
-
[Tavily alternative](https://scavio.dev/alternatives/tavily) and
|
|
1043
|
-
[SerpAPI alternative](https://scavio.dev/alternatives/serpapi), and with
|
|
1092
|
+
[Tavily alternative](https://scavio.dev/alternatives/tavily?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) and
|
|
1093
|
+
[SerpAPI alternative](https://scavio.dev/alternatives/serpapi?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), and with
|
|
1044
1094
|
`extract()` it also covers the read-any-URL job people reach for Firecrawl to do.
|
|
1045
1095
|
|
|
1046
1096
|
What teams build on it:
|
|
1047
1097
|
|
|
1048
|
-
- **SERP and answer engines** — [Google Search API](https://scavio.dev/google-search-api) for organic results, news, images, maps and the knowledge graph
|
|
1049
|
-
- **Price and catalog monitoring** — [Amazon Product API](https://scavio.dev/amazon-product-api), [Walmart Product API](https://scavio.dev/walmart-product-api), eBay, Target and Home Depot product, review and seller data
|
|
1098
|
+
- **SERP and answer engines** — [Google Search API](https://scavio.dev/google-search-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) for organic results, news, images, maps and the knowledge graph
|
|
1099
|
+
- **Price and catalog monitoring** — [Amazon Product API](https://scavio.dev/amazon-product-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [Walmart Product API](https://scavio.dev/walmart-product-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), eBay, Target and Home Depot product, review and seller data
|
|
1050
1100
|
- **Real estate and travel pipelines** — Zillow and Redfin listings and market stats, Booking, Airbnb and Tripadvisor rates and reviews
|
|
1051
|
-
- **Review mining and competitive research** — Yelp, G2, Capterra, Glassdoor, App Store and Google Play reviews on one shape
|
|
1101
|
+
- **Review mining and competitive research** — Trustpilot, Yelp, G2, Capterra, Glassdoor, App Store and Google Play reviews on one shape
|
|
1052
1102
|
- **Recruiting and company intelligence** — Indeed jobs, Glassdoor salaries, SEC EDGAR filings and XBRL facts, UK Companies House officers and filing history
|
|
1053
1103
|
- **Ad and creative intelligence** — Google Ads Transparency Center and Meta Ad Library creatives
|
|
1054
|
-
- **Social listening** — [YouTube API](https://scavio.dev/youtube-transcript-api), [TikTok API](https://scavio.dev/tiktok-api), [Instagram API](https://scavio.dev/instagram-api), [Reddit API](https://scavio.dev/reddit-api), [X API](https://scavio.dev/docs/x-search), [LinkedIn API](https://scavio.dev/docs/linkedin-person), Threads and Kuaishou
|
|
1104
|
+
- **Social listening** — [YouTube API](https://scavio.dev/youtube-transcript-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [TikTok API](https://scavio.dev/tiktok-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [Instagram API](https://scavio.dev/instagram-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [Reddit API](https://scavio.dev/reddit-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [X API](https://scavio.dev/docs/x-search?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [LinkedIn API](https://scavio.dev/docs/linkedin-person?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), Threads and Kuaishou
|
|
1055
1105
|
- **RAG ingestion** — `extract()` reads any URL and hands back readability Markdown ready to chunk and embed
|
|
1056
1106
|
|
|
1057
|
-
Teams choosing between providers can [compare Scavio vs alternatives](https://scavio.dev/compare) side by side.
|
|
1107
|
+
Teams choosing between providers can [compare Scavio vs alternatives](https://scavio.dev/compare?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) side by side.
|
|
1058
1108
|
|
|
1059
|
-
Get a free [API key](https://dashboard.scavio.dev/sign-up) and explore the [documentation](https://scavio.dev/docs/introduction).
|
|
1109
|
+
Get a free [API key](https://dashboard.scavio.dev/sign-up?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) and explore the [documentation](https://scavio.dev/docs/introduction?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme).
|
package/dist/index.cjs
CHANGED
|
@@ -60,6 +60,7 @@ __export(index_exports, {
|
|
|
60
60
|
TikTokNamespace: () => TikTokNamespace,
|
|
61
61
|
TikTokShopNamespace: () => TikTokShopNamespace,
|
|
62
62
|
TripadvisorNamespace: () => TripadvisorNamespace,
|
|
63
|
+
TrustpilotNamespace: () => TrustpilotNamespace,
|
|
63
64
|
WalmartNamespace: () => WalmartNamespace,
|
|
64
65
|
XNamespace: () => XNamespace,
|
|
65
66
|
YelpNamespace: () => YelpNamespace,
|
|
@@ -199,12 +200,25 @@ function parseRetryAfter(header) {
|
|
|
199
200
|
var BASE_URL = "https://api.scavio.dev";
|
|
200
201
|
var DEFAULT_TIMEOUT = 3e4;
|
|
201
202
|
var DEFAULT_MAX_RETRIES = 2;
|
|
203
|
+
var SDK_VERSION = true ? "0.19.0" : "0.0.0";
|
|
204
|
+
var USER_AGENT = `scavio-js/${SDK_VERSION}`;
|
|
205
|
+
function isServerRuntime() {
|
|
206
|
+
const g = globalThis;
|
|
207
|
+
if (typeof g.window !== "undefined" && typeof g.document !== "undefined") {
|
|
208
|
+
return false;
|
|
209
|
+
}
|
|
210
|
+
return typeof g.process?.versions?.node === "string";
|
|
211
|
+
}
|
|
202
212
|
function buildHeaders(apiKey) {
|
|
203
|
-
|
|
213
|
+
const headers = {
|
|
204
214
|
Authorization: `Bearer ${apiKey}`,
|
|
205
215
|
"Content-Type": "application/json",
|
|
206
216
|
"X-Client-Source": "scavio-js"
|
|
207
217
|
};
|
|
218
|
+
if (isServerRuntime()) {
|
|
219
|
+
headers["User-Agent"] = USER_AGENT;
|
|
220
|
+
}
|
|
221
|
+
return headers;
|
|
208
222
|
}
|
|
209
223
|
function stripUndefined(obj) {
|
|
210
224
|
const result = {};
|
|
@@ -2462,6 +2476,79 @@ var CostcoNamespace = class {
|
|
|
2462
2476
|
}
|
|
2463
2477
|
};
|
|
2464
2478
|
|
|
2479
|
+
// src/namespaces/trustpilot.ts
|
|
2480
|
+
var TrustpilotNamespace = class {
|
|
2481
|
+
constructor(client) {
|
|
2482
|
+
this.client = client;
|
|
2483
|
+
}
|
|
2484
|
+
client;
|
|
2485
|
+
/**
|
|
2486
|
+
* Search Trustpilot businesses by keyword in one country: TrustScore, star
|
|
2487
|
+
* rating, review count, categories, website, and the email, phone and
|
|
2488
|
+
* address the business published. Up to 100 businesses per page.
|
|
2489
|
+
*
|
|
2490
|
+
* Costs 2 credits.
|
|
2491
|
+
*/
|
|
2492
|
+
async search(options) {
|
|
2493
|
+
return this.client._post("/api/v1/trustpilot/search", options);
|
|
2494
|
+
}
|
|
2495
|
+
/**
|
|
2496
|
+
* A Trustpilot business profile: TrustScore, rating distribution, review
|
|
2497
|
+
* languages, categories, contact details, claimed and verification status,
|
|
2498
|
+
* reply rate and reply speed on negative reviews, consumer alerts, the AI
|
|
2499
|
+
* review summary and topics, similar businesses, and the 20 newest reviews
|
|
2500
|
+
* in the chosen language. Pass exactly one of domain or url.
|
|
2501
|
+
*
|
|
2502
|
+
* Costs 2 credits.
|
|
2503
|
+
*/
|
|
2504
|
+
async business(options) {
|
|
2505
|
+
return this.client._post("/api/v1/trustpilot/business", options);
|
|
2506
|
+
}
|
|
2507
|
+
/**
|
|
2508
|
+
* One page of 20 Trustpilot reviews for a business, filtered by stars,
|
|
2509
|
+
* language, date range, topics, text search, verified-only and
|
|
2510
|
+
* with-replies, sorted by recency or relevance. Returns total_filtered, so a
|
|
2511
|
+
* date range doubles as review velocity. Up to 10 pages (200 reviews) per
|
|
2512
|
+
* filter combination. Pass exactly one of domain or url.
|
|
2513
|
+
*
|
|
2514
|
+
* Costs 2 credits.
|
|
2515
|
+
*/
|
|
2516
|
+
async reviews(options) {
|
|
2517
|
+
return this.client._post("/api/v1/trustpilot/reviews", options);
|
|
2518
|
+
}
|
|
2519
|
+
/**
|
|
2520
|
+
* The full Trustpilot category tree (top-level categories and their
|
|
2521
|
+
* subcategories), or categories matching a name. Every category_id works
|
|
2522
|
+
* with category().
|
|
2523
|
+
*
|
|
2524
|
+
* Costs 2 credits.
|
|
2525
|
+
*/
|
|
2526
|
+
async categories(options = {}) {
|
|
2527
|
+
return this.client._post("/api/v1/trustpilot/categories", options);
|
|
2528
|
+
}
|
|
2529
|
+
/**
|
|
2530
|
+
* Businesses ranked in a Trustpilot category for one country, with
|
|
2531
|
+
* TrustScore, review count, location, website, email and phone. Sort by
|
|
2532
|
+
* relevance, review count or latest review, set a minimum star rating, or
|
|
2533
|
+
* keep claimed profiles only. 20 per page.
|
|
2534
|
+
*
|
|
2535
|
+
* Costs 2 credits.
|
|
2536
|
+
*/
|
|
2537
|
+
async category(options) {
|
|
2538
|
+
return this.client._post("/api/v1/trustpilot/category", options);
|
|
2539
|
+
}
|
|
2540
|
+
/**
|
|
2541
|
+
* One Trustpilot review by id: rating, title, full text, dates,
|
|
2542
|
+
* verification, the reviewer's public profile summary, the business reply,
|
|
2543
|
+
* and the business it is about.
|
|
2544
|
+
*
|
|
2545
|
+
* Costs 2 credits.
|
|
2546
|
+
*/
|
|
2547
|
+
async review(options) {
|
|
2548
|
+
return this.client._post("/api/v1/trustpilot/review", options);
|
|
2549
|
+
}
|
|
2550
|
+
};
|
|
2551
|
+
|
|
2465
2552
|
// src/client.ts
|
|
2466
2553
|
var DEFAULT_MAX_REQUESTS_PER_SECOND = 1;
|
|
2467
2554
|
var MAX_REQUESTS_PER_SECOND = 50;
|
|
@@ -2498,6 +2585,7 @@ var Scavio = class {
|
|
|
2498
2585
|
googleAds;
|
|
2499
2586
|
metaAds;
|
|
2500
2587
|
costco;
|
|
2588
|
+
trustpilot;
|
|
2501
2589
|
apiKey;
|
|
2502
2590
|
baseUrl;
|
|
2503
2591
|
timeout;
|
|
@@ -2550,6 +2638,7 @@ var Scavio = class {
|
|
|
2550
2638
|
this.googleAds = new GoogleAdsNamespace(this);
|
|
2551
2639
|
this.metaAds = new MetaAdsNamespace(this);
|
|
2552
2640
|
this.costco = new CostcoNamespace(this);
|
|
2641
|
+
this.trustpilot = new TrustpilotNamespace(this);
|
|
2553
2642
|
}
|
|
2554
2643
|
/** @internal */
|
|
2555
2644
|
async _post(path, body) {
|
|
@@ -2648,6 +2737,7 @@ var Scavio = class {
|
|
|
2648
2737
|
TikTokNamespace,
|
|
2649
2738
|
TikTokShopNamespace,
|
|
2650
2739
|
TripadvisorNamespace,
|
|
2740
|
+
TrustpilotNamespace,
|
|
2651
2741
|
WalmartNamespace,
|
|
2652
2742
|
XNamespace,
|
|
2653
2743
|
YelpNamespace,
|