scavio 0.21.0 → 0.21.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +35 -39
- package/dist/index.cjs +1 -1
- package/dist/index.js +1 -1
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,11 +1,10 @@
|
|
|
1
1
|
# Scavio
|
|
2
2
|
|
|
3
|
-
TypeScript SDK for the [Scavio API](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — real-time web
|
|
4
|
-
|
|
3
|
+
TypeScript SDK for the [Scavio API](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — real-time structured web
|
|
4
|
+
data across 36 platforms on one API key, plus `extract()` to read
|
|
5
5
|
any URL as clean Markdown.
|
|
6
6
|
|
|
7
|
-
Structured JSON in, structured JSON out. No
|
|
8
|
-
per-site parsers to maintain.
|
|
7
|
+
Structured JSON in, structured JSON out. No per-site parsers to maintain.
|
|
9
8
|
|
|
10
9
|
- **Search and SERP** — Google (organic, news, maps, shopping, flights, hotels, trends, AI Mode)
|
|
11
10
|
- **E-commerce** — Amazon, Walmart, Costco, Mercado Libre, eBay, Target, Home Depot, TikTok Shop
|
|
@@ -93,11 +92,11 @@ Credits depend on `mode`, not on a flat per-call price: `normal` costs 1,
|
|
|
93
92
|
`advanced` costs 1, `ultra` costs 2. Billing happens only on a successful
|
|
94
93
|
extraction — a dead link, bot wall or timeout costs nothing.
|
|
95
94
|
|
|
96
|
-
- `normal` is
|
|
97
|
-
- `advanced` renders the page
|
|
98
|
-
|
|
99
|
-
- `ultra`
|
|
100
|
-
|
|
95
|
+
- `normal` is the default. Start here.
|
|
96
|
+
- `advanced` renders the page's JavaScript. Use it when the content is built
|
|
97
|
+
client-side.
|
|
98
|
+
- `ultra` is the hardest-page tier. Use it only when a page blocks the other
|
|
99
|
+
two.
|
|
101
100
|
|
|
102
101
|
`html` is the raw page. `markdown` is a readability extraction with the
|
|
103
102
|
boilerplate stripped. `text` is that markdown flattened to plain text. URLs are
|
|
@@ -169,8 +168,8 @@ silently does nothing.
|
|
|
169
168
|
|
|
170
169
|
#### Amazon changed in 0.12.0 (breaking)
|
|
171
170
|
|
|
172
|
-
Amazon
|
|
173
|
-
|
|
171
|
+
The Amazon API now returns a normalized shape instead of the previous raw
|
|
172
|
+
payload.
|
|
174
173
|
|
|
175
174
|
- `search` returns `{query, page, total_results, total_results_text, count, products[], filters[], related_searches[]}`.
|
|
176
175
|
Each product is `{asin, title, url, image, price, currency, rating, reviews_count, is_sponsored, position, badge, sales_volume, delivery{is_free, date, fastest_date}}`.
|
|
@@ -308,12 +307,12 @@ warehouses on us and 2 per warehouse on ca). The clearance price codes are the
|
|
|
308
307
|
member-community decode of Costco shelf prices; Costco does not publish them,
|
|
309
308
|
and only items Costco lists online are covered.
|
|
310
309
|
|
|
311
|
-
### Mercado Libre
|
|
310
|
+
### [Mercado Libre](https://scavio.dev/docs/mercadolibre-search?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme)
|
|
312
311
|
|
|
313
312
|
Five endpoints, new in 0.21.0: `reviews` costs 1 credit, the other four 2. The
|
|
314
313
|
`best-sellers` endpoint is `bestSellers()`. Mexico (`mx`), Brazil (`br`,
|
|
315
314
|
Mercado Livre), Argentina (`ar`) and Colombia (`co`); prices come back in MXN,
|
|
316
|
-
BRL, ARS or COP. Chile is not supported.
|
|
315
|
+
BRL, ARS or COP. Chile is not supported. Docs: [search](https://scavio.dev/docs/mercadolibre-search?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [product](https://scavio.dev/docs/mercadolibre-product?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [reviews](https://scavio.dev/docs/mercadolibre-reviews?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [questions](https://scavio.dev/docs/mercadolibre-questions?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [best sellers](https://scavio.dev/docs/mercadolibre-best-sellers?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme); overview at the [Mercado Libre API](https://scavio.dev/mercadolibre-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) page.
|
|
317
316
|
|
|
318
317
|
```typescript
|
|
319
318
|
// Keyword search: about 48 rows per page, up to 42 pages.
|
|
@@ -410,7 +409,7 @@ await client.youtube.channelResolve({ channel: "@mkbhd" }); // handle/URL -> id
|
|
|
410
409
|
Every Reddit endpoint costs 1 credit.
|
|
411
410
|
|
|
412
411
|
`search()` takes only `query` and `cursor` — there is no result-type or sort
|
|
413
|
-
filter
|
|
412
|
+
filter, so anything else is dropped server-side. It returns
|
|
414
413
|
`data.results` with `next_cursor` and `has_more`. `post()` returns a flat post
|
|
415
414
|
object under `data` and carries no comments; use `postComments()` for those.
|
|
416
415
|
The subreddit and user feeds return `data.posts`.
|
|
@@ -496,9 +495,9 @@ Credit cost varies by endpoint: `job` costs 30; `personPosts`, `companyPosts`,
|
|
|
496
495
|
`searchJobs` and `postComments` cost 10 per page; `person`, `personAbout`,
|
|
497
496
|
`company` and `post` cost 1.
|
|
498
497
|
|
|
499
|
-
> **Retired endpoints.** The
|
|
500
|
-
> `
|
|
501
|
-
>
|
|
498
|
+
> **Retired endpoints.** The LinkedIn data behind `personContact`,
|
|
499
|
+
> `companyPeople`, `companyJobs`, `searchPeople` and `searchPosts` is no longer
|
|
500
|
+
> available. They remain callable but always return HTTP 410 and are never
|
|
502
501
|
> billed. `company()` returns `featured_employees` (a small sample of staff), and
|
|
503
502
|
> `searchJobs()` with a company name substitutes for `companyJobs()`.
|
|
504
503
|
|
|
@@ -549,7 +548,7 @@ await client.tiktok.userFollowings({ sec_user_id: "abc123" });
|
|
|
549
548
|
Every TikTok Shop endpoint costs 1 credit. Two limits to design around:
|
|
550
549
|
|
|
551
550
|
- `product()` resolves only about 44% of the product ids returned by `search()`.
|
|
552
|
-
|
|
551
|
+
TikTok Shop has no detail data for the rest, so an HTTP 404 is a normal outcome, not an
|
|
553
552
|
error — skip the item instead of retrying. Search to product is not a reliable
|
|
554
553
|
pipeline. `product()` **throws** `NotFoundError` on that 404 (there is no `data`
|
|
555
554
|
field in the body to test), so a loop over search ids must catch it:
|
|
@@ -561,7 +560,7 @@ Every TikTok Shop endpoint costs 1 credit. Two limits to design around:
|
|
|
561
560
|
try {
|
|
562
561
|
const detail = await client.tiktokShop.product({ product_id: productId });
|
|
563
562
|
} catch (e) {
|
|
564
|
-
if (e instanceof NotFoundError) continue; // no detail
|
|
563
|
+
if (e instanceof NotFoundError) continue; // no detail data; skip
|
|
565
564
|
throw e;
|
|
566
565
|
}
|
|
567
566
|
}
|
|
@@ -570,7 +569,7 @@ Every TikTok Shop endpoint costs 1 credit. Two limits to design around:
|
|
|
570
569
|
`productReviews()` often works for ids `product()` cannot resolve: of 8 such ids
|
|
571
570
|
tested, 8 returned HTTP 200 and 7 carried at least one review, so it is a useful
|
|
572
571
|
fallback source of product detail.
|
|
573
|
-
- `product()` does not return a price;
|
|
572
|
+
- `product()` does not return a price; TikTok masks it on the product page. Exact
|
|
574
573
|
prices come from `search()`, `shopProducts()`, and `categoryProducts()`.
|
|
575
574
|
|
|
576
575
|
```typescript
|
|
@@ -613,9 +612,6 @@ Credit cost varies by endpoint, in three tiers:
|
|
|
613
612
|
| 8 | `post`, `commentReplies` |
|
|
614
613
|
| 10 | `profile`, `userReels`, `userTagged`, `userStories`, `postComments`, `searchUsers`, `searchHashtags`, `userFollowers`, `userFollowings` |
|
|
615
614
|
|
|
616
|
-
The 10-credit endpoints run two upstream providers in parallel and bill both
|
|
617
|
-
legs; the 8-credit ones have no fallback leg to hedge against.
|
|
618
|
-
|
|
619
615
|
```typescript
|
|
620
616
|
// User profile
|
|
621
617
|
await client.instagram.profile({ username: "instagram" });
|
|
@@ -647,8 +643,8 @@ await client.instagram.userFollowings({ username: "instagram" });
|
|
|
647
643
|
### Threads
|
|
648
644
|
|
|
649
645
|
Six endpoints, and the credit cost is a function of the body: **2 credits when
|
|
650
|
-
you address a user by `user_id`, 4 when you address them by `username`.**
|
|
651
|
-
|
|
646
|
+
you address a user by `user_id`, 4 when you address them by `username`.** A
|
|
647
|
+
handle needs an extra lookup, so it costs a second call. Only `profile`,
|
|
652
648
|
`userPosts` and `userReplies` are username-keyed; `post`, `postComments` and
|
|
653
649
|
`searchUsers` always cost 2.
|
|
654
650
|
|
|
@@ -856,8 +852,8 @@ await client.booking.reviews({ hotel: "the-independente" });
|
|
|
856
852
|
1 credit per call. `checkin` and `checkout` must be sent together — Booking
|
|
857
853
|
ignores a lone check-in and prices its own date range. `hotel` and `reviews`
|
|
858
854
|
take dates for the same reason: Booking prices a stay, and the response echoes
|
|
859
|
-
whichever dates were used. `currency` defaults to USD; without it Booking
|
|
860
|
-
|
|
855
|
+
whichever dates were used. `currency` defaults to USD; without it Booking picks
|
|
856
|
+
a currency itself and two identical requests can disagree. A search with neither
|
|
861
857
|
`destination` nor `dest_id` returns Booking's homepage and still costs a credit.
|
|
862
858
|
|
|
863
859
|
### Airbnb
|
|
@@ -909,7 +905,7 @@ await client.yelp.reviews({ business_id: "...", page: 2 }); // page 2, not 1
|
|
|
909
905
|
```
|
|
910
906
|
|
|
911
907
|
2 credits per call. `location` is effectively required — without it Yelp
|
|
912
|
-
|
|
908
|
+
picks a location itself and the same request can answer about a different
|
|
913
909
|
metro run to run. **Reviews page 1 is redundant**: it re-fetches the document
|
|
914
910
|
`business()` already returned and costs another 2 credits, so start at page 2.
|
|
915
911
|
Page size is fixed at 10 and a page past the last is a 404.
|
|
@@ -944,7 +940,7 @@ only exists inside Glassdoor's `/Overview/` URLs.
|
|
|
944
940
|
const hits = await client.glassdoor.companies({ query: "Stripe" });
|
|
945
941
|
const company = await client.glassdoor.company({ employer_id: "671932" });
|
|
946
942
|
|
|
947
|
-
// Pass the URLs the company response returns -
|
|
943
|
+
// Pass the URLs the company response returns - saves a lookup
|
|
948
944
|
await client.glassdoor.reviews({ url: company.reviews_url as string });
|
|
949
945
|
await client.glassdoor.salaries({ url: company.salaries_url as string });
|
|
950
946
|
```
|
|
@@ -952,8 +948,8 @@ await client.glassdoor.salaries({ url: company.salaries_url as string });
|
|
|
952
948
|
1 credit per call. Glassdoor's login wall caps `reviews` at **three reviews per
|
|
953
949
|
response** — there is deliberately no `page` param; move the window with
|
|
954
950
|
`category` and `employment_status` and read `filtered_review_count` to see how
|
|
955
|
-
many match. Addressing `reviews` or `salaries` by `employer_id`
|
|
956
|
-
|
|
951
|
+
many match. Addressing `reviews` or `salaries` by `employer_id` needs an
|
|
952
|
+
extra lookup because the slugs are case-sensitive and must be read off the
|
|
957
953
|
profile, so prefer the `reviews_url` / `salaries_url` the company response hands
|
|
958
954
|
back. These endpoints are slow and flaky by nature (company about 3-47s, reviews
|
|
959
955
|
about 75s, salaries about 41s) — raise `timeout` and keep retries on.
|
|
@@ -997,9 +993,9 @@ await client.g2.product({ product_id: "notion" });
|
|
|
997
993
|
await client.g2.reviews({ product_id: "notion", page: 2, company_size: "enterprise" });
|
|
998
994
|
```
|
|
999
995
|
|
|
1000
|
-
**5 credits per call — the most expensive platform in the SDK
|
|
1001
|
-
|
|
1002
|
-
|
|
996
|
+
**5 credits per call — the most expensive platform in the SDK.** Retries are
|
|
997
|
+
deliberately conservative for that reason, and a blocked page arrives as a
|
|
998
|
+
billed 200 rather than an error. G2
|
|
1003
999
|
loads review text in a separate frame, so `product` carries no reviews — call
|
|
1004
1000
|
`reviews`, which is also the only place with exact per-star counts, pros/cons by
|
|
1005
1001
|
theme, and company-size / role / industry / region facets.
|
|
@@ -1080,9 +1076,9 @@ await client.sec.search({ query: "climate risk", form: "10-K" });
|
|
|
1080
1076
|
```
|
|
1081
1077
|
|
|
1082
1078
|
1 credit per call, including `include_history: true`, which is the one call that
|
|
1083
|
-
can
|
|
1084
|
-
spelling. XBRL concept tags are **case-sensitive** — `netincomeloss` is a 404
|
|
1085
|
-
|
|
1079
|
+
can take up to 10 lookups. Both `cik` and `ticker` accept either
|
|
1080
|
+
spelling. XBRL concept tags are **case-sensitive** — `netincomeloss` is a 404,
|
|
1081
|
+
so use `facts()` to see what a filer actually reports. EDGAR's
|
|
1086
1082
|
"recent" block is not a fixed window: a decade for a quiet filer, about a year
|
|
1087
1083
|
for a prolific one.
|
|
1088
1084
|
|
|
@@ -1180,7 +1176,7 @@ All error classes:
|
|
|
1180
1176
|
| `BadRequestError` | 400, 422 | Invalid request parameters |
|
|
1181
1177
|
| `InvalidAPIKeyError` | 401 | Invalid API key |
|
|
1182
1178
|
| `InsufficientCreditsError` | 402 | No credits remaining |
|
|
1183
|
-
| `NotFoundError` | 404 | No data
|
|
1179
|
+
| `NotFoundError` | 404 | No data for that id (see TikTok Shop above) |
|
|
1184
1180
|
| `RateLimitError` | 429 | Rate limit exceeded |
|
|
1185
1181
|
| `ScavioAPIError` | other | Catch-all (has `.statusCode`) |
|
|
1186
1182
|
|
|
@@ -1210,7 +1206,7 @@ MIT
|
|
|
1210
1206
|
|
|
1211
1207
|
[Scavio](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) is a unified web data and
|
|
1212
1208
|
[search API for AI agents](https://scavio.dev/search-api-for-ai-agents?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — one API
|
|
1213
|
-
key, structured JSON
|
|
1209
|
+
key, structured JSON. It is a real-time
|
|
1214
1210
|
[Tavily alternative](https://scavio.dev/alternatives/tavily?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) and
|
|
1215
1211
|
[SerpAPI alternative](https://scavio.dev/alternatives/serpapi?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), and with
|
|
1216
1212
|
`extract()` it also covers the read-any-URL job people reach for Firecrawl to do.
|
package/dist/index.cjs
CHANGED
|
@@ -203,7 +203,7 @@ function parseRetryAfter(header) {
|
|
|
203
203
|
var BASE_URL = "https://api.scavio.dev";
|
|
204
204
|
var DEFAULT_TIMEOUT = 3e4;
|
|
205
205
|
var DEFAULT_MAX_RETRIES = 2;
|
|
206
|
-
var SDK_VERSION = true ? "0.21.
|
|
206
|
+
var SDK_VERSION = true ? "0.21.1" : "0.0.0";
|
|
207
207
|
var USER_AGENT = `scavio-js/${SDK_VERSION}`;
|
|
208
208
|
function isServerRuntime() {
|
|
209
209
|
const g = globalThis;
|
package/dist/index.js
CHANGED
|
@@ -129,7 +129,7 @@ function parseRetryAfter(header) {
|
|
|
129
129
|
var BASE_URL = "https://api.scavio.dev";
|
|
130
130
|
var DEFAULT_TIMEOUT = 3e4;
|
|
131
131
|
var DEFAULT_MAX_RETRIES = 2;
|
|
132
|
-
var SDK_VERSION = true ? "0.21.
|
|
132
|
+
var SDK_VERSION = true ? "0.21.1" : "0.0.0";
|
|
133
133
|
var USER_AGENT = `scavio-js/${SDK_VERSION}`;
|
|
134
134
|
function isServerRuntime() {
|
|
135
135
|
const g = globalThis;
|
package/package.json
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "scavio",
|
|
3
|
-
"version": "0.21.
|
|
4
|
-
"description": "TypeScript SDK for the Scavio API — real-time web
|
|
3
|
+
"version": "0.21.1",
|
|
4
|
+
"description": "TypeScript SDK for the Scavio API — real-time structured web data across 36 platforms on one API key: Google SERP, Amazon, Walmart, Costco, Mercado Libre, eBay, Zillow, Redfin, Realtor.com, Rightmove, Booking, Airbnb, Yelp, Trustpilot, Indeed, Glassdoor, G2, App Store, Google Play, SEC EDGAR, Google Ads and Meta Ad Library, YouTube, TikTok, Instagram, X, LinkedIn, Reddit and more, plus extract() to read any URL as clean Markdown",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "./dist/index.cjs",
|
|
7
7
|
"module": "./dist/index.js",
|