scavio 0.21.0 → 0.21.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,11 +1,10 @@
1
1
  # Scavio
2
2
 
3
- TypeScript SDK for the [Scavio API](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — real-time web scraping
4
- and data extraction across 36 platforms on one API key, plus `extract()` to read
3
+ TypeScript SDK for the [Scavio API](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — real-time structured web
4
+ data across 36 platforms on one API key, plus `extract()` to read
5
5
  any URL as clean Markdown.
6
6
 
7
- Structured JSON in, structured JSON out. No proxies, no headless browsers, no
8
- per-site parsers to maintain.
7
+ Structured JSON in, structured JSON out. No per-site parsers to maintain.
9
8
 
10
9
  - **Search and SERP** — Google (organic, news, maps, shopping, flights, hotels, trends, AI Mode)
11
10
  - **E-commerce** — Amazon, Walmart, Costco, Mercado Libre, eBay, Target, Home Depot, TikTok Shop
@@ -93,11 +92,11 @@ Credits depend on `mode`, not on a flat per-call price: `normal` costs 1,
93
92
  `advanced` costs 1, `ultra` costs 2. Billing happens only on a successful
94
93
  extraction — a dead link, bot wall or timeout costs nothing.
95
94
 
96
- - `normal` is a plain datacenter fetch. Start here.
97
- - `advanced` renders the page in a headless browser. Use it when the content is
98
- built client-side.
99
- - `ultra` goes through residential IPs. Use it only when a bot wall blocks the
100
- other two.
95
+ - `normal` is the default. Start here.
96
+ - `advanced` renders the page's JavaScript. Use it when the content is built
97
+ client-side.
98
+ - `ultra` is the hardest-page tier. Use it only when a page blocks the other
99
+ two.
101
100
 
102
101
  `html` is the raw page. `markdown` is a readability extraction with the
103
102
  boilerplate stripped. `text` is that markdown flattened to plain text. URLs are
@@ -169,8 +168,8 @@ silently does nothing.
169
168
 
170
169
  #### Amazon changed in 0.12.0 (breaking)
171
170
 
172
- Amazon moved to a new upstream and the API now returns a normalized shape
173
- instead of the previous raw provider payload.
171
+ The Amazon API now returns a normalized shape instead of the previous raw
172
+ payload.
174
173
 
175
174
  - `search` returns `{query, page, total_results, total_results_text, count, products[], filters[], related_searches[]}`.
176
175
  Each product is `{asin, title, url, image, price, currency, rating, reviews_count, is_sponsored, position, badge, sales_volume, delivery{is_free, date, fastest_date}}`.
@@ -308,12 +307,12 @@ warehouses on us and 2 per warehouse on ca). The clearance price codes are the
308
307
  member-community decode of Costco shelf prices; Costco does not publish them,
309
308
  and only items Costco lists online are covered.
310
309
 
311
- ### Mercado Libre
310
+ ### [Mercado Libre](https://scavio.dev/docs/mercadolibre-search?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme)
312
311
 
313
312
  Five endpoints, new in 0.21.0: `reviews` costs 1 credit, the other four 2. The
314
313
  `best-sellers` endpoint is `bestSellers()`. Mexico (`mx`), Brazil (`br`,
315
314
  Mercado Livre), Argentina (`ar`) and Colombia (`co`); prices come back in MXN,
316
- BRL, ARS or COP. Chile is not supported.
315
+ BRL, ARS or COP. Chile is not supported. Docs: [search](https://scavio.dev/docs/mercadolibre-search?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [product](https://scavio.dev/docs/mercadolibre-product?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [reviews](https://scavio.dev/docs/mercadolibre-reviews?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [questions](https://scavio.dev/docs/mercadolibre-questions?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), [best sellers](https://scavio.dev/docs/mercadolibre-best-sellers?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme); overview at the [Mercado Libre API](https://scavio.dev/mercadolibre-api?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) page.
317
316
 
318
317
  ```typescript
319
318
  // Keyword search: about 48 rows per page, up to 42 pages.
@@ -410,7 +409,7 @@ await client.youtube.channelResolve({ channel: "@mkbhd" }); // handle/URL -> id
410
409
  Every Reddit endpoint costs 1 credit.
411
410
 
412
411
  `search()` takes only `query` and `cursor` — there is no result-type or sort
413
- filter upstream, so anything else is dropped server-side. It returns
412
+ filter, so anything else is dropped server-side. It returns
414
413
  `data.results` with `next_cursor` and `has_more`. `post()` returns a flat post
415
414
  object under `data` and carries no comments; use `postComments()` for those.
416
415
  The subreddit and user feeds return `data.posts`.
@@ -496,9 +495,9 @@ Credit cost varies by endpoint: `job` costs 30; `personPosts`, `companyPosts`,
496
495
  `searchJobs` and `postComments` cost 10 per page; `person`, `personAbout`,
497
496
  `company` and `post` cost 1.
498
497
 
499
- > **Retired endpoints.** The upstream provider withdrew the datasets behind
500
- > `personContact`, `companyPeople`, `companyJobs`, `searchPeople` and
501
- > `searchPosts`. They remain callable but always return HTTP 410 and are never
498
+ > **Retired endpoints.** The LinkedIn data behind `personContact`,
499
+ > `companyPeople`, `companyJobs`, `searchPeople` and `searchPosts` is no longer
500
+ > available. They remain callable but always return HTTP 410 and are never
502
501
  > billed. `company()` returns `featured_employees` (a small sample of staff), and
503
502
  > `searchJobs()` with a company name substitutes for `companyJobs()`.
504
503
 
@@ -549,7 +548,7 @@ await client.tiktok.userFollowings({ sec_user_id: "abc123" });
549
548
  Every TikTok Shop endpoint costs 1 credit. Two limits to design around:
550
549
 
551
550
  - `product()` resolves only about 44% of the product ids returned by `search()`.
552
- Upstream has no detail data for the rest, so an HTTP 404 is a normal outcome, not an
551
+ TikTok Shop has no detail data for the rest, so an HTTP 404 is a normal outcome, not an
553
552
  error — skip the item instead of retrying. Search to product is not a reliable
554
553
  pipeline. `product()` **throws** `NotFoundError` on that 404 (there is no `data`
555
554
  field in the body to test), so a loop over search ids must catch it:
@@ -561,7 +560,7 @@ Every TikTok Shop endpoint costs 1 credit. Two limits to design around:
561
560
  try {
562
561
  const detail = await client.tiktokShop.product({ product_id: productId });
563
562
  } catch (e) {
564
- if (e instanceof NotFoundError) continue; // no detail upstream; skip
563
+ if (e instanceof NotFoundError) continue; // no detail data; skip
565
564
  throw e;
566
565
  }
567
566
  }
@@ -570,7 +569,7 @@ Every TikTok Shop endpoint costs 1 credit. Two limits to design around:
570
569
  `productReviews()` often works for ids `product()` cannot resolve: of 8 such ids
571
570
  tested, 8 returned HTTP 200 and 7 carried at least one review, so it is a useful
572
571
  fallback source of product detail.
573
- - `product()` does not return a price; upstream masks it on the product page. Exact
572
+ - `product()` does not return a price; TikTok masks it on the product page. Exact
574
573
  prices come from `search()`, `shopProducts()`, and `categoryProducts()`.
575
574
 
576
575
  ```typescript
@@ -613,9 +612,6 @@ Credit cost varies by endpoint, in three tiers:
613
612
  | 8 | `post`, `commentReplies` |
614
613
  | 10 | `profile`, `userReels`, `userTagged`, `userStories`, `postComments`, `searchUsers`, `searchHashtags`, `userFollowers`, `userFollowings` |
615
614
 
616
- The 10-credit endpoints run two upstream providers in parallel and bill both
617
- legs; the 8-credit ones have no fallback leg to hedge against.
618
-
619
615
  ```typescript
620
616
  // User profile
621
617
  await client.instagram.profile({ username: "instagram" });
@@ -647,8 +643,8 @@ await client.instagram.userFollowings({ username: "instagram" });
647
643
  ### Threads
648
644
 
649
645
  Six endpoints, and the credit cost is a function of the body: **2 credits when
650
- you address a user by `user_id`, 4 when you address them by `username`.** The
651
- upstream handle lookup is dead, so a handle buys a second call. Only `profile`,
646
+ you address a user by `user_id`, 4 when you address them by `username`.** A
647
+ handle needs an extra lookup, so it costs a second call. Only `profile`,
652
648
  `userPosts` and `userReplies` are username-keyed; `post`, `postComments` and
653
649
  `searchUsers` always cost 2.
654
650
 
@@ -856,8 +852,8 @@ await client.booking.reviews({ hotel: "the-independente" });
856
852
  1 credit per call. `checkin` and `checkout` must be sent together — Booking
857
853
  ignores a lone check-in and prices its own date range. `hotel` and `reviews`
858
854
  take dates for the same reason: Booking prices a stay, and the response echoes
859
- whichever dates were used. `currency` defaults to USD; without it Booking prices
860
- off the proxy exit and two identical requests disagree. A search with neither
855
+ whichever dates were used. `currency` defaults to USD; without it Booking picks
856
+ a currency itself and two identical requests can disagree. A search with neither
861
857
  `destination` nor `dest_id` returns Booking's homepage and still costs a credit.
862
858
 
863
859
  ### Airbnb
@@ -909,7 +905,7 @@ await client.yelp.reviews({ business_id: "...", page: 2 }); // page 2, not 1
909
905
  ```
910
906
 
911
907
  2 credits per call. `location` is effectively required — without it Yelp
912
- geolocates off the proxy exit and the same request answers about a different
908
+ picks a location itself and the same request can answer about a different
913
909
  metro run to run. **Reviews page 1 is redundant**: it re-fetches the document
914
910
  `business()` already returned and costs another 2 credits, so start at page 2.
915
911
  Page size is fixed at 10 and a page past the last is a 404.
@@ -944,7 +940,7 @@ only exists inside Glassdoor's `/Overview/` URLs.
944
940
  const hits = await client.glassdoor.companies({ query: "Stripe" });
945
941
  const company = await client.glassdoor.company({ employer_id: "671932" });
946
942
 
947
- // Pass the URLs the company response returns - halves the upstream work
943
+ // Pass the URLs the company response returns - saves a lookup
948
944
  await client.glassdoor.reviews({ url: company.reviews_url as string });
949
945
  await client.glassdoor.salaries({ url: company.salaries_url as string });
950
946
  ```
@@ -952,8 +948,8 @@ await client.glassdoor.salaries({ url: company.salaries_url as string });
952
948
  1 credit per call. Glassdoor's login wall caps `reviews` at **three reviews per
953
949
  response** — there is deliberately no `page` param; move the window with
954
950
  `category` and `employment_status` and read `filtered_review_count` to see how
955
- many match. Addressing `reviews` or `salaries` by `employer_id` costs two
956
- upstream fetches because the slugs are case-sensitive and must be read off the
951
+ many match. Addressing `reviews` or `salaries` by `employer_id` needs an
952
+ extra lookup because the slugs are case-sensitive and must be read off the
957
953
  profile, so prefer the `reviews_url` / `salaries_url` the company response hands
958
954
  back. These endpoints are slow and flaky by nature (company about 3-47s, reviews
959
955
  about 75s, salaries about 41s) — raise `timeout` and keep retries on.
@@ -997,9 +993,9 @@ await client.g2.product({ product_id: "notion" });
997
993
  await client.g2.reviews({ product_id: "notion", page: 2, company_size: "enterprise" });
998
994
  ```
999
995
 
1000
- **5 credits per call — the most expensive platform in the SDK**, because g2.com
1001
- bills 25 upstream credits per fetch. Retries are deliberately conservative for
1002
- that reason, and a bot wall arrives as a billed 200 rather than an error. G2
996
+ **5 credits per call — the most expensive platform in the SDK.** Retries are
997
+ deliberately conservative for that reason, and a blocked page arrives as a
998
+ billed 200 rather than an error. G2
1003
999
  loads review text in a separate frame, so `product` carries no reviews — call
1004
1000
  `reviews`, which is also the only place with exact per-star counts, pros/cons by
1005
1001
  theme, and company-size / role / industry / region facets.
@@ -1080,9 +1076,9 @@ await client.sec.search({ query: "climate risk", form: "10-K" });
1080
1076
  ```
1081
1077
 
1082
1078
  1 credit per call, including `include_history: true`, which is the one call that
1083
- can buy up to 10 upstream fetches. Both `cik` and `ticker` accept either
1084
- spelling. XBRL concept tags are **case-sensitive** — `netincomeloss` is a 404
1085
- upstream, so use `facts()` to see what a filer actually reports. EDGAR's
1079
+ can take up to 10 lookups. Both `cik` and `ticker` accept either
1080
+ spelling. XBRL concept tags are **case-sensitive** — `netincomeloss` is a 404,
1081
+ so use `facts()` to see what a filer actually reports. EDGAR's
1086
1082
  "recent" block is not a fixed window: a decade for a quiet filer, about a year
1087
1083
  for a prolific one.
1088
1084
 
@@ -1180,7 +1176,7 @@ All error classes:
1180
1176
  | `BadRequestError` | 400, 422 | Invalid request parameters |
1181
1177
  | `InvalidAPIKeyError` | 401 | Invalid API key |
1182
1178
  | `InsufficientCreditsError` | 402 | No credits remaining |
1183
- | `NotFoundError` | 404 | No data upstream for that id (see TikTok Shop above) |
1179
+ | `NotFoundError` | 404 | No data for that id (see TikTok Shop above) |
1184
1180
  | `RateLimitError` | 429 | Rate limit exceeded |
1185
1181
  | `ScavioAPIError` | other | Catch-all (has `.statusCode`) |
1186
1182
 
@@ -1210,7 +1206,7 @@ MIT
1210
1206
 
1211
1207
  [Scavio](https://scavio.dev?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) is a unified web data and
1212
1208
  [search API for AI agents](https://scavio.dev/search-api-for-ai-agents?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) — one API
1213
- key, structured JSON, no proxies or browser farms to run. It is a real-time
1209
+ key, structured JSON. It is a real-time
1214
1210
  [Tavily alternative](https://scavio.dev/alternatives/tavily?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme) and
1215
1211
  [SerpAPI alternative](https://scavio.dev/alternatives/serpapi?utm_source=npm-scavio-js&utm_medium=package&utm_campaign=readme), and with
1216
1212
  `extract()` it also covers the read-any-URL job people reach for Firecrawl to do.
package/dist/index.cjs CHANGED
@@ -203,7 +203,7 @@ function parseRetryAfter(header) {
203
203
  var BASE_URL = "https://api.scavio.dev";
204
204
  var DEFAULT_TIMEOUT = 3e4;
205
205
  var DEFAULT_MAX_RETRIES = 2;
206
- var SDK_VERSION = true ? "0.21.0" : "0.0.0";
206
+ var SDK_VERSION = true ? "0.21.1" : "0.0.0";
207
207
  var USER_AGENT = `scavio-js/${SDK_VERSION}`;
208
208
  function isServerRuntime() {
209
209
  const g = globalThis;
package/dist/index.js CHANGED
@@ -129,7 +129,7 @@ function parseRetryAfter(header) {
129
129
  var BASE_URL = "https://api.scavio.dev";
130
130
  var DEFAULT_TIMEOUT = 3e4;
131
131
  var DEFAULT_MAX_RETRIES = 2;
132
- var SDK_VERSION = true ? "0.21.0" : "0.0.0";
132
+ var SDK_VERSION = true ? "0.21.1" : "0.0.0";
133
133
  var USER_AGENT = `scavio-js/${SDK_VERSION}`;
134
134
  function isServerRuntime() {
135
135
  const g = globalThis;
package/package.json CHANGED
@@ -1,7 +1,7 @@
1
1
  {
2
2
  "name": "scavio",
3
- "version": "0.21.0",
4
- "description": "TypeScript SDK for the Scavio API — real-time web scraping and data extraction across 36 platforms on one API key: Google SERP, Amazon, Walmart, Costco, Mercado Libre, eBay, Zillow, Redfin, Realtor.com, Rightmove, Booking, Airbnb, Yelp, Trustpilot, Indeed, Glassdoor, G2, App Store, Google Play, SEC EDGAR, Google Ads and Meta Ad Library, YouTube, TikTok, Instagram, X, LinkedIn, Reddit and more, plus extract() to read any URL as clean Markdown",
3
+ "version": "0.21.1",
4
+ "description": "TypeScript SDK for the Scavio API — real-time structured web data across 36 platforms on one API key: Google SERP, Amazon, Walmart, Costco, Mercado Libre, eBay, Zillow, Redfin, Realtor.com, Rightmove, Booking, Airbnb, Yelp, Trustpilot, Indeed, Glassdoor, G2, App Store, Google Play, SEC EDGAR, Google Ads and Meta Ad Library, YouTube, TikTok, Instagram, X, LinkedIn, Reddit and more, plus extract() to read any URL as clean Markdown",
5
5
  "type": "module",
6
6
  "main": "./dist/index.cjs",
7
7
  "module": "./dist/index.js",