scavio 0.16.0 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +95 -10
- package/dist/index.cjs +163 -3
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +364 -4
- package/dist/index.d.ts +364 -4
- package/dist/index.js +162 -3
- package/dist/index.js.map +1 -1
- package/package.json +3 -2
package/README.md
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
# Scavio
|
|
2
2
|
|
|
3
3
|
TypeScript SDK for the [Scavio API](https://scavio.dev) — real-time web scraping
|
|
4
|
-
and data extraction across
|
|
4
|
+
and data extraction across 32 platforms on one API key, plus `extract()` to read
|
|
5
5
|
any URL as clean Markdown.
|
|
6
6
|
|
|
7
7
|
Structured JSON in, structured JSON out. No proxies, no headless browsers, no
|
|
8
8
|
per-site parsers to maintain.
|
|
9
9
|
|
|
10
10
|
- **Search and SERP** — Google (organic, news, maps, shopping, flights, hotels, trends, AI Mode)
|
|
11
|
-
- **E-commerce** — Amazon, Walmart, eBay, Target, Home Depot, TikTok Shop
|
|
11
|
+
- **E-commerce** — Amazon, Walmart, Costco, eBay, Target, Home Depot, TikTok Shop
|
|
12
12
|
- **Real estate and travel** — Zillow, Redfin, Booking, Airbnb, Tripadvisor
|
|
13
13
|
- **Reviews and local** — Yelp, G2, Capterra, Glassdoor, App Store, Google Play
|
|
14
14
|
- **Jobs and companies** — Indeed, Glassdoor, SEC EDGAR, Companies House
|
|
@@ -178,8 +178,8 @@ instead of the previous raw provider payload.
|
|
|
178
178
|
|
|
179
179
|
### Walmart
|
|
180
180
|
|
|
181
|
-
|
|
182
|
-
|
|
181
|
+
Eight endpoints. `stores` and store targeting on `search` / `product` are new in
|
|
182
|
+
0.17.0.
|
|
183
183
|
|
|
184
184
|
```typescript
|
|
185
185
|
await client.walmart.search({
|
|
@@ -197,18 +197,49 @@ await client.walmart.category({ category_id: "3944_133251_1095191" });
|
|
|
197
197
|
await client.walmart.offers({ product_id: "13544111159" });
|
|
198
198
|
await client.walmart.seller({ seller_id: "101138578" });
|
|
199
199
|
await client.walmart.sellerProducts({ seller_id: "101138578" });
|
|
200
|
+
|
|
201
|
+
// Store targeting: find stores for a ZIP, then pass one with the ZIP.
|
|
202
|
+
const { data } = await client.walmart.stores({ zipcode: "50036" });
|
|
203
|
+
await client.walmart.search({
|
|
204
|
+
query: "whole milk",
|
|
205
|
+
delivery_zip: "50036",
|
|
206
|
+
store_id: data.stores[0].store_id!, // "1389", Boone Supercenter
|
|
207
|
+
});
|
|
208
|
+
|
|
209
|
+
// walmart.ca works the same way with a postal code and domain "ca".
|
|
210
|
+
await client.walmart.stores({ zipcode: "M5V 2T6", domain: "ca" });
|
|
211
|
+
await client.walmart.product({
|
|
212
|
+
product_id: "10220526",
|
|
213
|
+
domain: "ca",
|
|
214
|
+
delivery_zip: "M5V 2T6",
|
|
215
|
+
store_id: "3106",
|
|
216
|
+
});
|
|
200
217
|
```
|
|
201
218
|
|
|
202
|
-
Credits are a function of the body
|
|
203
|
-
"ca"
|
|
219
|
+
Credits are a function of the body: `search` and `category` cost 1 credit on
|
|
220
|
+
`domain` "com" or "ca" and 2 on "com.mx"; a store-targeted `search` or `product`
|
|
221
|
+
costs 2. `stores` and the other endpoints cost 1.
|
|
222
|
+
|
|
223
|
+
#### Walmart store targeting (0.17.0)
|
|
224
|
+
|
|
225
|
+
- `delivery_zip` and `store_id` must be sent **together**, on `search` or
|
|
226
|
+
`product`. They work on walmart.com and walmart.ca (`domain: "ca"` with a
|
|
227
|
+
Canadian postal code); walmart.com.mx does not support store targeting.
|
|
228
|
+
- Get `store_id` from `stores()` on the same domain. `store_id` decides the
|
|
229
|
+
store; `data.location` in the response confirms which store was used.
|
|
230
|
+
- Store-targeted calls take 10-60 seconds — set a generous timeout. A
|
|
231
|
+
`store_id` that does not exist returns 400, and a product the store does not
|
|
232
|
+
carry returns 404.
|
|
233
|
+
- `product` accepts `domain: "ca"` only together with a store target.
|
|
204
234
|
|
|
205
235
|
#### Walmart changed in 0.15.0 (breaking)
|
|
206
236
|
|
|
207
|
-
- `device
|
|
208
|
-
|
|
237
|
+
- `device` is retired. Sending it still returns 200, with a top-level `warnings`
|
|
238
|
+
array naming what was ignored. (`delivery_zip` and `store_id` were retired in
|
|
239
|
+
0.15.0 and came back in 0.17.0 as store targeting.)
|
|
209
240
|
- `domain` is **not** retired — it is the price-bearing param, and it is accepted
|
|
210
|
-
on `search` and `category
|
|
211
|
-
|
|
241
|
+
on `search` and `category`. Product-keyed endpoints are walmart.com only,
|
|
242
|
+
except `product` with a store target (0.17.0).
|
|
212
243
|
- `page` (1-indexed) is the paging param; `start_page` remains a deprecated alias.
|
|
213
244
|
- `sort_by` gained `rating_high` and `new`.
|
|
214
245
|
- `fulfillment_speed` is `today` or `tomorrow` only. There is deliberately no
|
|
@@ -219,6 +250,60 @@ Credits are a function of the body on `search` and `category`: `domain` "com" an
|
|
|
219
250
|
- `sellerProducts` has **no pagination** — roughly the first 40 server-rendered
|
|
220
251
|
items. `total_count` reports the seller's real catalog size.
|
|
221
252
|
|
|
253
|
+
### Costco
|
|
254
|
+
|
|
255
|
+
Thirteen endpoints, new in 0.18.0. `country` is `"us"` (costco.com, default) or
|
|
256
|
+
`"ca"` (costco.ca); `search`, `product` and `warehouses` also take `"uk"`,
|
|
257
|
+
`"au"`, `"mx"`, `"jp"`, `"kr"` and `"tw"` (keyword, sort and paging only).
|
|
258
|
+
|
|
259
|
+
```typescript
|
|
260
|
+
// Warehouse numbers come from warehouses().
|
|
261
|
+
await client.costco.warehouses({ zip: "10025", limit: 3 });
|
|
262
|
+
|
|
263
|
+
// warehouse_id adds that warehouse's in-store price, stock and price code to
|
|
264
|
+
// every result on search, category and deals.
|
|
265
|
+
await client.costco.search({ query: "olive oil", warehouse_id: "1062", page_size: 5 });
|
|
266
|
+
await client.costco.category({ category: "televisions", sort_by: "price_low" });
|
|
267
|
+
await client.costco.categories(); // departments
|
|
268
|
+
await client.costco.categories({ category_id: "30001" }); // one department's tree
|
|
269
|
+
await client.costco.product({ item_ids: ["100334960", "1492456"] }); // up to 20
|
|
270
|
+
await client.costco.reviews({ product_id: "100334960", sort_by: "most_helpful" });
|
|
271
|
+
await client.costco.autocomplete({ query: "kirk" });
|
|
272
|
+
|
|
273
|
+
// Online vs in-warehouse price at up to 10 warehouses, with discount, final
|
|
274
|
+
// price, promotion dates and price code.
|
|
275
|
+
await client.costco.prices({ item_id: "1492456", warehouse_ids: ["1062", "1107"] });
|
|
276
|
+
|
|
277
|
+
// In-warehouse stock status (never a quantity), pickup and same-day delivery.
|
|
278
|
+
await client.costco.availability({ item_number: "1492456", warehouse_ids: ["1062"] });
|
|
279
|
+
|
|
280
|
+
// Regular, premium and diesel prices near a location, or at given warehouses.
|
|
281
|
+
await client.costco.gas({ zip: "90001" });
|
|
282
|
+
await client.costco.gas({ warehouse_ids: ["1062", "1107"] });
|
|
283
|
+
|
|
284
|
+
// The current member coupon book (US only, no parameters).
|
|
285
|
+
await client.costco.coupons();
|
|
286
|
+
|
|
287
|
+
// Deal feeds: new, while_supplies_last, treasure_hunt, member_favorites,
|
|
288
|
+
// online_only, on_sale.
|
|
289
|
+
await client.costco.deals({ type: "treasure_hunt", warehouse_id: "1062" });
|
|
290
|
+
|
|
291
|
+
// Markdown finder at one warehouse: .97 clearance and .00/.88 manager
|
|
292
|
+
// markdowns by default; add special_buy for .49/.79/.89 endings.
|
|
293
|
+
await client.costco.clearance({
|
|
294
|
+
warehouse_id: "1062",
|
|
295
|
+
query: "snacks",
|
|
296
|
+
price_codes: ["clearance", "manager_markdown", "special_buy"],
|
|
297
|
+
});
|
|
298
|
+
```
|
|
299
|
+
|
|
300
|
+
1 credit per call except where the body scales the work: `prices` costs 2 at 10
|
|
301
|
+
warehouses, `availability` 1 per 5 warehouses, `warehouses` 2 with
|
|
302
|
+
`country: "ca"`, and `gas` 2 by location on ca (by `warehouse_ids`, 1 per 5
|
|
303
|
+
warehouses on us and 2 per warehouse on ca). The clearance price codes are the
|
|
304
|
+
member-community decode of Costco shelf prices; Costco does not publish them,
|
|
305
|
+
and only items Costco lists online are covered.
|
|
306
|
+
|
|
222
307
|
### YouTube
|
|
223
308
|
|
|
224
309
|
Credit cost varies by endpoint: `transcript` costs 8; `streams` costs 3;
|
package/dist/index.cjs
CHANGED
|
@@ -27,6 +27,7 @@ __export(index_exports, {
|
|
|
27
27
|
BookingNamespace: () => BookingNamespace,
|
|
28
28
|
CapterraNamespace: () => CapterraNamespace,
|
|
29
29
|
CompaniesHouseNamespace: () => CompaniesHouseNamespace,
|
|
30
|
+
CostcoNamespace: () => CostcoNamespace,
|
|
30
31
|
DEFAULT_MAX_REQUESTS_PER_SECOND: () => DEFAULT_MAX_REQUESTS_PER_SECOND,
|
|
31
32
|
EbayNamespace: () => EbayNamespace,
|
|
32
33
|
G2Namespace: () => G2Namespace,
|
|
@@ -680,7 +681,11 @@ var WalmartNamespace = class {
|
|
|
680
681
|
* Structured Walmart search results: `products[]`, `products_count` and the
|
|
681
682
|
* resolved `location`. Page through with `page` (1-indexed).
|
|
682
683
|
*
|
|
683
|
-
*
|
|
684
|
+
* Pass `delivery_zip` + `store_id` to get one store's results (walmart.com
|
|
685
|
+
* and walmart.ca); `data.location` confirms the store.
|
|
686
|
+
*
|
|
687
|
+
* Costs 1 credit for `domain` "com" or "ca", 2 credits for "com.mx", and 2
|
|
688
|
+
* credits when store-targeted.
|
|
684
689
|
*/
|
|
685
690
|
async search(options) {
|
|
686
691
|
return this.client._post("/api/v1/walmart/search", options);
|
|
@@ -689,8 +694,11 @@ var WalmartNamespace = class {
|
|
|
689
694
|
* Full product detail: price, rating, images, specifications, availability
|
|
690
695
|
* and seller.
|
|
691
696
|
*
|
|
692
|
-
*
|
|
693
|
-
*
|
|
697
|
+
* Pass `delivery_zip` + `store_id` for one store's price and availability.
|
|
698
|
+
* `domain: "ca"` is available only together with a store target.
|
|
699
|
+
*
|
|
700
|
+
* Costs 1 credit, or 2 when store-targeted. A store that does not carry the
|
|
701
|
+
* item returns 404.
|
|
694
702
|
*/
|
|
695
703
|
async product(options) {
|
|
696
704
|
return this.client._post("/api/v1/walmart/product", options);
|
|
@@ -741,6 +749,20 @@ var WalmartNamespace = class {
|
|
|
741
749
|
async sellerProducts(options) {
|
|
742
750
|
return this.client._post("/api/v1/walmart/seller-products", options);
|
|
743
751
|
}
|
|
752
|
+
/**
|
|
753
|
+
* Walmart stores near a US ZIP (or a Canadian postal code with `domain: "ca"`),
|
|
754
|
+
* nearest first, with `store_id`, address, distance, coordinates and hours.
|
|
755
|
+
* Use a `store_id` with `delivery_zip` on `search()` or `product()`, on the
|
|
756
|
+
* same domain, to target that store.
|
|
757
|
+
*
|
|
758
|
+
* Costs 1 credit.
|
|
759
|
+
*/
|
|
760
|
+
async stores(options) {
|
|
761
|
+
return await this.client._post(
|
|
762
|
+
"/api/v1/walmart/stores",
|
|
763
|
+
options
|
|
764
|
+
);
|
|
765
|
+
}
|
|
744
766
|
};
|
|
745
767
|
|
|
746
768
|
// src/namespaces/youtube.ts
|
|
@@ -2305,6 +2327,141 @@ var MetaAdsNamespace = class {
|
|
|
2305
2327
|
}
|
|
2306
2328
|
};
|
|
2307
2329
|
|
|
2330
|
+
// src/namespaces/costco.ts
|
|
2331
|
+
var CostcoNamespace = class {
|
|
2332
|
+
constructor(client) {
|
|
2333
|
+
this.client = client;
|
|
2334
|
+
}
|
|
2335
|
+
client;
|
|
2336
|
+
/**
|
|
2337
|
+
* Search Costco by keyword or item number: online price and original price,
|
|
2338
|
+
* ratings, member-only and stock flags, promotions and facets. Pass
|
|
2339
|
+
* `warehouse_id` to add that warehouse's in-store price and stock to every
|
|
2340
|
+
* result.
|
|
2341
|
+
*
|
|
2342
|
+
* Costs 1 credit.
|
|
2343
|
+
*/
|
|
2344
|
+
async search(options) {
|
|
2345
|
+
return this.client._post("/api/v1/costco/search", options);
|
|
2346
|
+
}
|
|
2347
|
+
/**
|
|
2348
|
+
* Products in a Costco category, with the same sort, filters and
|
|
2349
|
+
* per-warehouse pricing as search().
|
|
2350
|
+
*
|
|
2351
|
+
* Costs 1 credit.
|
|
2352
|
+
*/
|
|
2353
|
+
async category(options) {
|
|
2354
|
+
return this.client._post("/api/v1/costco/category", options);
|
|
2355
|
+
}
|
|
2356
|
+
/**
|
|
2357
|
+
* The Costco department list, or the full subcategory tree under one
|
|
2358
|
+
* department when `category_id` is set.
|
|
2359
|
+
*
|
|
2360
|
+
* Costs 1 credit.
|
|
2361
|
+
*/
|
|
2362
|
+
async categories(options = {}) {
|
|
2363
|
+
return this.client._post("/api/v1/costco/categories", options);
|
|
2364
|
+
}
|
|
2365
|
+
/**
|
|
2366
|
+
* Full details for up to 20 items: description, features, specifications,
|
|
2367
|
+
* variants, member-only and purchase limits, delivery fee, and promotions
|
|
2368
|
+
* with start and end dates. Pass `item_id` or `item_ids`.
|
|
2369
|
+
*
|
|
2370
|
+
* Costs 1 credit.
|
|
2371
|
+
*/
|
|
2372
|
+
async product(options) {
|
|
2373
|
+
return this.client._post("/api/v1/costco/product", options);
|
|
2374
|
+
}
|
|
2375
|
+
/**
|
|
2376
|
+
* Online price versus in-warehouse price for up to 20 items across up to 10
|
|
2377
|
+
* warehouses, with each location's discount, final price, promotion dates
|
|
2378
|
+
* and price code.
|
|
2379
|
+
*
|
|
2380
|
+
* Costs 1 credit for up to 9 warehouses and 2 credits for 10.
|
|
2381
|
+
*/
|
|
2382
|
+
async prices(options) {
|
|
2383
|
+
return this.client._post("/api/v1/costco/prices", options);
|
|
2384
|
+
}
|
|
2385
|
+
/**
|
|
2386
|
+
* In-warehouse stock status for one item at up to 10 warehouses, plus
|
|
2387
|
+
* pickup and same-day delivery availability. A status, never a quantity.
|
|
2388
|
+
*
|
|
2389
|
+
* Costs 1 credit per 5 warehouses.
|
|
2390
|
+
*/
|
|
2391
|
+
async availability(options) {
|
|
2392
|
+
return this.client._post("/api/v1/costco/availability", options);
|
|
2393
|
+
}
|
|
2394
|
+
/**
|
|
2395
|
+
* Customer reviews for a Costco product with the rating distribution and
|
|
2396
|
+
* recommend count; sort, star filter and up to 100 reviews per page.
|
|
2397
|
+
*
|
|
2398
|
+
* Costs 1 credit.
|
|
2399
|
+
*/
|
|
2400
|
+
async reviews(options) {
|
|
2401
|
+
return this.client._post("/api/v1/costco/reviews", options);
|
|
2402
|
+
}
|
|
2403
|
+
/**
|
|
2404
|
+
* Costco warehouses near a US zip code or coordinates, nearest first:
|
|
2405
|
+
* warehouse_id, address, hours, departments, services, and gas station hours
|
|
2406
|
+
* and prices.
|
|
2407
|
+
*
|
|
2408
|
+
* Costs 1 credit, or 2 with country "ca".
|
|
2409
|
+
*/
|
|
2410
|
+
async warehouses(options) {
|
|
2411
|
+
return this.client._post("/api/v1/costco/warehouses", options);
|
|
2412
|
+
}
|
|
2413
|
+
/**
|
|
2414
|
+
* Current regular, premium and diesel prices at Costco gas stations near a
|
|
2415
|
+
* location, or at specific warehouses.
|
|
2416
|
+
*
|
|
2417
|
+
* Costs 1 credit by location (2 with country "ca"); by `warehouse_ids`,
|
|
2418
|
+
* 1 credit per 5 warehouses on us and 2 credits per warehouse on ca.
|
|
2419
|
+
*/
|
|
2420
|
+
async gas(options) {
|
|
2421
|
+
return this.client._post("/api/v1/costco/gas", options);
|
|
2422
|
+
}
|
|
2423
|
+
/**
|
|
2424
|
+
* The current Costco member coupon book (US): every offer with its item
|
|
2425
|
+
* number, discount, final price, warehouse/online scope, limits and the
|
|
2426
|
+
* book's valid dates. Takes no parameters.
|
|
2427
|
+
*
|
|
2428
|
+
* Costs 1 credit.
|
|
2429
|
+
*/
|
|
2430
|
+
async coupons(options = {}) {
|
|
2431
|
+
return this.client._post("/api/v1/costco/coupons", options);
|
|
2432
|
+
}
|
|
2433
|
+
/**
|
|
2434
|
+
* Costco deal feeds: new items, while supplies last, treasure hunt, member
|
|
2435
|
+
* favorites, online-only and everything on sale, with the same filters and
|
|
2436
|
+
* optional per-warehouse pricing as search().
|
|
2437
|
+
*
|
|
2438
|
+
* Costs 1 credit.
|
|
2439
|
+
*/
|
|
2440
|
+
async deals(options) {
|
|
2441
|
+
return this.client._post("/api/v1/costco/deals", options);
|
|
2442
|
+
}
|
|
2443
|
+
/**
|
|
2444
|
+
* Markdown finder for one warehouse: scans a query or category and keeps
|
|
2445
|
+
* the items whose in-warehouse price ends in a markdown code - .97
|
|
2446
|
+
* clearance and .00/.88 manager markdown by default, .49/.79/.89 special
|
|
2447
|
+
* buys on request. The codes are the member-community decode; Costco does
|
|
2448
|
+
* not publish them. Only items Costco lists online are covered.
|
|
2449
|
+
*
|
|
2450
|
+
* Costs 1 credit.
|
|
2451
|
+
*/
|
|
2452
|
+
async clearance(options) {
|
|
2453
|
+
return this.client._post("/api/v1/costco/clearance", options);
|
|
2454
|
+
}
|
|
2455
|
+
/**
|
|
2456
|
+
* Costco search-box suggestions for a partial query.
|
|
2457
|
+
*
|
|
2458
|
+
* Costs 1 credit.
|
|
2459
|
+
*/
|
|
2460
|
+
async autocomplete(options) {
|
|
2461
|
+
return this.client._post("/api/v1/costco/autocomplete", options);
|
|
2462
|
+
}
|
|
2463
|
+
};
|
|
2464
|
+
|
|
2308
2465
|
// src/client.ts
|
|
2309
2466
|
var DEFAULT_MAX_REQUESTS_PER_SECOND = 1;
|
|
2310
2467
|
var MAX_REQUESTS_PER_SECOND = 50;
|
|
@@ -2340,6 +2497,7 @@ var Scavio = class {
|
|
|
2340
2497
|
companiesHouse;
|
|
2341
2498
|
googleAds;
|
|
2342
2499
|
metaAds;
|
|
2500
|
+
costco;
|
|
2343
2501
|
apiKey;
|
|
2344
2502
|
baseUrl;
|
|
2345
2503
|
timeout;
|
|
@@ -2391,6 +2549,7 @@ var Scavio = class {
|
|
|
2391
2549
|
this.companiesHouse = new CompaniesHouseNamespace(this);
|
|
2392
2550
|
this.googleAds = new GoogleAdsNamespace(this);
|
|
2393
2551
|
this.metaAds = new MetaAdsNamespace(this);
|
|
2552
|
+
this.costco = new CostcoNamespace(this);
|
|
2394
2553
|
}
|
|
2395
2554
|
/** @internal */
|
|
2396
2555
|
async _post(path, body) {
|
|
@@ -2456,6 +2615,7 @@ var Scavio = class {
|
|
|
2456
2615
|
BookingNamespace,
|
|
2457
2616
|
CapterraNamespace,
|
|
2458
2617
|
CompaniesHouseNamespace,
|
|
2618
|
+
CostcoNamespace,
|
|
2459
2619
|
DEFAULT_MAX_REQUESTS_PER_SECOND,
|
|
2460
2620
|
EbayNamespace,
|
|
2461
2621
|
G2Namespace,
|