scrapeunblocker 0.14.1 → 0.15.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -0
- data/README.md +4 -3
- data/lib/scrapeunblocker/plugins.rb +811 -3
- data/lib/scrapeunblocker/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 873882960dc063a6f097269c0eabe1310fedcc29c207ce7c049dae4d22dada23
|
|
4
|
+
data.tar.gz: 6e14cfaa7afc02e1efc5934347b6ec91286a49beeb7daa7b85fa329afa0f0b41
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: f1fecd1b6ad4b3027a96f2d87e53c9935cca6c2071403c4a9175f032e00cf7e5fc439307515d305b5e814013dcde6d04c77682c9500955a8262c341d7b13417c
|
|
7
|
+
data.tar.gz: 05362a2204cd66d3fb1bde32486547c5b300a5c2f3d6a9082ad9e583bde3de13c08f98b04576aaddb2ea877ee48c9602f2ba76fb0c1cdca26050ed71859d0555
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,18 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.15.0 (2026-10-29)
|
|
4
|
+
|
|
5
|
+
- 23 new plugin methods on `Client`:
|
|
6
|
+
- Groceries: `kroger_search(query)` and `kroger_product(product)` (Kroger and its 18 banner stores such as Ralphs and Fred Meyer, priced for one store), `instacart_search(query, retailer: ...)` and `instacart_product(product)` (any retailer on Instacart, priced for a ZIP code), `publix_search(query)` and `publix_product(product)`.
|
|
7
|
+
- E-commerce: `amazon_offers` (every seller's offer for a product, by `asin:` or `url:`), `amazon_seller` (a seller's public profile and feedback, by `seller_id:` or `url:`), `costco_search(query)` and `costco_product(product)`, `flipkart_search(query)` and `flipkart_product(product_id)`, `lazada_search(query)` and `lazada_product(item_id)` (six markets, `country:`), `allegro_search(query)` and `allegro_product(offer_id)`, `falabella_search(query)` and `falabella_product` (Chile, Colombia and Peru), `magazineluiza_search(query)` and `magazineluiza_product`.
|
|
8
|
+
- Google: `google_ai_mode(q)` (the AI Mode answer with the pages it cites) and `google_lens(url)` (Google Lens visual matches for an image URL).
|
|
9
|
+
- AI: `gemini(prompt)` (Google Gemini's answer with its citations and sources).
|
|
10
|
+
- The first argument of `flipkart_product`, `lazada_product`, `allegro_product`, `falabella_search` and `magazineluiza_search` is optional, for calls that pass `url:` instead. `falabella_product` and `magazineluiza_product` take `product_id:` or `url:`.
|
|
11
|
+
- `instacart_search` needs `retailer:` (the retailer's Instacart slug, e.g. `"costco"`); the API answers 400 without it. `instacart_product` takes it too, unless `product` is a link that names the retailer.
|
|
12
|
+
- `shein_search` docs: results also carry `reviewCountText`, and the price filters apply to `price`.
|
|
13
|
+
|
|
14
|
+
No breaking changes.
|
|
15
|
+
|
|
3
16
|
## 0.14.1 (2026-10-25)
|
|
4
17
|
|
|
5
18
|
- `serp(captcha_pause:)` docs: the API ignores `captcha_pause`; it is kept for compatibility. CAPTCHAs are handled server-side by retrying from other exits.
|
data/README.md
CHANGED
|
@@ -149,11 +149,12 @@ jobs = su.career_site_jobs("https://jobs.ashbyhq.com/ramp", keyword: "engineer")
|
|
|
149
149
|
| Category | Methods |
|
|
150
150
|
| --- | --- |
|
|
151
151
|
| Travel | `skyscanner.flights`, `skyscanner.hotels`, `skyscanner.carhire`, `booking_hotels`, `booking_reviews`, `airbnb_search`, `airbnb_room`, `airbnb_reviews`, `tripadvisor_list`, `tripadvisor_reviews` |
|
|
152
|
-
| Google | `google_local`, `google_images`, `maps_place_details`, `google_trends`, `google_shopping`, `google_jobs`, `google_hotels`, `google_hotels_property`, `google_news`, `google_ads_transparency`, `google_ads_transparency_advertisers` |
|
|
152
|
+
| Google | `google_local`, `google_images`, `maps_place_details`, `google_trends`, `google_shopping`, `google_jobs`, `google_hotels`, `google_hotels_property`, `google_news`, `google_ads_transparency`, `google_ads_transparency_advertisers`, `google_ai_mode`, `google_lens` |
|
|
153
|
+
| AI | `gemini` |
|
|
153
154
|
| Search engines | `bing_search`, `bing_images`, `duckduckgo_search`, `yahoo_search`, `yahoo_images`, `brave_search`, `brave_images` |
|
|
154
|
-
| E-commerce | `amazon_product`, `amazon_search`, `amazon_bestsellers`, `amazon_reviews`, `walmart_search`, `target_search`, `bestbuy_search`, `homedepot_search`, `etsy_search`, `shein_search`, `zalando_search`, `aliexpress_search`, `alibaba_search`, `alibaba_product`, `alibaba1688_search`, `alibaba1688_product`, `tiktok_shop_search`, `tiktok_shop_product`, `tiktok_shop_store`, `oopbuy_search`, `smyths_inventory`, `smyths_details`, `smyths_search`, `smyths_category`, `wish_search`, `wish_product`, `shopee_search`, `shopee_product`, `taobao_search`, `taobao_product` |
|
|
155
|
+
| E-commerce | `amazon_product`, `amazon_search`, `amazon_bestsellers`, `amazon_reviews`, `amazon_offers`, `amazon_seller`, `walmart_search`, `target_search`, `bestbuy_search`, `homedepot_search`, `costco_search`, `costco_product`, `etsy_search`, `shein_search`, `zalando_search`, `aliexpress_search`, `alibaba_search`, `alibaba_product`, `alibaba1688_search`, `alibaba1688_product`, `tiktok_shop_search`, `tiktok_shop_product`, `tiktok_shop_store`, `oopbuy_search`, `smyths_inventory`, `smyths_details`, `smyths_search`, `smyths_category`, `wish_search`, `wish_product`, `shopee_search`, `shopee_product`, `taobao_search`, `taobao_product`, `flipkart_search`, `flipkart_product`, `lazada_search`, `lazada_product`, `allegro_search`, `allegro_product`, `falabella_search`, `falabella_product`, `magazineluiza_search`, `magazineluiza_product` |
|
|
155
156
|
| Classifieds & Resale | `ebay_search`, `vinted_search`, `leboncoin_search` |
|
|
156
|
-
| Groceries | `woolworths_search`, `woolworths_category`, `woolworths_specials`, `woolworths_product`, `coles_search`, `coles_category`, `coles_specials`, `coles_product`, `aldi_au_search`, `aldi_au_category`, `aldi_au_specials`, `aldi_au_product` |
|
|
157
|
+
| Groceries | `woolworths_search`, `woolworths_category`, `woolworths_specials`, `woolworths_product`, `coles_search`, `coles_category`, `coles_specials`, `coles_product`, `aldi_au_search`, `aldi_au_category`, `aldi_au_specials`, `aldi_au_product`, `kroger_search`, `kroger_product`, `instacart_search`, `instacart_product`, `publix_search`, `publix_product` |
|
|
157
158
|
| Real estate | `zillow_search`, `zillow_detail`, `redfin_search`, `redfin_detail`, `rightmove_search`, `idealista_search`, `idealista_detail`, `immobilienscout24_search` |
|
|
158
159
|
| Vehicles | `mobile_de`, `autoscout24_search` |
|
|
159
160
|
| Jobs | `indeed_jobs`, `glassdoor_jobs`, `glassdoor_reviews`, `glassdoor_salaries`, `naukri_jobs`, `wellfound_jobs`, `wellfound_job`, `wellfound_company`, `wellfound_company_search`, `career_site_jobs`, `upwork_jobs` |
|
|
@@ -105,6 +105,102 @@ module ScrapeUnblocker
|
|
|
105
105
|
})
|
|
106
106
|
end
|
|
107
107
|
|
|
108
|
+
# Scrape every offer (all sellers) for an Amazon product.
|
|
109
|
+
#
|
|
110
|
+
# Returns the offers Amazon lists for a product under "All buying options", as JSON. Each
|
|
111
|
+
# offer has the seller (`sellerName`, `sellerId`, `sellerUrl`, `sellerRating`,
|
|
112
|
+
# `sellerRatingCount`, `sellerPositivePercent`), `price` + `currency`, `listPrice` and
|
|
113
|
+
# `savingsPercent`, `shippingPrice` and `totalPrice`, `condition` (as Amazon words it) with
|
|
114
|
+
# `conditionType` (new, used, collectible, refurbished) and the seller's `conditionNote`,
|
|
115
|
+
# `delivery` / `deliveryDate`, `alternativeDelivery` (the faster or Prime-member option
|
|
116
|
+
# Amazon also offers), `shipsFrom`, `soldByAmazon`, `fulfilledByAmazon`, `prime`, the
|
|
117
|
+
# `offerId` and a `buyBoxWinner` flag. The response also carries the product `title`,
|
|
118
|
+
# `rating`, `reviewCount`, `image` and `totalOffers`. Page 1 starts with the Buy Box
|
|
119
|
+
# (featured) offer, also returned as `buyBox`; Amazon shows it whatever filter is set.
|
|
120
|
+
# Amazon lists 10 other offers per page: follow `nextPage`, or read up to 5 pages in one
|
|
121
|
+
# call with `pages`. Filter with `condition`, `free_shipping` and `prime`. `shippingPrice`
|
|
122
|
+
# is 0 for free delivery and null when Amazon shows no delivery promise for the location.
|
|
123
|
+
# `proxy_country` defaults to the marketplace's home country so prices and delivery are the
|
|
124
|
+
# ones a local shopper sees; the proxy that served the request is reported as `exitCountry`
|
|
125
|
+
# and `exitProvider`.
|
|
126
|
+
#
|
|
127
|
+
# @param asin [String] Amazon product id (10-char ASIN, e.g. 'B07VGRJDFY'; books use their
|
|
128
|
+
# ISBN-10). Give either `asin` (with `marketplace`) or a product `url`.
|
|
129
|
+
# @param url [String] Full Amazon product url, e.g. 'https://www.amazon.de/dp/1847941834'.
|
|
130
|
+
# The ASIN and marketplace are read from it.
|
|
131
|
+
# @param marketplace [String] Regional Amazon site whose offers to list. Prices, sellers and
|
|
132
|
+
# delivery differ per marketplace. One of: amazon.com, amazon.co.uk, amazon.de, amazon.fr,
|
|
133
|
+
# amazon.it, amazon.es, amazon.nl, amazon.se, amazon.pl, amazon.com.be, amazon.com.tr,
|
|
134
|
+
# amazon.ca, amazon.com.mx, amazon.com.br, amazon.co.jp, amazon.in, amazon.ae, amazon.sa,
|
|
135
|
+
# amazon.sg, amazon.com.au. API default: amazon.com.
|
|
136
|
+
# @param condition [String] Only offers in this condition: all, new, used, used_like_new,
|
|
137
|
+
# used_very_good, used_good, used_acceptable. Default 'all'. API default: all.
|
|
138
|
+
# @param free_shipping [Boolean] Only offers with free delivery. API default: false.
|
|
139
|
+
# @param prime [Boolean] Only Prime-eligible offers. API default: false.
|
|
140
|
+
# @param page [Integer] 1-based offers page to start at. Amazon lists 10 offers per page
|
|
141
|
+
# (page 1 also carries the Buy Box offer and the product header). API default: 1.
|
|
142
|
+
# @param pages [Integer] How many consecutive pages to read in this call (1-5). Reading
|
|
143
|
+
# stops early at the last page. Each page delivered is billed as one request. API default:
|
|
144
|
+
# 1.
|
|
145
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US',
|
|
146
|
+
# 'DE'). Defaults to the marketplace's home country, so prices, shipping costs and delivery
|
|
147
|
+
# dates are the ones a local shopper sees.
|
|
148
|
+
# Calls POST /marketplace/amazon-offers.
|
|
149
|
+
def amazon_offers(asin: nil, url: nil, marketplace: nil, condition: nil, free_shipping: nil, prime: nil, page: nil, pages: nil, proxy_country: nil)
|
|
150
|
+
post_json("/marketplace/amazon-offers", {
|
|
151
|
+
asin: asin,
|
|
152
|
+
url: url,
|
|
153
|
+
marketplace: marketplace,
|
|
154
|
+
condition: condition,
|
|
155
|
+
free_shipping: free_shipping,
|
|
156
|
+
prime: prime,
|
|
157
|
+
page: page,
|
|
158
|
+
pages: pages,
|
|
159
|
+
proxy_country: proxy_country,
|
|
160
|
+
})
|
|
161
|
+
end
|
|
162
|
+
|
|
163
|
+
# Scrape an Amazon seller profile.
|
|
164
|
+
#
|
|
165
|
+
# Returns an Amazon third-party seller's public profile as JSON: `name`, `logo`,
|
|
166
|
+
# `storefrontUrl`, the headline `rating`, `ratingCount` and `positivePercent` Amazon shows
|
|
167
|
+
# (the last 12 months, or lifetime for a seller with few recent ratings; `ratingPeriod` says
|
|
168
|
+
# which), and `feedbackSummary` for `30d`, `90d`, `12m` and `lifetime` - each with the
|
|
169
|
+
# average `rating`, `ratingCount`, `positivePercent` / `neutralPercent` / `negativePercent`
|
|
170
|
+
# (4-5, 3 and 1-2 stars), the matching counts and the 5..1-star histogram. Also `about` (the
|
|
171
|
+
# seller's own description), `businessName`, `businessAddress` (+ `businessAddressLines`),
|
|
172
|
+
# `businessDetails` (every row of Amazon's "Detailed Seller Information", e.g. business
|
|
173
|
+
# type, trade register and VAT number where the marketplace shows them), `contactUrl` and
|
|
174
|
+
# `recentFeedback` (each with `rating`, `text`, `author`, `date`, `struckThrough` and
|
|
175
|
+
# `amazonMessage` when Amazon took responsibility for the order). Identify the seller with
|
|
176
|
+
# `seller_id` (plus `marketplace`), e.g. the `sellerId` of an offer from
|
|
177
|
+
# `/marketplace/amazon-offers`, or with any Amazon `url` naming the seller. Ratings are per
|
|
178
|
+
# marketplace. Amazon's own retail sellers show no ratings; those fields are null.
|
|
179
|
+
#
|
|
180
|
+
# @param seller_id [String] Amazon seller (merchant) id, e.g. 'A3PEV2IAOYFKM6' - the
|
|
181
|
+
# `sellerId` of an offer from /marketplace/amazon-offers. Give either `seller_id` (with
|
|
182
|
+
# `marketplace`) or a `url`.
|
|
183
|
+
# @param url [String] Any Amazon url that names the seller: the profile
|
|
184
|
+
# ('https://www.amazon.de/sp?seller=A1J6G6R957S11K'), a storefront ('/s?me=...') or a sold-
|
|
185
|
+
# by link. The seller id and marketplace are read from it.
|
|
186
|
+
# @param marketplace [String] Regional Amazon site the seller sells on. Ratings and feedback
|
|
187
|
+
# are per marketplace. One of: amazon.com, amazon.co.uk, amazon.de, amazon.fr, amazon.it,
|
|
188
|
+
# amazon.es, amazon.nl, amazon.se, amazon.pl, amazon.com.be, amazon.com.tr, amazon.ca,
|
|
189
|
+
# amazon.com.mx, amazon.com.br, amazon.co.jp, amazon.in, amazon.ae, amazon.sa, amazon.sg,
|
|
190
|
+
# amazon.com.au. API default: amazon.com.
|
|
191
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US',
|
|
192
|
+
# 'DE'). Optional: a seller profile is the same from every country, so by default any
|
|
193
|
+
# country is used.
|
|
194
|
+
# Calls POST /marketplace/amazon-seller.
|
|
195
|
+
def amazon_seller(seller_id: nil, url: nil, marketplace: nil, proxy_country: nil)
|
|
196
|
+
post_json("/marketplace/amazon-seller", {
|
|
197
|
+
seller_id: seller_id,
|
|
198
|
+
url: url,
|
|
199
|
+
marketplace: marketplace,
|
|
200
|
+
proxy_country: proxy_country,
|
|
201
|
+
})
|
|
202
|
+
end
|
|
203
|
+
|
|
108
204
|
# Scrape Walmart product search.
|
|
109
205
|
#
|
|
110
206
|
# Searches Walmart for `query` and returns the products as JSON: `name`, `price` and
|
|
@@ -221,6 +317,72 @@ module ScrapeUnblocker
|
|
|
221
317
|
})
|
|
222
318
|
end
|
|
223
319
|
|
|
320
|
+
# Scrape Costco product search.
|
|
321
|
+
#
|
|
322
|
+
# Searches Costco (costco.com, or costco.ca with `domain=ca`) for `query` and returns
|
|
323
|
+
# `products`: `productId`, `itemNumber`, `title`, `brand`, `url`, `image`, `price` (and
|
|
324
|
+
# `priceMax` for a price range), `wasPrice`, `currency`, `promotions`, `marketingStatement`,
|
|
325
|
+
# `rating`, `reviewCount`, `memberOnly`, `onlineOnly`, `warehouseOnly`, `badges`,
|
|
326
|
+
# `availability`, `inWarehouse`, `shipIt`, `buyableOnline`, `priceShownInCartOnly`, `model`,
|
|
327
|
+
# `features`, `category`, `categoryPath` and `variantCount`, plus `totalResults`. A keyword
|
|
328
|
+
# Costco sends to a category page (e.g. 'tv') returns that category, named in
|
|
329
|
+
# `redirectedTo`. `availability` is for the delivery location in `deliveryLocation`. Sort
|
|
330
|
+
# with `sort`, start at `page` and collect up to `max_results`.
|
|
331
|
+
#
|
|
332
|
+
# @param query [String] What to search for, e.g. 'paper towels', 'kirkland coffee' or 'tv'.
|
|
333
|
+
# @param sort [String] Result order: relevance, price_low, price_high, rating, newest,
|
|
334
|
+
# brand. API default: relevance.
|
|
335
|
+
# @param page [Integer] 1-based results page to start from (24 products per page, as on
|
|
336
|
+
# costco.com). API default: 1.
|
|
337
|
+
# @param max_results [Integer] Products to collect from `page` on, 1-240. Up to 120 are
|
|
338
|
+
# billed as one request. API default: 24.
|
|
339
|
+
# @param include_out_of_stock [Boolean] Also return products that are out of stock (Costco
|
|
340
|
+
# hides them by default). API default: false.
|
|
341
|
+
# @param domain [String] Costco storefront: com, ca ('com' is costco.com, 'ca' is
|
|
342
|
+
# costco.ca). API default: com.
|
|
343
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US').
|
|
344
|
+
# Leave blank to let the API pick. Costco computes `availability` for the delivery location
|
|
345
|
+
# it derives from the visitor; it is named in `deliveryLocation`.
|
|
346
|
+
# Calls POST /marketplace/costco-search.
|
|
347
|
+
def costco_search(query, sort: nil, page: nil, max_results: nil, include_out_of_stock: nil, domain: nil, proxy_country: nil)
|
|
348
|
+
post_json("/marketplace/costco-search", {
|
|
349
|
+
query: query,
|
|
350
|
+
sort: sort,
|
|
351
|
+
page: page,
|
|
352
|
+
max_results: max_results,
|
|
353
|
+
include_out_of_stock: include_out_of_stock,
|
|
354
|
+
domain: domain,
|
|
355
|
+
proxy_country: proxy_country,
|
|
356
|
+
})
|
|
357
|
+
end
|
|
358
|
+
|
|
359
|
+
# Scrape a Costco product.
|
|
360
|
+
#
|
|
361
|
+
# Returns one Costco product (costco.com, or costco.ca with `domain=ca`) as JSON:
|
|
362
|
+
# `productId`, `itemNumber`, `title`, `brand`, `url`, `price`, `discount`, `currency`,
|
|
363
|
+
# `rating`, `reviewCount`, `memberOnly`, `onlineOnly`, `buyable`, `inWarehouse`, `shipIt`,
|
|
364
|
+
# `description`, `details`, `features`, `marketingStatement`, `promotionalStatement`,
|
|
365
|
+
# `shipping`, `manufacturer`, `manufacturerPartNumber`, `upc`, `minQuantity`, `maxQuantity`,
|
|
366
|
+
# `categoryPath`, `specifications`, `images`, `options` (e.g. every color and size) and
|
|
367
|
+
# `variants` (item number, option values, UPC). `product` takes a product id, an item number
|
|
368
|
+
# or a product link.
|
|
369
|
+
#
|
|
370
|
+
# @param product [String] A Costco product id ('100361434'), item number ('17767') or
|
|
371
|
+
# product link (https://www.costco.com/p/-/kirkland-signature-colombian-coffee-dark-
|
|
372
|
+
# roast-3-lbs/100361434).
|
|
373
|
+
# @param domain [String] Costco storefront: com, ca ('com' is costco.com, 'ca' is
|
|
374
|
+
# costco.ca). Defaults to the link's storefront, else 'com'.
|
|
375
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US').
|
|
376
|
+
# Leave blank to let the API pick.
|
|
377
|
+
# Calls POST /marketplace/costco-product.
|
|
378
|
+
def costco_product(product, domain: nil, proxy_country: nil)
|
|
379
|
+
post_json("/marketplace/costco-product", {
|
|
380
|
+
product: product,
|
|
381
|
+
domain: domain,
|
|
382
|
+
proxy_country: proxy_country,
|
|
383
|
+
})
|
|
384
|
+
end
|
|
385
|
+
|
|
224
386
|
# Scrape Etsy product search.
|
|
225
387
|
#
|
|
226
388
|
# Searches Etsy for `query` and returns the products as JSON: `title`, `price` + `currency`
|
|
@@ -258,9 +420,10 @@ module ScrapeUnblocker
|
|
|
258
420
|
#
|
|
259
421
|
# Searches SHEIN (US store) for `query` and returns the products as JSON: `id`, `sku`,
|
|
260
422
|
# `title`, `url`, `price`, `originalPrice`, `currency`, `discountPercent`, `rating`,
|
|
261
|
-
# `reviewCount`, `category`, `image`, `soldOut` and `storeCode`. Order
|
|
262
|
-
# with `min_price`/`max_price
|
|
263
|
-
# SHEIN does not serve deeper pages to automated
|
|
423
|
+
# `reviewCount`, `reviewCountText`, `category`, `image`, `soldOut` and `storeCode`. Order
|
|
424
|
+
# with `sort`, filter with `min_price`/`max_price` (enforced on `price`). Returns SHEIN's
|
|
425
|
+
# first results page (up to 20 products); SHEIN does not serve deeper pages to automated
|
|
426
|
+
# clients.
|
|
264
427
|
#
|
|
265
428
|
# @param query [String] What to search, e.g. 'summer dress' or 'phone case'.
|
|
266
429
|
# @param sort [String] Ordering: relevance, price_low, price_high. API default: relevance.
|
|
@@ -777,6 +940,341 @@ module ScrapeUnblocker
|
|
|
777
940
|
})
|
|
778
941
|
end
|
|
779
942
|
|
|
943
|
+
# Scrape Flipkart product search.
|
|
944
|
+
#
|
|
945
|
+
# Searches Flipkart (India) for `query` and returns `products`, each with `productId` (the
|
|
946
|
+
# FSN, e.g. MOBHQV9YCUG4ZQZK), `itemId`, `listingId`, `title`, `brand`, `price`,
|
|
947
|
+
# `originalPrice` (MRP when discounted), `discountPercent`, `currency` (INR), `rating`,
|
|
948
|
+
# `ratingCount`, `reviewCount`, `assured` (F-Assured), `sponsored`, `badge` (e.g.
|
|
949
|
+
# Bestseller), `availability` / `inStock`, `keySpecs`, `category`, `image`, `images` and
|
|
950
|
+
# `url`. Also `total` (results Flipkart reports), `totalPages` and `nextPage`. `sort` orders
|
|
951
|
+
# the results, `min_price` / `max_price` filter by price, `page` picks the first page and
|
|
952
|
+
# `max_results` collects across pages. One unit per Flipkart page read.
|
|
953
|
+
#
|
|
954
|
+
# @param query [String] What to search for, e.g. 'iphone 17' or 'running shoes'.
|
|
955
|
+
# @param sort [String] Result order: relevance, popularity, price_asc, price_desc, newest.
|
|
956
|
+
# API default: relevance.
|
|
957
|
+
# @param page [Integer] First Flipkart results page to read (1-100). API default: 1.
|
|
958
|
+
# @param min_price [Integer] Lowest price in rupees (INR).
|
|
959
|
+
# @param max_price [Integer] Highest price in rupees (INR).
|
|
960
|
+
# @param max_results [Integer] Products to collect across pages, 1-240. A page holds 24
|
|
961
|
+
# products (40 in fashion grids); each page is one unit. API default: 24.
|
|
962
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US',
|
|
963
|
+
# 'DE'). Leave blank to let the API pick.
|
|
964
|
+
# Calls POST /marketplace/flipkart-search.
|
|
965
|
+
def flipkart_search(query, sort: nil, page: nil, min_price: nil, max_price: nil, max_results: nil, proxy_country: nil)
|
|
966
|
+
post_json("/marketplace/flipkart-search", {
|
|
967
|
+
query: query,
|
|
968
|
+
sort: sort,
|
|
969
|
+
page: page,
|
|
970
|
+
min_price: min_price,
|
|
971
|
+
max_price: max_price,
|
|
972
|
+
max_results: max_results,
|
|
973
|
+
proxy_country: proxy_country,
|
|
974
|
+
})
|
|
975
|
+
end
|
|
976
|
+
|
|
977
|
+
# Scrape a Flipkart product.
|
|
978
|
+
#
|
|
979
|
+
# Returns one Flipkart (India) product by `product_id` (FSN or `itm` id) or `url`:
|
|
980
|
+
# `productId`, `itemId`, `listingId`, `title`, `brand`, `description`, `category`,
|
|
981
|
+
# `breadcrumbs`, `price`, `originalPrice` (MRP), `discountPercent`, `bestPrice` (with bank
|
|
982
|
+
# offers), `currency`, `availability` / `inStock`, `rating`, `ratingCount`, `reviewCount`,
|
|
983
|
+
# `ratingBreakdown` (1-5 stars), `aspectRatings` (e.g. Camera, Battery), `assured`,
|
|
984
|
+
# `selectedAttributes`, `highlights`, `specifications` (group / name / value), `images`,
|
|
985
|
+
# `variants` (dimension, value, productId, price, available, url), `seller` (name, id,
|
|
986
|
+
# rating, years on Flipkart), `offerCount`, `offers` (bank and exchange offers),
|
|
987
|
+
# `deliveryDays`, `cashOnDelivery`, `returnPolicy`, `reviews` (top reviews) and `url`. One
|
|
988
|
+
# unit per call.
|
|
989
|
+
#
|
|
990
|
+
# @param product_id [String] Flipkart product id: the FSN (`pid`, e.g. MOBHQV9YCUG4ZQZK) or
|
|
991
|
+
# the `itm...` id from a product link.
|
|
992
|
+
# @param url [String] A Flipkart product link, e.g. https://www.flipkart.com/apple-
|
|
993
|
+
# iphone-17-black-256-gb/p/itmc61b40459dd98?pid=MOBHQV9YCUG4ZQZK. Used instead of
|
|
994
|
+
# `product_id`.
|
|
995
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US',
|
|
996
|
+
# 'DE'). Leave blank to let the API pick.
|
|
997
|
+
# Calls POST /marketplace/flipkart-product.
|
|
998
|
+
def flipkart_product(product_id = nil, url: nil, proxy_country: nil)
|
|
999
|
+
post_json("/marketplace/flipkart-product", {
|
|
1000
|
+
product_id: product_id,
|
|
1001
|
+
url: url,
|
|
1002
|
+
proxy_country: proxy_country,
|
|
1003
|
+
})
|
|
1004
|
+
end
|
|
1005
|
+
|
|
1006
|
+
# Scrape Lazada product search.
|
|
1007
|
+
#
|
|
1008
|
+
# Searches a Lazada market (Singapore, Malaysia, Thailand, Philippines, Vietnam or Indonesia
|
|
1009
|
+
# - `country`) for `query` and returns `products`, each with `position`, `itemId`, `skuId`,
|
|
1010
|
+
# `name`, `price`, `priceText`, `originalPrice` (crossed-out price), `discountPercent`,
|
|
1011
|
+
# `rating`, `reviewCount`, `sold` (units sold, as Lazada rounds it) and `soldText`, `brand`,
|
|
1012
|
+
# `seller` / `sellerId`, `location` (ships from), `inStock`, `lazMall`, `freeShipping`,
|
|
1013
|
+
# `sponsored`, `image` and `url`. Also `total` (results Lazada lets you page through),
|
|
1014
|
+
# `matches` (all matches it reports), `currency` and `nextPage`. `sort` orders the results,
|
|
1015
|
+
# `min_price` / `max_price` filter by price, `page` picks the first page and `max_results`
|
|
1016
|
+
# collects across pages (40 a page). One unit per Lazada page read.
|
|
1017
|
+
#
|
|
1018
|
+
# @param query [String] What to search for, e.g. 'iphone case' or 'air fryer'.
|
|
1019
|
+
# @param country [String] Lazada market: 'sg' (lazada.sg), 'my' (lazada.com.my), 'th'
|
|
1020
|
+
# (lazada.co.th), 'ph' (lazada.com.ph), 'vn' (lazada.vn), 'id' (lazada.co.id). Default 'sg'.
|
|
1021
|
+
# Prices are in that market's currency.
|
|
1022
|
+
# @param sort [String] Result order: relevance, price_asc, price_desc. API default:
|
|
1023
|
+
# relevance.
|
|
1024
|
+
# @param page [Integer] First Lazada results page to read (1-102). API default: 1.
|
|
1025
|
+
# @param min_price [Float] Lowest price, in the market's currency.
|
|
1026
|
+
# @param max_price [Float] Highest price, in the market's currency.
|
|
1027
|
+
# @param max_results [Integer] Products to collect across pages, 1-400. A page holds 40
|
|
1028
|
+
# products; each page is one unit. API default: 40.
|
|
1029
|
+
# Calls POST /marketplace/lazada-search.
|
|
1030
|
+
def lazada_search(query, country: nil, sort: nil, page: nil, min_price: nil, max_price: nil, max_results: nil)
|
|
1031
|
+
post_json("/marketplace/lazada-search", {
|
|
1032
|
+
query: query,
|
|
1033
|
+
country: country,
|
|
1034
|
+
sort: sort,
|
|
1035
|
+
page: page,
|
|
1036
|
+
min_price: min_price,
|
|
1037
|
+
max_price: max_price,
|
|
1038
|
+
max_results: max_results,
|
|
1039
|
+
})
|
|
1040
|
+
end
|
|
1041
|
+
|
|
1042
|
+
# Scrape a Lazada product.
|
|
1043
|
+
#
|
|
1044
|
+
# Returns one Lazada product by `url`, or by `item_id` (+ optional `sku_id`) and `country`:
|
|
1045
|
+
# `itemId`, `skuId`, `title`, `brand`, `description`, `highlights`, `categories`, `price`,
|
|
1046
|
+
# `priceText`, `originalPrice`, `discountPercent`, `currency`, `rating`, `reviewCount`,
|
|
1047
|
+
# `ratingBreakdown` (1-5 stars), `inStock`, `maxQuantity`, `images`, `specifications`,
|
|
1048
|
+
# `variantDimensions` (e.g. colour and storage with their values), `variants` (every SKU
|
|
1049
|
+
# with its attributes, price, image, availability and link), `seller` (id, shop id, name,
|
|
1050
|
+
# positive rating, store highlights, link), `delivery` (option and estimate), `warranties`
|
|
1051
|
+
# and `url`. One unit per call.
|
|
1052
|
+
#
|
|
1053
|
+
# @param item_id [String] Lazada item id (the number after '-i' in a product link, e.g.
|
|
1054
|
+
# 13907853520).
|
|
1055
|
+
# @param url [String] A Lazada product link, e.g. https://www.lazada.sg/products/apple-
|
|
1056
|
+
# iphone-18-pro-max-i13907853520-s124925162667.html. Used instead of `item_id` / `country`.
|
|
1057
|
+
# @param sku_id [String] Optional SKU id (the number after '-s') to select one variant.
|
|
1058
|
+
# @param country [String] Lazada market: 'sg' (lazada.sg), 'my' (lazada.com.my), 'th'
|
|
1059
|
+
# (lazada.co.th), 'ph' (lazada.com.ph), 'vn' (lazada.vn), 'id' (lazada.co.id). Default 'sg'.
|
|
1060
|
+
# Prices are in that market's currency.
|
|
1061
|
+
# Calls POST /marketplace/lazada-product.
|
|
1062
|
+
def lazada_product(item_id = nil, url: nil, sku_id: nil, country: nil)
|
|
1063
|
+
post_json("/marketplace/lazada-product", {
|
|
1064
|
+
item_id: item_id,
|
|
1065
|
+
url: url,
|
|
1066
|
+
sku_id: sku_id,
|
|
1067
|
+
country: country,
|
|
1068
|
+
})
|
|
1069
|
+
end
|
|
1070
|
+
|
|
1071
|
+
# Scrape Allegro search.
|
|
1072
|
+
#
|
|
1073
|
+
# Searches Allegro (Poland) for `query` and returns `products` (offers), each with
|
|
1074
|
+
# `position`, `offerId`, `productId`, `title`, `price`, `currency` (PLN),
|
|
1075
|
+
# `priceWithDelivery`, `deliveryFrom`, `freeDelivery`, `smart` (Allegro Smart!),
|
|
1076
|
+
# `deliveryLabel`, `rating`, `ratingCount`, `recentBuyers` (bought in the last 30 days),
|
|
1077
|
+
# `offersOfProduct`, `quantity`, `sponsored`, `promoted`, `superSeller`, `seller` (id,
|
|
1078
|
+
# login, company, positive feedback %), `parameters`, `image`, `images` and `url`. Also
|
|
1079
|
+
# `total` (all matches), `available` (offers Allegro lets you page through), `lastPage`,
|
|
1080
|
+
# `category` and `nextPage`. Filter with `min_price` / `max_price`, `condition`,
|
|
1081
|
+
# `shipping_from`, `delivery_time`, `offer_type`, `store_city` and `store_region`; `sort`
|
|
1082
|
+
# orders the results; `page` and `max_results` page through them. One unit per Allegro page
|
|
1083
|
+
# read.
|
|
1084
|
+
#
|
|
1085
|
+
# @param query [String] What to search for, e.g. 'iphone 15' or 'lego'.
|
|
1086
|
+
# @param sort [String] Result order: relevance, price_asc, price_desc,
|
|
1087
|
+
# price_with_delivery_asc, price_with_delivery_desc, popularity, ending_soon, newest,
|
|
1088
|
+
# rating. API default: relevance.
|
|
1089
|
+
# @param page [Integer] First Allegro results page to read (1-100). API default: 1.
|
|
1090
|
+
# @param min_price [Float] Lowest price in zloty (PLN).
|
|
1091
|
+
# @param max_price [Float] Highest price in zloty (PLN).
|
|
1092
|
+
# @param condition [String] Item condition: new, like_new, refurbished, used, damaged.
|
|
1093
|
+
# @param shipping_from [String] Where the item ships from: pl, cz, sk, hu, cn, eu,
|
|
1094
|
+
# allegro_warehouse.
|
|
1095
|
+
# @param delivery_time [String] Delivery speed: one_day, two_days, three_days (one_day =
|
|
1096
|
+
# tomorrow).
|
|
1097
|
+
# @param offer_type [String] Offer type: buy_now, auction (buy_now = fixed price).
|
|
1098
|
+
# @param store_city [String] The seller's town, e.g. 'Kraków'.
|
|
1099
|
+
# @param store_region [String] The seller's voivodeship, e.g. 'mazowieckie' or 'slaskie' (or
|
|
1100
|
+
# its number 1-16).
|
|
1101
|
+
# @param max_results [Integer] Offers to collect across pages, 1-300. A page holds about 60
|
|
1102
|
+
# offers; each page is one unit. API default: 60.
|
|
1103
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'DE',
|
|
1104
|
+
# 'CZ'). Leave blank to let the API pick (recommended: Allegro refuses some countries'
|
|
1105
|
+
# exits).
|
|
1106
|
+
# Calls POST /marketplace/allegro-search.
|
|
1107
|
+
def allegro_search(query, sort: nil, page: nil, min_price: nil, max_price: nil, condition: nil, shipping_from: nil, delivery_time: nil, offer_type: nil, store_city: nil, store_region: nil, max_results: nil, proxy_country: nil)
|
|
1108
|
+
post_json("/marketplace/allegro-search", {
|
|
1109
|
+
query: query,
|
|
1110
|
+
sort: sort,
|
|
1111
|
+
page: page,
|
|
1112
|
+
min_price: min_price,
|
|
1113
|
+
max_price: max_price,
|
|
1114
|
+
condition: condition,
|
|
1115
|
+
shipping_from: shipping_from,
|
|
1116
|
+
delivery_time: delivery_time,
|
|
1117
|
+
offer_type: offer_type,
|
|
1118
|
+
store_city: store_city,
|
|
1119
|
+
store_region: store_region,
|
|
1120
|
+
max_results: max_results,
|
|
1121
|
+
proxy_country: proxy_country,
|
|
1122
|
+
})
|
|
1123
|
+
end
|
|
1124
|
+
|
|
1125
|
+
# Scrape an Allegro offer.
|
|
1126
|
+
#
|
|
1127
|
+
# Returns one Allegro (Poland) offer by `offer_id` or `url`: `offerId`, `productId`,
|
|
1128
|
+
# `title`, `productName`, `brand`, `gtin`, `price`, `originalPrice`, `currency`,
|
|
1129
|
+
# `sellingMode` (buy_now / auction), `available` (stock), `condition`, `rating`,
|
|
1130
|
+
# `ratingCount`, `ratingBreakdown` (1-5 stars), `recentBuyers`, `category`, `breadcrumbs`,
|
|
1131
|
+
# `parameters` (group / name / value), `description`, `images`, `variants`, `seller` (id,
|
|
1132
|
+
# login, Super Seller, feedback, company name and tax id), `location`, `smart`, `delivery`,
|
|
1133
|
+
# `shippingMethods` (name, price, Smart! price), `returnDays`, `warranty`, `endsAt`,
|
|
1134
|
+
# `reviews` and `url`. One unit per call.
|
|
1135
|
+
#
|
|
1136
|
+
# @param offer_id [String] Allegro offer number, e.g. 16680923539 (the number at the end of
|
|
1137
|
+
# an /oferta/ link or its offerId).
|
|
1138
|
+
# @param url [String] An Allegro offer or product link, e.g.
|
|
1139
|
+
# https://allegro.pl/oferta/16680923539 or
|
|
1140
|
+
# https://allegro.pl/produkt/...?offerId=16680923539. Used instead of `offer_id`.
|
|
1141
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'DE',
|
|
1142
|
+
# 'CZ'). Leave blank to let the API pick (recommended: Allegro refuses some countries'
|
|
1143
|
+
# exits).
|
|
1144
|
+
# Calls POST /marketplace/allegro-product.
|
|
1145
|
+
def allegro_product(offer_id = nil, url: nil, proxy_country: nil)
|
|
1146
|
+
post_json("/marketplace/allegro-product", {
|
|
1147
|
+
offer_id: offer_id,
|
|
1148
|
+
url: url,
|
|
1149
|
+
proxy_country: proxy_country,
|
|
1150
|
+
})
|
|
1151
|
+
end
|
|
1152
|
+
|
|
1153
|
+
# Scrape Falabella product search (Chile, Colombia, Peru).
|
|
1154
|
+
#
|
|
1155
|
+
# Searches a Falabella store (`country`: cl, co or pe) for `query` and returns the products
|
|
1156
|
+
# as JSON: `productId`, `skuId`, `title`, `brand`, `url`, `price` (and `priceMax` for a
|
|
1157
|
+
# price range), `cmrPrice` (price with the Falabella CMR card), `originalPrice` (crossed-out
|
|
1158
|
+
# list price), `discountPercent`, `currency`, `rating`, `reviewCount`, `sellerId` /
|
|
1159
|
+
# `sellerName`, `sponsored`, `bestSeller`, `freeShipping`, `deliveryLabels`, `badges`,
|
|
1160
|
+
# `installmentsWithoutInterest`, `image` / `images`, `topSpecifications` and `variants`
|
|
1161
|
+
# (colour / size options). Sort with `sort`, start at `page`, and collect up to
|
|
1162
|
+
# `max_results` across pages (48 per page, each page billed as one request, see
|
|
1163
|
+
# `pagesFetched`). `url` takes a Falabella search link instead of `query`.
|
|
1164
|
+
#
|
|
1165
|
+
# @param query [String] What to search, e.g. 'zapatillas nike', 'televisor 55' or
|
|
1166
|
+
# 'notebook'.
|
|
1167
|
+
# @param country [String] Falabella store: 'cl' (www.falabella.com, prices in CLP), 'co'
|
|
1168
|
+
# (www.falabella.com.co, prices in COP), 'pe' (www.falabella.com.pe, prices in PEN). Default
|
|
1169
|
+
# 'cl'.
|
|
1170
|
+
# @param sort [String] Order: relevance, price_low, price_high, rating, brand. API default:
|
|
1171
|
+
# relevance.
|
|
1172
|
+
# @param page [Integer] 1-based results page to start from (1-200). API default: 1.
|
|
1173
|
+
# @param max_results [Integer] Products to collect across pages (1-480); Falabella serves 48
|
|
1174
|
+
# per page and each page fetched is billed as one request. API default: 48.
|
|
1175
|
+
# @param url [String] Instead of `query`: a Falabella search link, e.g.
|
|
1176
|
+
# https://www.falabella.com/falabella-cl/search?Ntt=zapatillas (its store, sort and page are
|
|
1177
|
+
# used).
|
|
1178
|
+
# Calls POST /marketplace/falabella-search.
|
|
1179
|
+
def falabella_search(query = nil, country: nil, sort: nil, page: nil, max_results: nil, url: nil)
|
|
1180
|
+
post_json("/marketplace/falabella-search", {
|
|
1181
|
+
query: query,
|
|
1182
|
+
country: country,
|
|
1183
|
+
sort: sort,
|
|
1184
|
+
page: page,
|
|
1185
|
+
max_results: max_results,
|
|
1186
|
+
url: url,
|
|
1187
|
+
})
|
|
1188
|
+
end
|
|
1189
|
+
|
|
1190
|
+
# Scrape one Falabella product's full detail.
|
|
1191
|
+
#
|
|
1192
|
+
# Fetches one Falabella product (by `product_id` + `country`, or `url`) and returns it as
|
|
1193
|
+
# JSON: `productId`, `skuId`, `url`, `title`, `brand`, `description` / `longDescription`,
|
|
1194
|
+
# `price` / `priceMax`, `cmrPrice`, `originalPrice`, `discountPercent`, `currency`,
|
|
1195
|
+
# `available`, `stockUnits` (only when Falabella shows a low-stock count), `rating`,
|
|
1196
|
+
# `reviewCount`, `sellerId` / `sellerName`, `freeShipping`, `deliveryLabels`, `badges`,
|
|
1197
|
+
# `installmentsWithoutInterest`, `breadcrumbs`, `topSpecifications`, `specifications` (name
|
|
1198
|
+
# / value), `images`, `videos`, `returnPolicy` and `variants` (every colour / size SKU with
|
|
1199
|
+
# its own price, availability, delivery options and images). The top-level price fields are
|
|
1200
|
+
# those of `sku` (default the main variant). Companion to `/marketplace/falabella-search`.
|
|
1201
|
+
#
|
|
1202
|
+
# @param product_id [String] Falabella product id, e.g. '17776888' or 'prod11930024' (the
|
|
1203
|
+
# part after /product/ in a product link). Needs `country` unless it is 'cl'.
|
|
1204
|
+
# @param url [String] Instead of `product_id`: a Falabella product link, e.g.
|
|
1205
|
+
# https://www.falabella.com/falabella-cl/product/17776888/iphone-duo-256gb-night-
|
|
1206
|
+
# sky/17776887 (its store and SKU are used).
|
|
1207
|
+
# @param sku [String] Variant (SKU) whose price and availability fill the top-level fields;
|
|
1208
|
+
# default the product's main variant. Every variant is listed in `variants` either way.
|
|
1209
|
+
# @param country [String] With `product_id`: Falabella store: 'cl' (www.falabella.com,
|
|
1210
|
+
# prices in CLP), 'co' (www.falabella.com.co, prices in COP), 'pe' (www.falabella.com.pe,
|
|
1211
|
+
# prices in PEN). Default 'cl'.
|
|
1212
|
+
# Calls POST /marketplace/falabella-product.
|
|
1213
|
+
def falabella_product(product_id: nil, url: nil, sku: nil, country: nil)
|
|
1214
|
+
post_json("/marketplace/falabella-product", {
|
|
1215
|
+
product_id: product_id,
|
|
1216
|
+
url: url,
|
|
1217
|
+
sku: sku,
|
|
1218
|
+
country: country,
|
|
1219
|
+
})
|
|
1220
|
+
end
|
|
1221
|
+
|
|
1222
|
+
# Scrape Magazine Luiza (Magalu) product search.
|
|
1223
|
+
#
|
|
1224
|
+
# Searches Magazine Luiza (magazineluiza.com.br, Brazil) for `query` and returns the
|
|
1225
|
+
# products as JSON: `productId`, `offerId`, `title`, `brand`, `url`, `price`, `bestPrice`
|
|
1226
|
+
# with `bestPriceMethod` (usually the Pix price), `originalPrice` (list price),
|
|
1227
|
+
# `discountPercent`, `currency` (BRL), `installments` (count, amount, interest-free),
|
|
1228
|
+
# `rating`, `reviewCount`, `sellerId` / `sellerName`, `available`, `sponsored`,
|
|
1229
|
+
# `magaluIndica`, `category` / `subcategory`, `badges`, `image` and `variations`. Sort with
|
|
1230
|
+
# `sort`, start at `page`, and collect up to `max_results` across pages (40 per page, each
|
|
1231
|
+
# page billed as one request, see `pagesFetched`). `url` takes a Magalu search link instead
|
|
1232
|
+
# of `query`.
|
|
1233
|
+
#
|
|
1234
|
+
# @param query [String] What to search, e.g. 'geladeira', 'smart tv 55' or 'air fryer'.
|
|
1235
|
+
# @param sort [String] Order: relevance, price_low, price_high, best_selling, rating,
|
|
1236
|
+
# newest, fastest_delivery, price_with_shipping_low. API default: relevance.
|
|
1237
|
+
# @param page [Integer] 1-based results page to start from (1-25). API default: 1.
|
|
1238
|
+
# @param max_results [Integer] Products to collect across pages (1-200); Magalu serves 40
|
|
1239
|
+
# per page and each page fetched is billed as one request. API default: 40.
|
|
1240
|
+
# @param url [String] Instead of `query`: a Magazine Luiza search link, e.g.
|
|
1241
|
+
# https://www.magazineluiza.com.br/busca/geladeira/ (its sort and page are used).
|
|
1242
|
+
# Calls POST /marketplace/magazineluiza-search.
|
|
1243
|
+
def magazineluiza_search(query = nil, sort: nil, page: nil, max_results: nil, url: nil)
|
|
1244
|
+
post_json("/marketplace/magazineluiza-search", {
|
|
1245
|
+
query: query,
|
|
1246
|
+
sort: sort,
|
|
1247
|
+
page: page,
|
|
1248
|
+
max_results: max_results,
|
|
1249
|
+
url: url,
|
|
1250
|
+
})
|
|
1251
|
+
end
|
|
1252
|
+
|
|
1253
|
+
# Scrape one Magazine Luiza (Magalu) product's full detail.
|
|
1254
|
+
#
|
|
1255
|
+
# Fetches one Magazine Luiza product (by `product_id` or `url`) and returns it as JSON:
|
|
1256
|
+
# `productId`, `offerId`, `url`, `title`, `brand`, `reference`, `description`,
|
|
1257
|
+
# `keySellingPoints`, `price`, `bestPrice` / `bestPriceMethod`, `originalPrice`,
|
|
1258
|
+
# `discountPercent`, `currency` (BRL), `installments`, `available`, `rating`, `reviewCount`,
|
|
1259
|
+
# `commentCount`, `ratingHistogram`, `ratingAspects`, `reviews` (first page: rating, text,
|
|
1260
|
+
# author, date, photos), `seller` (id, name, legal name, city / state, seller since, score,
|
|
1261
|
+
# total sales, answer and delivery rate), `coupon`, `badges`, `breadcrumbs`,
|
|
1262
|
+
# `specifications` (group / name / value), `variations` (colour, size, capacity... with the
|
|
1263
|
+
# product id of each option), `images`, `videos` and package `dimensions`. Companion to
|
|
1264
|
+
# `/marketplace/magazineluiza-search`.
|
|
1265
|
+
#
|
|
1266
|
+
# @param product_id [String] Magalu product id, e.g. '240354400' or 'jc60hbaa6d' (the part
|
|
1267
|
+
# after /p/ in a product link).
|
|
1268
|
+
# @param url [String] Instead of `product_id`: a Magazine Luiza product link, e.g.
|
|
1269
|
+
# https://www.magazineluiza.com.br/notebook-acer-aspire-go/p/240354400/in/nota/.
|
|
1270
|
+
# Calls POST /marketplace/magazineluiza-product.
|
|
1271
|
+
def magazineluiza_product(product_id: nil, url: nil)
|
|
1272
|
+
post_json("/marketplace/magazineluiza-product", {
|
|
1273
|
+
product_id: product_id,
|
|
1274
|
+
url: url,
|
|
1275
|
+
})
|
|
1276
|
+
end
|
|
1277
|
+
|
|
780
1278
|
# Scrape Vinted product search.
|
|
781
1279
|
#
|
|
782
1280
|
# Searches Vinted for `query` and returns the listings as JSON: `title`, `price` and
|
|
@@ -1300,6 +1798,228 @@ module ScrapeUnblocker
|
|
|
1300
1798
|
})
|
|
1301
1799
|
end
|
|
1302
1800
|
|
|
1801
|
+
# Search Kroger and its banner stores (Ralphs, Fred Meyer, King Soopers, ...).
|
|
1802
|
+
#
|
|
1803
|
+
# Products matching `query` on Kroger or one of its banner sites (`banner`: kroger, ralphs,
|
|
1804
|
+
# fredmeyer, kingsoopers, frysfood, smithsfoodanddrug, qfc, dillons, bakersplus, citymarket,
|
|
1805
|
+
# gerbes, foodfourless, harristeeter, marianos, metromarket, picknsave, jaycfoods, payless,
|
|
1806
|
+
# foodsco), priced for one store. Returns `products` in the site's order with `productId`
|
|
1807
|
+
# (Kroger's 13-digit id), `upc`, `gtin14`, `name`, `brand`, `size`, `url`, `image`, `price`
|
|
1808
|
+
# (what the shopper pays now), `regularPrice`, `salePrice`, `onSale`, `savings`, `promotion`
|
|
1809
|
+
# (`type` sale / price_drop / multibuy / must_buy / bogo, `text`, multibuy `quantity` and
|
|
1810
|
+
# `quantityPrice`, `expiresAt`), `unitPrice` (`price`, `unit`, `text`), `soldBy` (unit /
|
|
1811
|
+
# weight; weighed items carry Kroger's per-item estimate), `available`, `stockLevel` (HIGH /
|
|
1812
|
+
# LOW / TEMPORARILY_OUT_OF_STOCK), `maxOrderQuantity`, `fulfillment` (`pickup`, `delivery`,
|
|
1813
|
+
# `inStore`, `ship`: whether the store offers the product that way), `aisle` (`number`,
|
|
1814
|
+
# `description`, `side`, `bay`, `shelf`), `rating`, `reviewCount`, `categories`,
|
|
1815
|
+
# `snapEligible` and `offers` (extra deals such as 'Save $2 each when you buy 3'), plus
|
|
1816
|
+
# `position` and `sponsored`. `totalResults` is the site's total match count; one call
|
|
1817
|
+
# returns the first page (up to 24 products). `brand` narrows the search to one or more
|
|
1818
|
+
# brands. `store` names the store the prices are for: `storeId`, `name`, `banner`,
|
|
1819
|
+
# `address`, `latitude`, `longitude`, `fulfillmentMethods`, `hours`, `pricesFor` (pickup or
|
|
1820
|
+
# inStore) and `selectedBy` (store_id / zip / default); with `zip`, `nearbyStores` lists the
|
|
1821
|
+
# banner's other stores near it. Prices are in USD and depend on the store. One billed unit
|
|
1822
|
+
# per call.
|
|
1823
|
+
#
|
|
1824
|
+
# @param query [String] What to search for, e.g. 'milk', 'coca cola', 'gluten free bread'.
|
|
1825
|
+
# @param banner [String] Which Kroger-family store site to use: 'kroger' (Kroger), 'ralphs'
|
|
1826
|
+
# (Ralphs), 'fredmeyer' (Fred Meyer), 'kingsoopers' (King Soopers), 'frysfood' (Fry's Food),
|
|
1827
|
+
# 'smithsfoodanddrug' (Smith's), 'qfc' (QFC), 'dillons' (Dillons), 'bakersplus' (Baker's),
|
|
1828
|
+
# 'citymarket' (City Market), 'gerbes' (Gerbes), 'foodfourless' (Food 4 Less),
|
|
1829
|
+
# 'harristeeter' (Harris Teeter), 'marianos' (Mariano's), 'metromarket' (Metro Market),
|
|
1830
|
+
# 'picknsave' (Pick 'n Save), 'jaycfoods' (Jay C Food Stores), 'payless' (Pay Less),
|
|
1831
|
+
# 'foodsco' (FoodsCo). Host names (ralphs.com) and common spellings (food4less, smiths,
|
|
1832
|
+
# frys) work too. API default: kroger.
|
|
1833
|
+
# @param brand [String] Only products of these brands, e.g. 'Kroger' or 'Kroger|Simple
|
|
1834
|
+
# Truth' (separate several with | or ,).
|
|
1835
|
+
# @param store_id [String] Kroger store id (8 digits, e.g. '01400929'): prices, stock and
|
|
1836
|
+
# aisles are that store's. The id is shown as `store.storeId` in every answer and in
|
|
1837
|
+
# `nearbyStores` when you search by `zip`.
|
|
1838
|
+
# @param zip [String] US ZIP code: prices for the banner's nearest store to it that takes
|
|
1839
|
+
# online orders (the nearest store at all when none does). The answer lists the nearby
|
|
1840
|
+
# stores in `nearbyStores`. Leave both `store_id` and `zip` blank for the banner's default
|
|
1841
|
+
# store.
|
|
1842
|
+
# @param max_results [Integer] Products to return, 1-24 (the first page of results). API
|
|
1843
|
+
# default: 24.
|
|
1844
|
+
# Calls POST /groceries/kroger-search.
|
|
1845
|
+
def kroger_search(query, banner: nil, brand: nil, store_id: nil, zip: nil, max_results: nil)
|
|
1846
|
+
post_json("/groceries/kroger-search", {
|
|
1847
|
+
query: query,
|
|
1848
|
+
banner: banner,
|
|
1849
|
+
brand: brand,
|
|
1850
|
+
store_id: store_id,
|
|
1851
|
+
zip: zip,
|
|
1852
|
+
max_results: max_results,
|
|
1853
|
+
})
|
|
1854
|
+
end
|
|
1855
|
+
|
|
1856
|
+
# Kroger product details (Kroger, Ralphs, Fred Meyer, King Soopers, ...).
|
|
1857
|
+
#
|
|
1858
|
+
# One product from Kroger or one of its banner sites, by Kroger product id, UPC / GTIN
|
|
1859
|
+
# barcode or product link, priced for one store. `product` has `productId` (Kroger's
|
|
1860
|
+
# 13-digit id), `upc`, `gtin14`, `name`, `brand`, `size`, `url`, `image`, `price` (what the
|
|
1861
|
+
# shopper pays now), `regularPrice`, `salePrice`, `onSale`, `savings`, `promotion` (`type`
|
|
1862
|
+
# sale / price_drop / multibuy / must_buy / bogo, `text`, multibuy `quantity` and
|
|
1863
|
+
# `quantityPrice`, `expiresAt`), `unitPrice` (`price`, `unit`, `text`), `soldBy` (unit /
|
|
1864
|
+
# weight; weighed items carry Kroger's per-item estimate), `available`, `stockLevel` (HIGH /
|
|
1865
|
+
# LOW / TEMPORARILY_OUT_OF_STOCK), `maxOrderQuantity`, `fulfillment` (`pickup`, `delivery`,
|
|
1866
|
+
# `inStore`, `ship`: whether the store offers the product that way), `aisle` (`number`,
|
|
1867
|
+
# `description`, `side`, `bay`, `shelf`), `rating`, `reviewCount`, `categories`,
|
|
1868
|
+
# `snapEligible` and `offers` (extra deals such as 'Save $2 each when you buy 3'), and the
|
|
1869
|
+
# product page details: `description`, `highlights`, `images` (every angle, large),
|
|
1870
|
+
# `ingredients`, `allergens`, `nutrition` (`servingSize`, `servingsPerContainer`,
|
|
1871
|
+
# `nutrients` with amount, unit and % daily value), `dietary`, `claims`, `organic`,
|
|
1872
|
+
# `glutenFree`, `kosher`, `nonGmo`, `taxonomy` (category paths), `countryOfOrigin`,
|
|
1873
|
+
# `storage`, `dimensions`, `weight`, `ratingBreakdown`, `alcohol` and `ageRestricted`.
|
|
1874
|
+
# `store` names the store the prices are for: `storeId`, `name`, `banner`, `address`,
|
|
1875
|
+
# `latitude`, `longitude`, `fulfillmentMethods`, `hours`, `pricesFor` (pickup or inStore)
|
|
1876
|
+
# and `selectedBy` (store_id / zip / default). One billed unit per call.
|
|
1877
|
+
#
|
|
1878
|
+
# @param product [String] The product: Kroger's 13-digit product id ('0001111041700'), a UPC
|
|
1879
|
+
# or GTIN barcode ('011110417005'), or a product link from any banner
|
|
1880
|
+
# (https://www.kroger.com/p/kroger-2-reduced-fat-milk/0001111041700). A link picks its
|
|
1881
|
+
# banner by itself.
|
|
1882
|
+
# @param banner [String] Which Kroger-family store site to use: 'kroger' (Kroger), 'ralphs'
|
|
1883
|
+
# (Ralphs), 'fredmeyer' (Fred Meyer), 'kingsoopers' (King Soopers), 'frysfood' (Fry's Food),
|
|
1884
|
+
# 'smithsfoodanddrug' (Smith's), 'qfc' (QFC), 'dillons' (Dillons), 'bakersplus' (Baker's),
|
|
1885
|
+
# 'citymarket' (City Market), 'gerbes' (Gerbes), 'foodfourless' (Food 4 Less),
|
|
1886
|
+
# 'harristeeter' (Harris Teeter), 'marianos' (Mariano's), 'metromarket' (Metro Market),
|
|
1887
|
+
# 'picknsave' (Pick 'n Save), 'jaycfoods' (Jay C Food Stores), 'payless' (Pay Less),
|
|
1888
|
+
# 'foodsco' (FoodsCo). Host names (ralphs.com) and common spellings (food4less, smiths,
|
|
1889
|
+
# frys) work too. Default 'kroger'.
|
|
1890
|
+
# @param store_id [String] Kroger store id (8 digits, e.g. '01400929'): prices, stock and
|
|
1891
|
+
# aisles are that store's. The id is shown as `store.storeId` in every answer and in
|
|
1892
|
+
# `nearbyStores` when you search by `zip`.
|
|
1893
|
+
# @param zip [String] US ZIP code: prices for the banner's nearest store to it that takes
|
|
1894
|
+
# online orders (the nearest store at all when none does). The answer lists the nearby
|
|
1895
|
+
# stores in `nearbyStores`. Leave both `store_id` and `zip` blank for the banner's default
|
|
1896
|
+
# store.
|
|
1897
|
+
# Calls POST /groceries/kroger-product.
|
|
1898
|
+
def kroger_product(product, banner: nil, store_id: nil, zip: nil)
|
|
1899
|
+
post_json("/groceries/kroger-product", {
|
|
1900
|
+
product: product,
|
|
1901
|
+
banner: banner,
|
|
1902
|
+
store_id: store_id,
|
|
1903
|
+
zip: zip,
|
|
1904
|
+
})
|
|
1905
|
+
end
|
|
1906
|
+
|
|
1907
|
+
# Scrape Instacart product search.
|
|
1908
|
+
#
|
|
1909
|
+
# Searches one retailer's Instacart catalogue (Costco, Aldi, Publix, Safeway, Kroger, CVS,
|
|
1910
|
+
# Sprouts, ... named by `retailer`) for `query` and returns `products` in Instacart's order,
|
|
1911
|
+
# priced at the store Instacart assigns to `zip`: `id` (item id), `productId`, `name`,
|
|
1912
|
+
# `brand`, `size`, `packageSize`, `price`, `priceText`, `wasPrice`, `onSale`, `unitPrice`
|
|
1913
|
+
# (`price`, `per`, `text`), `soldBy` (each / weight), `estimatedWeight`, `promotions`
|
|
1914
|
+
# (`text`, `type`), `available`, `stockLevel`, `rating`, `reviewCount`, `category`,
|
|
1915
|
+
# `storeBrand`, `dietary`, `aisle`, `image` and `url`, plus `store` (the shop, its location
|
|
1916
|
+
# id and the ZIP code used) and `totalResults`. Prices are Instacart's prices in USD, which
|
|
1917
|
+
# can differ from the retailer's in-store prices. Up to `max_results` products, billed one
|
|
1918
|
+
# unit per 30.
|
|
1919
|
+
#
|
|
1920
|
+
# @param query [String] What to search for, e.g. 'milk', 'greek yogurt', 'paper towels'.
|
|
1921
|
+
# @param retailer [String] The retailer's Instacart slug: the part after /store/ in an
|
|
1922
|
+
# Instacart link, e.g. 'costco', 'aldi', 'publix', 'safeway', 'kroger', 'cvs', 'sprouts',
|
|
1923
|
+
# 'wegmans', 'sams-club'. An Instacart store link is accepted too.
|
|
1924
|
+
# @param zip [String] 5-digit US ZIP code of the delivery area. Instacart prices and stock
|
|
1925
|
+
# belong to the store serving it. Default 94105 (San Francisco).
|
|
1926
|
+
# @param max_results [Integer] Products to return, 1-300, in Instacart's order. Billed one
|
|
1927
|
+
# unit per 30. API default: 30.
|
|
1928
|
+
# Calls POST /groceries/instacart-search.
|
|
1929
|
+
def instacart_search(query, retailer: nil, zip: nil, max_results: nil)
|
|
1930
|
+
post_json("/groceries/instacart-search", {
|
|
1931
|
+
query: query,
|
|
1932
|
+
retailer: retailer,
|
|
1933
|
+
zip: zip,
|
|
1934
|
+
max_results: max_results,
|
|
1935
|
+
})
|
|
1936
|
+
end
|
|
1937
|
+
|
|
1938
|
+
# Scrape an Instacart product.
|
|
1939
|
+
#
|
|
1940
|
+
# One product of a retailer on Instacart, priced at the store Instacart assigns to `zip`:
|
|
1941
|
+
# the search fields (`id`, `productId`, `name`, `brand`, `size`, `packageSize`, `price`,
|
|
1942
|
+
# `wasPrice`, `onSale`, `unitPrice`, `soldBy`, `promotions`, `available`, `stockLevel`,
|
|
1943
|
+
# `rating`, `reviewCount`, `category`, `dietary`, `aisle`, `image`, `url`) plus
|
|
1944
|
+
# `description`, `ingredients`, `directions`, `warnings`, the product page `sections` and
|
|
1945
|
+
# the `images` gallery, and `store`. `product` is an Instacart product id, an item id from a
|
|
1946
|
+
# search result or a product link. One unit per call.
|
|
1947
|
+
#
|
|
1948
|
+
# @param product [String] An Instacart product id ('16902650'), an item id from a search
|
|
1949
|
+
# result ('items_4142-16902650') or a product link
|
|
1950
|
+
# (https://www.instacart.com/products/16902650-...).
|
|
1951
|
+
# @param retailer [String] The retailer's Instacart slug: the part after /store/ in an
|
|
1952
|
+
# Instacart link, e.g. 'costco', 'aldi', 'publix', 'safeway', 'kroger', 'cvs', 'sprouts',
|
|
1953
|
+
# 'wegmans', 'sams-club'. An Instacart store link is accepted too. Optional when `product`
|
|
1954
|
+
# is a link carrying `retailerSlug`.
|
|
1955
|
+
# @param zip [String] 5-digit US ZIP code of the delivery area. Instacart prices and stock
|
|
1956
|
+
# belong to the store serving it. Default 94105 (San Francisco).
|
|
1957
|
+
# Calls POST /groceries/instacart-product.
|
|
1958
|
+
def instacart_product(product, retailer: nil, zip: nil)
|
|
1959
|
+
post_json("/groceries/instacart-product", {
|
|
1960
|
+
product: product,
|
|
1961
|
+
retailer: retailer,
|
|
1962
|
+
zip: zip,
|
|
1963
|
+
})
|
|
1964
|
+
end
|
|
1965
|
+
|
|
1966
|
+
# Scrape Publix grocery search.
|
|
1967
|
+
#
|
|
1968
|
+
# Searches publix.com (US) for `query` at one store and returns `products`: `productId`,
|
|
1969
|
+
# `itemCode`, `title`, `brand`, `size`, `url`, `image`, `price`, `priceText`, `unit` (e.g.
|
|
1970
|
+
# 'lb' for items sold by weight), `wasPrice`, `currency`, `onSale`, `promotion` (`type` such
|
|
1971
|
+
# as BOGO, `message`, `conditions`, `validThrough`, `savings`), `hasCoupon`, `soldByWeight`,
|
|
1972
|
+
# `aisle`, `category`, `categoryPath` and `storeId`, plus `store` (the store the prices are
|
|
1973
|
+
# for), `totalResults`, `totalPages` and `nextPage`. Publix publishes a price online only
|
|
1974
|
+
# while an item is on promotion, so `price` is null for the rest. A keyword Publix sends to
|
|
1975
|
+
# a category returns that category, named in `redirectedTo`; one it sends to a page that
|
|
1976
|
+
# lists no products (the weekly ad, cake ordering, store locations) answers with no products
|
|
1977
|
+
# and `sentElsewhere` true.
|
|
1978
|
+
#
|
|
1979
|
+
# @param query [String] What to search for, e.g. 'milk', 'cheerios' or 'chicken breast'.
|
|
1980
|
+
# @param store_id [String] Publix store number whose prices and promotions to return, e.g.
|
|
1981
|
+
# '1614'. Give this or `zip`; with neither, a store in Lakeland, FL is used. The store used
|
|
1982
|
+
# is named in `store`.
|
|
1983
|
+
# @param zip [String] US ZIP code; the nearest Publix store is used, e.g. '33801'.
|
|
1984
|
+
# @param page [Integer] 1-based results page to start from (48 products per page). API
|
|
1985
|
+
# default: 1.
|
|
1986
|
+
# @param max_results [Integer] Products to collect from `page` on, 1-480. Each page fetched
|
|
1987
|
+
# is billed one unit. API default: 48.
|
|
1988
|
+
# Calls POST /groceries/publix-search.
|
|
1989
|
+
def publix_search(query, store_id: nil, zip: nil, page: nil, max_results: nil)
|
|
1990
|
+
post_json("/groceries/publix-search", {
|
|
1991
|
+
query: query,
|
|
1992
|
+
store_id: store_id,
|
|
1993
|
+
zip: zip,
|
|
1994
|
+
page: page,
|
|
1995
|
+
max_results: max_results,
|
|
1996
|
+
})
|
|
1997
|
+
end
|
|
1998
|
+
|
|
1999
|
+
# Scrape a Publix product.
|
|
2000
|
+
#
|
|
2001
|
+
# Returns one publix.com (US) product at one store as `product`: the search fields
|
|
2002
|
+
# (`productId`, `itemCode`, `title`, `brand`, `size`, `price`, `wasPrice`, `promotion`,
|
|
2003
|
+
# `aisle` and so on) plus `images`, `description`, `ingredients`, `directions`, `allergens`,
|
|
2004
|
+
# `available`, `categories`, `badges`, `couponCount`, `smartLabelUrl` and `relatedProducts`,
|
|
2005
|
+
# and `store`, the store the price is for. Publix publishes a price online only while an
|
|
2006
|
+
# item is on promotion.
|
|
2007
|
+
#
|
|
2008
|
+
# @param product [String] A Publix product id ('RIO-PCI-103438') or product link
|
|
2009
|
+
# (https://www.publix.com/pd/cheerios-cereal-large-size/RIO-PCI-103438).
|
|
2010
|
+
# @param store_id [String] Publix store number whose prices and promotions to return, e.g.
|
|
2011
|
+
# '1614'. Give this or `zip`; with neither, a store in Lakeland, FL is used. The store used
|
|
2012
|
+
# is named in `store`.
|
|
2013
|
+
# @param zip [String] US ZIP code; the nearest Publix store is used, e.g. '33801'.
|
|
2014
|
+
# Calls POST /groceries/publix-product.
|
|
2015
|
+
def publix_product(product, store_id: nil, zip: nil)
|
|
2016
|
+
post_json("/groceries/publix-product", {
|
|
2017
|
+
product: product,
|
|
2018
|
+
store_id: store_id,
|
|
2019
|
+
zip: zip,
|
|
2020
|
+
})
|
|
2021
|
+
end
|
|
2022
|
+
|
|
1303
2023
|
# Smyths Toys UK product stock, price, details and listings.
|
|
1304
2024
|
#
|
|
1305
2025
|
# Smyths Toys UK (smythstoys.com) as JSON, in four modes. `mode=product` (the default when
|
|
@@ -2109,6 +2829,94 @@ module ScrapeUnblocker
|
|
|
2109
2829
|
})
|
|
2110
2830
|
end
|
|
2111
2831
|
|
|
2832
|
+
# Scrape a Google AI Mode answer.
|
|
2833
|
+
#
|
|
2834
|
+
# Asks Google AI Mode (the `udm=50` AI Mode tab, not the AI Overview on a results page) the
|
|
2835
|
+
# question `q` and returns its full answer as JSON: `answer.text` (plain text),
|
|
2836
|
+
# `answer.markdown` and `answer.blocks` (paragraphs, headings, lists, tables and code, in
|
|
2837
|
+
# order), `references` (the pages the answer cites: `title`, `url`, `domain`, `source`,
|
|
2838
|
+
# `snippet`), `links` (links written into the answer text) and `followUps` (the follow-up
|
|
2839
|
+
# questions Google closes the answer with, if any). Set `proxy_country` to ask from a market
|
|
2840
|
+
# - the Google market (`gl`) follows it - and `hl` for the answer's language. Answers differ
|
|
2841
|
+
# between identical requests, as they do for a person.
|
|
2842
|
+
#
|
|
2843
|
+
# @param q [String] The question to ask Google AI Mode, e.g. 'how to learn python fast'.
|
|
2844
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US',
|
|
2845
|
+
# 'DE'). The Google market (`gl`) follows it unless `gl` is set.
|
|
2846
|
+
# @param gl [String] Optional Google market override (ISO-2, e.g. 'us', 'de'). Default:
|
|
2847
|
+
# `proxy_country`, else 'us'.
|
|
2848
|
+
# @param hl [String] Language Google answers in, e.g. 'en', 'de', 'pt-BR'. Default 'en'.
|
|
2849
|
+
# Calls POST /search/google-ai-mode.
|
|
2850
|
+
def google_ai_mode(q, proxy_country: nil, gl: nil, hl: nil)
|
|
2851
|
+
post_json("/search/google-ai-mode", {
|
|
2852
|
+
q: q,
|
|
2853
|
+
proxy_country: proxy_country,
|
|
2854
|
+
gl: gl,
|
|
2855
|
+
hl: hl,
|
|
2856
|
+
})
|
|
2857
|
+
end
|
|
2858
|
+
|
|
2859
|
+
# Search Google Lens with an image URL.
|
|
2860
|
+
#
|
|
2861
|
+
# Runs a Google Lens search for the image at `url` and returns its visual matches as JSON
|
|
2862
|
+
# (usually ~60). Every match carries `title`, `url` (the page the image is on), `domain`,
|
|
2863
|
+
# `source` (site name), `imageUrl` (the matching image), `thumbnailUrl`, `width`, `height`
|
|
2864
|
+
# and `position`; product matches add `price` and `currency` (and `product` details) when
|
|
2865
|
+
# Google has them, and `priceText`, `rating` and `reviews` as the tile shows them.
|
|
2866
|
+
# `relatedSearches` lists what Lens recognised in the image (e.g. 'Eiffel Tower', with its
|
|
2867
|
+
# Knowledge Graph id) and `aiOverview` is Google's own description of the image when it is
|
|
2868
|
+
# ready in time (else null). Shopping matches follow the country the request comes from -
|
|
2869
|
+
# set `proxy_country` for a market (a US request is served from a European exit, where
|
|
2870
|
+
# Google loads Lens matches; `proxyCountry` in the response says which).
|
|
2871
|
+
#
|
|
2872
|
+
# @param url [String] Public http(s) URL of the image to search with, e.g. a product photo.
|
|
2873
|
+
# Google fetches the image itself.
|
|
2874
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US',
|
|
2875
|
+
# 'DE'). Shopping matches and prices follow the country Google sees the request from.
|
|
2876
|
+
# @param gl [String] Optional Google market override (ISO-2). Default: `proxy_country`, else
|
|
2877
|
+
# 'us'.
|
|
2878
|
+
# @param hl [String] Google UI language, e.g. 'en', 'de'. Default 'en'.
|
|
2879
|
+
# @param max_results [Integer] Optional cap on the visual matches returned (1-100; default:
|
|
2880
|
+
# all, usually ~60).
|
|
2881
|
+
# Calls POST /images/google-lens.
|
|
2882
|
+
def google_lens(url, proxy_country: nil, gl: nil, hl: nil, max_results: nil)
|
|
2883
|
+
post_json("/images/google-lens", {
|
|
2884
|
+
url: url,
|
|
2885
|
+
proxy_country: proxy_country,
|
|
2886
|
+
gl: gl,
|
|
2887
|
+
hl: hl,
|
|
2888
|
+
max_results: max_results,
|
|
2889
|
+
})
|
|
2890
|
+
end
|
|
2891
|
+
|
|
2892
|
+
# Ask Google Gemini and get its answer and cited sources.
|
|
2893
|
+
#
|
|
2894
|
+
# Asks Google Gemini (gemini.google.com, signed out) one question and returns `answer`
|
|
2895
|
+
# (plain text) and `answerMarkdown`, `citations` (pages cited inline: `position`, `title`,
|
|
2896
|
+
# `url`, `domain`, `siteName`, `description`, `citedText` - the answer sentences that page
|
|
2897
|
+
# backs - and `firstCitedAt`, the character offset of the first of them), `sources` (pages
|
|
2898
|
+
# Gemini consulted when it searched the web: `position`, `title`, `url`, `domain`),
|
|
2899
|
+
# `webSearch` (whether it searched the web), `followUpPrompts` (`label`, `prompt`),
|
|
2900
|
+
# `richBlocks` (cards Gemini drew that have no text, e.g. a sports score or image strip,
|
|
2901
|
+
# left out of the answer), `model` (the model label the app shows), `location` and
|
|
2902
|
+
# `locationCountry` (where Gemini placed the visitor), `complete` and `conversationId`.
|
|
2903
|
+
# `country` asks from that country. Answers vary between identical calls, as they do for
|
|
2904
|
+
# people. One unit per call.
|
|
2905
|
+
#
|
|
2906
|
+
# @param prompt [String] The question to ask Gemini, as a user would type it, e.g. 'best CRM
|
|
2907
|
+
# for small business' (max 4000 characters). Each call starts a new conversation.
|
|
2908
|
+
# @param country [String] Two-letter country code to ask from, e.g. US, GB, DE. Gemini
|
|
2909
|
+
# answers for where it locates the visitor (local results, currency, language defaults);
|
|
2910
|
+
# `locationCountry` in the answer says where it placed this call. Leave blank for any
|
|
2911
|
+
# European location.
|
|
2912
|
+
# Calls POST /ai/gemini.
|
|
2913
|
+
def gemini(prompt, country: nil)
|
|
2914
|
+
post_json("/ai/gemini", {
|
|
2915
|
+
prompt: prompt,
|
|
2916
|
+
country: country,
|
|
2917
|
+
})
|
|
2918
|
+
end
|
|
2919
|
+
|
|
2112
2920
|
# Search Bing.
|
|
2113
2921
|
#
|
|
2114
2922
|
# Searches Bing for `keyword` in a real browser and returns the results page as JSON:
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: scrapeunblocker
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.15.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- ScrapeUnblocker
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-10-
|
|
11
|
+
date: 2026-10-29 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
|
|
14
14
|
plus Google SERP and 100+ ready-made site plugins (Amazon, Walmart, Zillow, Google
|