scrapeunblocker 0.17.0 → 0.17.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +6 -0
- data/README.md +1 -1
- data/lib/scrapeunblocker/plugins.rb +59 -1
- data/lib/scrapeunblocker/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4a7407a3294512988eb4cc9c842879b2a239bbf0ec11a9f237330419f474566c
|
|
4
|
+
data.tar.gz: 1de06b6e0edea74520225e6e036f5e669efa28458bab9f4ab55cbaa8568e3858
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 47df1c68bc32800c2b86a6466de1d5b9c7090ec87f2c7594cd3f6c95480c26ddd81ace11cea0b45750bab2ecfdf971157157663eb2de6c4fc6bf3afeb385af75
|
|
7
|
+
data.tar.gz: 1890257bc228863e02cf9830889ab0d38da7068357d3a0df580801b143f699c3f8cba39c7d894d95c2853b6d2f622e10bd194f6308363204d8ff0e02b0968aed
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,11 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.17.1 (2026-10-30)
|
|
4
|
+
|
|
5
|
+
- 2 new plugin methods on `Client`: `bodegaaurrera_search(query)` and `bodegaaurrera_product` (Bodega Aurrera, Walmart de Mexico's discount chain; prices in MXN). The search's first argument is optional for calls that pass `url:` instead; the product method takes `product_id:` or `url:`.
|
|
6
|
+
|
|
7
|
+
No breaking changes.
|
|
8
|
+
|
|
3
9
|
## 0.17.0 (2026-10-30)
|
|
4
10
|
|
|
5
11
|
- 29 new plugin methods on `Client`:
|
data/README.md
CHANGED
|
@@ -154,7 +154,7 @@ jobs = su.career_site_jobs("https://jobs.ashbyhq.com/ramp", keyword: "engineer")
|
|
|
154
154
|
| Search engines | `bing_search`, `bing_images`, `duckduckgo_search`, `yahoo_search`, `yahoo_images`, `brave_search`, `brave_images` |
|
|
155
155
|
| E-commerce | `amazon_product`, `amazon_search`, `amazon_bestsellers`, `amazon_reviews`, `amazon_offers`, `amazon_seller`, `walmart_search`, `walmart_product`, `target_search`, `target_category`, `target_product`, `bestbuy_search`, `bestbuy_product`, `homedepot_search`, `costco_search`, `costco_product`, `etsy_search`, `etsy_product`, `shein_search`, `zalando_search`, `aliexpress_search`, `aliexpress_product`, `alibaba_search`, `alibaba_product`, `alibaba1688_search`, `alibaba1688_product`, `tiktok_shop_search`, `tiktok_shop_product`, `tiktok_shop_store`, `oopbuy_search`, `smyths_inventory`, `smyths_details`, `smyths_search`, `smyths_category`, `wish_search`, `wish_product`, `shopee_search`, `shopee_product`, `taobao_search`, `taobao_product`, `flipkart_search`, `flipkart_product`, `lazada_search`, `lazada_product`, `allegro_search`, `allegro_product`, `falabella_search`, `falabella_product`, `magazineluiza_search`, `magazineluiza_product`, `mediamarkt_search`, `mediamarkt_product`, `cdiscount_search`, `cdiscount_product`, `idealo_search`, `idealo_offers`, `rakuten_search`, `rakuten_product`, `indiamart_search`, `indiamart_product`, `bedbathandbeyond_search`, `bedbathandbeyond_product`, `petco_search`, `petco_product`, `neobits_product`, `lowes_search`, `lowes_product`, `menards_search`, `menards_product`, `grainger_search`, `grainger_product` |
|
|
156
156
|
| Classifieds & Resale | `ebay_product`, `ebay_search`, `vinted_search`, `leboncoin_search` |
|
|
157
|
-
| Groceries | `woolworths_search`, `woolworths_category`, `woolworths_specials`, `woolworths_product`, `coles_search`, `coles_category`, `coles_specials`, `coles_product`, `aldi_au_search`, `aldi_au_category`, `aldi_au_specials`, `aldi_au_product`, `kroger_search`, `kroger_product`, `instacart_search`, `instacart_product`, `publix_search`, `publix_product` |
|
|
157
|
+
| Groceries | `woolworths_search`, `woolworths_category`, `woolworths_specials`, `woolworths_product`, `coles_search`, `coles_category`, `coles_specials`, `coles_product`, `aldi_au_search`, `aldi_au_category`, `aldi_au_specials`, `aldi_au_product`, `kroger_search`, `kroger_product`, `instacart_search`, `instacart_product`, `publix_search`, `publix_product`, `bodegaaurrera_search`, `bodegaaurrera_product` |
|
|
158
158
|
| Real estate | `zillow_search`, `zillow_detail`, `redfin_search`, `redfin_detail`, `rightmove_search`, `idealista_search`, `idealista_detail`, `immobilienscout24_search` |
|
|
159
159
|
| Vehicles | `mobile_de`, `autoscout24_search` |
|
|
160
160
|
| Jobs | `indeed_jobs`, `glassdoor_jobs`, `glassdoor_reviews`, `glassdoor_salaries`, `naukri_jobs`, `wellfound_jobs`, `wellfound_job`, `wellfound_company`, `wellfound_company_search`, `career_site_jobs`, `upwork_jobs` |
|
|
@@ -2948,6 +2948,64 @@ module ScrapeUnblocker
|
|
|
2948
2948
|
})
|
|
2949
2949
|
end
|
|
2950
2950
|
|
|
2951
|
+
# Scrape Bodega Aurrera (Mexico) product search.
|
|
2952
|
+
#
|
|
2953
|
+
# Searches Bodega Aurrera (bodegaaurrera.com.mx, Walmart de Mexico) for `query` and returns
|
|
2954
|
+
# `products`, each with `position`, `productId`, `name`, `brand`, `url`, `price` (MXN),
|
|
2955
|
+
# `wasPrice` and `savings` (only on a promotion), `unitPrice`, `priceRange`, `rating`,
|
|
2956
|
+
# `reviewCount`, `availability` / `availabilityText` / `inStock`, `seller` / `sellerType`,
|
|
2957
|
+
# `salesUnit`, `department`, `category` (path), `badges`, `fulfillment` (pickup / delivery
|
|
2958
|
+
# labels), `sponsored` and `image`. Also `totalResults`, `storeId` (the store the prices and
|
|
2959
|
+
# availability are for: the one the storefront assigns a new visitor - a store or postal
|
|
2960
|
+
# code cannot be chosen) and `nextPage`. Sort with `sort` (relevance, price_low,
|
|
2961
|
+
# price_high), start at `page` and collect up to `max_results` across pages (about 40 per
|
|
2962
|
+
# page, each page billed as one request, see `pagesFetched`). `url` takes a Bodega Aurrera
|
|
2963
|
+
# search link instead of `query`. A results page usually takes 1-3 s; a page that has to be
|
|
2964
|
+
# rendered in a browser takes 20-60 s.
|
|
2965
|
+
#
|
|
2966
|
+
# @param query [String] What to search, e.g. 'leche', 'arroz', 'detergente' or 'pañales'.
|
|
2967
|
+
# @param sort [String] Order: relevance, price_low, price_high. API default: relevance.
|
|
2968
|
+
# @param page [Integer] 1-based results page to start from (1-25). API default: 1.
|
|
2969
|
+
# @param max_results [Integer] Products to collect across pages (1-220); about 44 per page,
|
|
2970
|
+
# and each page read is billed as one request. API default: 44.
|
|
2971
|
+
# @param url [String] Instead of `query`: a Bodega Aurrera search link, e.g.
|
|
2972
|
+
# https://www.bodegaaurrera.com.mx/search?q=leche (its sort and page are used).
|
|
2973
|
+
# Calls POST /groceries/bodegaaurrera-search.
|
|
2974
|
+
def bodegaaurrera_search(query = nil, sort: nil, page: nil, max_results: nil, url: nil)
|
|
2975
|
+
post_json("/groceries/bodegaaurrera-search", {
|
|
2976
|
+
query: query,
|
|
2977
|
+
sort: sort,
|
|
2978
|
+
page: page,
|
|
2979
|
+
max_results: max_results,
|
|
2980
|
+
url: url,
|
|
2981
|
+
})
|
|
2982
|
+
end
|
|
2983
|
+
|
|
2984
|
+
# Scrape one Bodega Aurrera (Mexico) product.
|
|
2985
|
+
#
|
|
2986
|
+
# Fetches one Bodega Aurrera product (by `product_id` or `url`) and returns it as JSON:
|
|
2987
|
+
# `productId`, `name`, `brand`, `url`, `upc`, `model`, `price` (MXN), `wasPrice` / `savings`
|
|
2988
|
+
# (only on a promotion), `unitPrice`, `installments`, `availability` / `availabilityText` /
|
|
2989
|
+
# `inStock`, `orderLimit`, `orderMinimum`, `salesUnit`, `seller` / `sellerId` /
|
|
2990
|
+
# `sellerType`, `rating`, `reviewCount`, `ratingBreakdown` (1-5 stars), `category` (path),
|
|
2991
|
+
# `productType`, `shortDescription`, `description`, `ingredients`, `specifications` (name /
|
|
2992
|
+
# value), `badges`, `fulfillment` (pickup / delivery messages), `fulfillmentOptions`,
|
|
2993
|
+
# `store` (the store the price and availability are for - the one the storefront assigns a
|
|
2994
|
+
# new visitor) and `image` / `images`. A call usually takes 1-3 s, or 20-60 s when the page
|
|
2995
|
+
# has to be rendered in a browser. Companion to `/groceries/bodegaaurrera-search`.
|
|
2996
|
+
#
|
|
2997
|
+
# @param product_id [String] Bodega Aurrera item id, e.g. '00750649502153' (the number at
|
|
2998
|
+
# the end of an /ip/ link; search results give it as `productId`).
|
|
2999
|
+
# @param url [String] Instead of `product_id`: a Bodega Aurrera product link, e.g.
|
|
3000
|
+
# https://www.bodegaaurrera.com.mx/ip/leche-great-value-entera-1-l/00750649502153.
|
|
3001
|
+
# Calls POST /groceries/bodegaaurrera-product.
|
|
3002
|
+
def bodegaaurrera_product(product_id: nil, url: nil)
|
|
3003
|
+
post_json("/groceries/bodegaaurrera-product", {
|
|
3004
|
+
product_id: product_id,
|
|
3005
|
+
url: url,
|
|
3006
|
+
})
|
|
3007
|
+
end
|
|
3008
|
+
|
|
2951
3009
|
# Smyths Toys UK product stock, price, details and listings.
|
|
2952
3010
|
#
|
|
2953
3011
|
# Smyths Toys UK (smythstoys.com) as JSON, in four modes. `mode=product` (the default when
|
|
@@ -5731,7 +5789,7 @@ module ScrapeUnblocker
|
|
|
5731
5789
|
# @param video [String] Video id (dQw4w9WgXcQ) or any YouTube video URL.
|
|
5732
5790
|
# @param lang [String] Preferred caption language code, e.g. 'en'. Omit for the
|
|
5733
5791
|
# default/first track. A prefix like 'en' matches 'en-US'.
|
|
5734
|
-
# @param format [String] Output shape:
|
|
5792
|
+
# @param format [String] Output shape: text, segments. 'segments' returns timestamps; 'text'
|
|
5735
5793
|
# returns only the joined plaintext. API default: segments.
|
|
5736
5794
|
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US',
|
|
5737
5795
|
# 'DE') for the watch page. Defaults to US.
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: scrapeunblocker
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.17.
|
|
4
|
+
version: 0.17.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- ScrapeUnblocker
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-10-
|
|
11
|
+
date: 2026-10-30 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
|
|
14
14
|
plus Google SERP and 100+ ready-made site plugins (Amazon, Walmart, Zillow, Google
|