scrapeunblocker 0.15.0 → 0.16.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +8 -0
- data/README.md +1 -1
- data/lib/scrapeunblocker/plugins.rb +73 -17
- data/lib/scrapeunblocker/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 28020ffea6eb49ad4418747a5f2fa7c4a55c0aa7b98fbc73abb1ad433bb9c0e5
|
|
4
|
+
data.tar.gz: 57624c5f914ef34100ef24788abe751d6844c68a2deda593075e2d032a6e93c1
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: a8710aac0acb04703254f1358abf8ea9f1096d6392232772730c6c4d95f06c8271784db5c6cbbad4c22ac1ddd2be09db35acf0bba3ac5332b2e9ebbdf6fb8651
|
|
7
|
+
data.tar.gz: 5e720ee147514e819bc97b33c8187784c9797dfbb6304ed9577c5e916039ca84561c2376eb669a0b946976fb6463c89a34fdeb8b3889c6c1c8340cd7e1ae1bab
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.16.0 (2026-10-29)
|
|
4
|
+
|
|
5
|
+
- Add `maps_reviews`: Google Maps reviews of a place in Google's sort orders (relevant, newest, highest, lowest) with exact ISO dates, original and translated text, authors, photos, owner replies and page tokens. `POST /maps/reviews`.
|
|
6
|
+
|
|
7
|
+
## 0.15.1 (2026-10-29)
|
|
8
|
+
|
|
9
|
+
- `shopee_product` and `shopee_search` docs: describe `availability` and `inStock`, sold-out variations (`models[].available`, `tierIndex`), `categories` as the breadcrumb path plus `browseCategories`, `isPixPrice` (Brazilian Pix prices) and the JSON `{"status": "not_found", ...}` answer for a product that does not exist. Signatures are unchanged.
|
|
10
|
+
|
|
3
11
|
## 0.15.0 (2026-10-29)
|
|
4
12
|
|
|
5
13
|
- 23 new plugin methods on `Client`:
|
data/README.md
CHANGED
|
@@ -149,7 +149,7 @@ jobs = su.career_site_jobs("https://jobs.ashbyhq.com/ramp", keyword: "engineer")
|
|
|
149
149
|
| Category | Methods |
|
|
150
150
|
| --- | --- |
|
|
151
151
|
| Travel | `skyscanner.flights`, `skyscanner.hotels`, `skyscanner.carhire`, `booking_hotels`, `booking_reviews`, `airbnb_search`, `airbnb_room`, `airbnb_reviews`, `tripadvisor_list`, `tripadvisor_reviews` |
|
|
152
|
-
| Google | `google_local`, `google_images`, `maps_place_details`, `google_trends`, `google_shopping`, `google_jobs`, `google_hotels`, `google_hotels_property`, `google_news`, `google_ads_transparency`, `google_ads_transparency_advertisers`, `google_ai_mode`, `google_lens` |
|
|
152
|
+
| Google | `google_local`, `google_images`, `maps_place_details`, `maps_reviews`, `google_trends`, `google_shopping`, `google_jobs`, `google_hotels`, `google_hotels_property`, `google_news`, `google_ads_transparency`, `google_ads_transparency_advertisers`, `google_ai_mode`, `google_lens` |
|
|
153
153
|
| AI | `gemini` |
|
|
154
154
|
| Search engines | `bing_search`, `bing_images`, `duckduckgo_search`, `yahoo_search`, `yahoo_images`, `brave_search`, `brave_images` |
|
|
155
155
|
| E-commerce | `amazon_product`, `amazon_search`, `amazon_bestsellers`, `amazon_reviews`, `amazon_offers`, `amazon_seller`, `walmart_search`, `target_search`, `bestbuy_search`, `homedepot_search`, `costco_search`, `costco_product`, `etsy_search`, `shein_search`, `zalando_search`, `aliexpress_search`, `alibaba_search`, `alibaba_product`, `alibaba1688_search`, `alibaba1688_product`, `tiktok_shop_search`, `tiktok_shop_product`, `tiktok_shop_store`, `oopbuy_search`, `smyths_inventory`, `smyths_details`, `smyths_search`, `smyths_category`, `wish_search`, `wish_product`, `shopee_search`, `shopee_product`, `taobao_search`, `taobao_product`, `flipkart_search`, `flipkart_product`, `lazada_search`, `lazada_product`, `allegro_search`, `allegro_product`, `falabella_search`, `falabella_product`, `magazineluiza_search`, `magazineluiza_product` |
|
|
@@ -793,15 +793,18 @@ module ScrapeUnblocker
|
|
|
793
793
|
# Searches a Shopee market (Singapore, Indonesia, Malaysia, Thailand, Vietnam, Philippines,
|
|
794
794
|
# Taiwan or Brazil - `country`) for `q` and returns one page of results: up to 40
|
|
795
795
|
# `products`, each with `position`, `itemId`, `shopId`, `name`, `url`, `image`, `price`,
|
|
796
|
-
# `originalPrice` (when Shopee shows the crossed-out price), `discountPercent`,
|
|
797
|
-
# `rating` (average stars), `location` (where the seller ships
|
|
798
|
-
# badges). `sort` orders the results, `min_price` /
|
|
799
|
-
# one shop's products (or lists the shop without
|
|
800
|
-
# category link instead. Shopee shows one page per
|
|
801
|
-
# price ranges for more products. `price` is what
|
|
802
|
-
# can be a promotional or new-customer price (see
|
|
803
|
-
#
|
|
804
|
-
#
|
|
796
|
+
# `originalPrice` (when Shopee shows the crossed-out price), `discountPercent`,
|
|
797
|
+
# `isPixPrice`, `currency`, `rating` (average stars), `location` (where the seller ships
|
|
798
|
+
# from) and `labels` (promotion badges). `sort` orders the results, `min_price` /
|
|
799
|
+
# `max_price` filter them, `shop_id` keeps one shop's products (or lists the shop without
|
|
800
|
+
# `q`), and `url` takes a Shopee search or category link instead. Shopee shows one page per
|
|
801
|
+
# search, so run several sort orders or price ranges for more products. `price` is what
|
|
802
|
+
# Shopee shows a signed-out shopper, which can be a promotional or new-customer price (see
|
|
803
|
+
# `labels`). In Brazil a card often shows the Pix price (`isPixPrice: true`): `price` is
|
|
804
|
+
# then the Pix price and `discountPercent` is measured against the list price, which Shopee
|
|
805
|
+
# does not show on such a card, so `originalPrice` is null. Not available: per-variation
|
|
806
|
+
# prices, stock counts, units sold, rating counts and reviews (Shopee does not publish them
|
|
807
|
+
# on the pages these endpoints read). One unit per call.
|
|
805
808
|
#
|
|
806
809
|
# @param q [String] What to search for, e.g. 'iphone case' or 'air fryer'. Optional with
|
|
807
810
|
# `shop_id`.
|
|
@@ -841,14 +844,22 @@ module ScrapeUnblocker
|
|
|
841
844
|
# (average stars), `labels`, `images`, `shopLocation` and `shop` (`name`, `username`, `url`,
|
|
842
845
|
# `rating`, `ratingCount`, `followers`, `itemCount`, `responseRatePercent`,
|
|
843
846
|
# `isOfficialShop`, `isPreferredPlusSeller`, `location`, `joinedAt`). With `dataLevel: full`
|
|
844
|
-
# it also has `
|
|
845
|
-
#
|
|
846
|
-
#
|
|
847
|
-
#
|
|
848
|
-
#
|
|
849
|
-
#
|
|
850
|
-
#
|
|
851
|
-
#
|
|
847
|
+
# it also has `availability` (`in_stock`, `sold_out` or `unavailable` for a withdrawn
|
|
848
|
+
# listing) and `inStock`, `description`, `brand`, `condition`, `categories` (the path the
|
|
849
|
+
# product page's breadcrumb shows, with ids), `browseCategories` (the browse tree, whose ids
|
|
850
|
+
# Shopee's category links use), `videos`, `variations` (option names), `models` (`modelId`,
|
|
851
|
+
# `name`, `available` - false for a sold-out variation - and `tierIndex`, the model's option
|
|
852
|
+
# in each variation), `attributes`, `isOfficialShop`, `isPreOrder`, `estimatedShippingDays`
|
|
853
|
+
# and `createdAt`; `dataLevel: meta` means Shopee showed only the product's title and main
|
|
854
|
+
# image (some markets), with price, rating and shop still included. A product Shopee says
|
|
855
|
+
# does not exist (removed, or never existed) answers 404 with JSON `{"status": "not_found",
|
|
856
|
+
# ...}`; a 502 means the call could not get an answer from Shopee and can be retried.
|
|
857
|
+
# `price` is what Shopee shows a signed-out shopper, which can be a promotional or new-
|
|
858
|
+
# customer price (see `labels`). In Brazil a card often shows the Pix price (`isPixPrice:
|
|
859
|
+
# true`): `price` is then the Pix price and `discountPercent` is measured against the list
|
|
860
|
+
# price, which Shopee does not show on such a card, so `originalPrice` is null. Not
|
|
861
|
+
# available: per-variation prices, stock counts, units sold, rating counts and reviews
|
|
862
|
+
# (Shopee does not publish them on the pages these endpoints read). One unit per call.
|
|
852
863
|
#
|
|
853
864
|
# @param url [String] A Shopee product link, e.g. https://shopee.sg/Beats-
|
|
854
865
|
# iPhone-17-Case-i.442800909.40270108836 or
|
|
@@ -2627,6 +2638,51 @@ module ScrapeUnblocker
|
|
|
2627
2638
|
})
|
|
2628
2639
|
end
|
|
2629
2640
|
|
|
2641
|
+
# Scrape Google Maps reviews.
|
|
2642
|
+
#
|
|
2643
|
+
# Returns one Google Maps place's reviews as JSON, sorted the way Google sorts them:
|
|
2644
|
+
# `relevant`, `newest`, `highest` or `lowest`. Each review has `rating` (1-5), `date` (exact
|
|
2645
|
+
# ISO 8601, UTC) + `timestamp`, `relativeDate`, `text` (original) + `language`,
|
|
2646
|
+
# `translatedText` (into `hl`, when the review is in another language), `author`, `photos`,
|
|
2647
|
+
# `details` (guided answers such as 'Rooms: 5'), `ownerResponse` and `reviewUrl`. The answer
|
|
2648
|
+
# also carries a `place` summary and `nextPageToken` for paging. Identify the place with
|
|
2649
|
+
# `feature_id` (0xHEX:0xHEX), `place_id` (ChIJ...) or a Maps `url`. Reviews come in pages of
|
|
2650
|
+
# 20; each page fetched is one billed unit.
|
|
2651
|
+
#
|
|
2652
|
+
# @param url [String] A Google Maps place URL (share link or /maps/place/... URL). The
|
|
2653
|
+
# feature id or place_id is read out of it.
|
|
2654
|
+
# @param place_id [String] A Google place_id, e.g. 'ChIJN1t_tDeuEmsRUsoyG83frY4'.
|
|
2655
|
+
# @param feature_id [String] A Maps feature id '0xHEX:0xHEX', e.g.
|
|
2656
|
+
# '0x4776901a5d4813b3:0x2a497c8b414893c'. Fastest - no resolve step.
|
|
2657
|
+
# @param sort [String] Review order: 'relevant' (Google's default), 'newest', 'highest' or
|
|
2658
|
+
# 'lowest'. SerpApi's 'qualityScore', 'newestFirst', 'ratingHigh' and 'ratingLow' are
|
|
2659
|
+
# accepted too. API default: relevant.
|
|
2660
|
+
# @param max_reviews [Integer] How many reviews to return, 1-200. Reviews come in pages of
|
|
2661
|
+
# 20; each page fetched is one billed unit. API default: 20.
|
|
2662
|
+
# @param next_page_token [String] Continue from a previous answer's `nextPageToken` (same
|
|
2663
|
+
# place and sort). When `max_reviews` is not a multiple of 20 the next page can repeat a few
|
|
2664
|
+
# reviews - dedupe on `reviewId`.
|
|
2665
|
+
# @param hl [String] Language for Google's labels and the review translation
|
|
2666
|
+
# (`translatedText`), e.g. 'en', 'de'. The original text is always returned in `text`. API
|
|
2667
|
+
# default: en.
|
|
2668
|
+
# @param gl [String] Google country, ISO-2 lowercase, e.g. 'us'. API default: us.
|
|
2669
|
+
# @param proxy_country [String] Proxy country the request is sent from (ISO-2, e.g. 'US',
|
|
2670
|
+
# 'DE'). Reviews are the same from any country. API default: us.
|
|
2671
|
+
# Calls POST /maps/reviews.
|
|
2672
|
+
def maps_reviews(url: nil, place_id: nil, feature_id: nil, sort: nil, max_reviews: nil, next_page_token: nil, hl: nil, gl: nil, proxy_country: nil)
|
|
2673
|
+
post_json("/maps/reviews", {
|
|
2674
|
+
url: url,
|
|
2675
|
+
place_id: place_id,
|
|
2676
|
+
feature_id: feature_id,
|
|
2677
|
+
sort: sort,
|
|
2678
|
+
max_reviews: max_reviews,
|
|
2679
|
+
next_page_token: next_page_token,
|
|
2680
|
+
hl: hl,
|
|
2681
|
+
gl: gl,
|
|
2682
|
+
proxy_country: proxy_country,
|
|
2683
|
+
})
|
|
2684
|
+
end
|
|
2685
|
+
|
|
2630
2686
|
# Scrape Google Hotels search results (prices, ratings, amenities).
|
|
2631
2687
|
#
|
|
2632
2688
|
# Runs a Google Hotels search for `q` and returns the properties as JSON: `name`, `type`
|