scrapeunblocker 0.1.7 → 0.1.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -0
- data/README.md +32 -0
- data/lib/scrapeunblocker/client.rb +57 -0
- data/lib/scrapeunblocker/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 28108195aac1cc524421665dfd8eafcf8c41e0419f0e692172b72f55ab3912c8
|
|
4
|
+
data.tar.gz: 9f80e41ef6047c3b12e32338739c9edd3a0d796211933a09f47b5dd3b70708e8
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 04040b3a41778d5a98307082d147ca90be55992453dd64409e09b94a7cbee9b5c767585fccb3150d426505ba3f6f533215dafdf63a2e675ce024f6909fe2d925
|
|
7
|
+
data.tar.gz: 65a01b202aea9d3e94126692df4de80d35312821ee6a14425c6cca8f63e2cf59a0d8f51c38cd1e54e972cbe844e80e0b1a461b38948ce8855682d5ad6a7d67c3
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,18 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.1.9 (2026-08-28)
|
|
4
|
+
|
|
5
|
+
- Added `amazon_product` and `amazon_search` for the new Amazon plugin. `amazon_product(asin:/url:)` returns one product - title, brand, numeric price and currency, list price and savings, availability, rating, review count, seller, feature bullets, categories and images. `amazon_search(keyword, ...)` returns a keyword search's cards - asin, title, price, list price, rating, review count, a clean product URL, image and the sponsored/prime flags - on any of 20 regional marketplaces.
|
|
6
|
+
- Prices come back in the right currency automatically: `proxy_country` defaults to the marketplace's home country (amazon.com -> US, amazon.de -> DE), pinning the exit over our ISP pool.
|
|
7
|
+
|
|
8
|
+
## 0.1.8 (2026-07-31)
|
|
9
|
+
|
|
10
|
+
- Added `ebay_search` for the new eBay Search plugin: listings from any of the 19 regional eBay marketplaces as structured JSON - title, numeric price and currency, condition with a normalised `conditionCode`, seller username and feedback, shipping cost, sold/watcher/bid counts, image and a clean item URL.
|
|
11
|
+
- Filters map straight onto the plugin: `marketplace`, `condition`, `sort`, `listing_type`, `min_price`/`max_price`, `free_shipping`, `seller`, `category`, plus `page`/`page_size` (60, 120 or 240).
|
|
12
|
+
- The response carries `exactMatches`; it is `false` when eBay found no match for the keyword and answered with its own loosely-related suggestions.
|
|
13
|
+
|
|
14
|
+
No breaking changes.
|
|
15
|
+
|
|
3
16
|
## 0.1.7 (2026-07-27)
|
|
4
17
|
|
|
5
18
|
- Registry and README links to scrapeunblocker.com now carry UTM parameters so traffic from package registries is attributable. No functional changes.
|
data/README.md
CHANGED
|
@@ -96,6 +96,38 @@ goods["results"].each { |item| puts "#{item['title']} #{item['price']} #{item['u
|
|
|
96
96
|
|
|
97
97
|
`channel` is one of `"1688"` (default), `"taobao"` or `"official"`. `sort` is one of `"default"`, `"price_asc"`, `"price_desc"` or `"best_selling"`. `page_size` max is 60. Oopbuy trademark-blocks brand keywords at its own backend: those come back as a successful `200` with `keywordRejected: true` and an empty `results` array, not an error.
|
|
98
98
|
|
|
99
|
+
## Amazon
|
|
100
|
+
|
|
101
|
+
Product and search data as a Hash, priced in the marketplace's own currency:
|
|
102
|
+
|
|
103
|
+
```ruby
|
|
104
|
+
# One product by ASIN (or url: "https://www.amazon.de/dp/B0BSHF7WHW")
|
|
105
|
+
product = su.amazon_product(asin: "B0BSHF7WHW", marketplace: "amazon.com")
|
|
106
|
+
puts [product["title"], product["price"], product["currency"], product["rating"]].join(" | ")
|
|
107
|
+
|
|
108
|
+
# Keyword search
|
|
109
|
+
results = su.amazon_search("wireless headphones", sort: "price_asc")
|
|
110
|
+
results["results"].each do |item|
|
|
111
|
+
puts [item["title"], item["price"], item["currency"], item["asin"]].join(" | ")
|
|
112
|
+
end
|
|
113
|
+
```
|
|
114
|
+
|
|
115
|
+
`proxy_country` defaults to the marketplace's home country (`amazon.com` -> US, `amazon.de` -> DE), so prices come back in the right currency with no configuration.
|
|
116
|
+
|
|
117
|
+
## eBay search
|
|
118
|
+
|
|
119
|
+
```ruby
|
|
120
|
+
items = su.ebay_search("iphone 13", marketplace: "ebay.com", condition: "used", sort: "newly_listed")
|
|
121
|
+
|
|
122
|
+
if items["exactMatches"]
|
|
123
|
+
items["results"].each do |item|
|
|
124
|
+
puts [item["title"], item["price"], item["currency"], item["condition"]].join(" | ")
|
|
125
|
+
end
|
|
126
|
+
end
|
|
127
|
+
```
|
|
128
|
+
|
|
129
|
+
`marketplace` is any of the 19 regional eBay hosts (`ebay.com` default). `condition` is one of `"new"`, `"open_box"`, `"refurbished"`, `"used"` or `"for_parts"`; `sort` is one of `"best_match"` (default), `"newly_listed"`, `"ending_soon"`, `"price_asc"` or `"price_desc"`; `page_size` is 60, 120 or 240. `exactMatches` is `false` when eBay found nothing for the keyword and answered with its own loosely-related suggestions instead, so check it before using the listings.
|
|
130
|
+
|
|
99
131
|
## Cookies and the serving proxy
|
|
100
132
|
|
|
101
133
|
```ruby
|
|
@@ -93,6 +93,63 @@ module ScrapeUnblocker
|
|
|
93
93
|
page_size: page_size, sort: sort, proxy_country: proxy_country)
|
|
94
94
|
end
|
|
95
95
|
|
|
96
|
+
# Search eBay and return the listings as a Hash.
|
|
97
|
+
#
|
|
98
|
+
# Each listing carries title, numeric price and currency, condition (with a
|
|
99
|
+
# normalised conditionCode), seller username and feedback, shipping cost,
|
|
100
|
+
# sold/watcher/bid counts, image and a clean item URL.
|
|
101
|
+
#
|
|
102
|
+
# +marketplace+ is a regional eBay host such as "ebay.com" (default) or
|
|
103
|
+
# "ebay.de"; +condition+ is one of "new", "open_box", "refurbished", "used"
|
|
104
|
+
# or "for_parts"; +sort+ is one of "best_match" (default), "newly_listed",
|
|
105
|
+
# "ending_soon", "price_asc" or "price_desc"; +listing_type+ is "all"
|
|
106
|
+
# (default), "buy_it_now" or "auction"; +page_size+ is 60, 120 or 240.
|
|
107
|
+
#
|
|
108
|
+
# When eBay finds no exact match it still serves a page of loosely related
|
|
109
|
+
# suggestions, and the response then carries <tt>exactMatches: false</tt>.
|
|
110
|
+
def ebay_search(keyword, marketplace: "ebay.com", page: 1, page_size: 60,
|
|
111
|
+
condition: nil, sort: "best_match", listing_type: "all",
|
|
112
|
+
min_price: nil, max_price: nil, free_shipping: false,
|
|
113
|
+
seller: nil, category: nil, proxy_country: nil)
|
|
114
|
+
post_json("/marketplace/ebay-search",
|
|
115
|
+
keyword: keyword, marketplace: marketplace, page: page,
|
|
116
|
+
page_size: page_size, condition: condition, sort: sort,
|
|
117
|
+
listing_type: listing_type, min_price: min_price,
|
|
118
|
+
max_price: max_price,
|
|
119
|
+
free_shipping: free_shipping ? true : nil,
|
|
120
|
+
seller: seller, category: category,
|
|
121
|
+
proxy_country: proxy_country)
|
|
122
|
+
end
|
|
123
|
+
|
|
124
|
+
# Scrape one Amazon product by ASIN or URL and return it as a Hash.
|
|
125
|
+
#
|
|
126
|
+
# Returns title, brand, numeric price and currency, list price and savings,
|
|
127
|
+
# availability, rating, review count, seller, feature bullets, categories
|
|
128
|
+
# and images. Prices come back in the marketplace's own currency:
|
|
129
|
+
# +proxy_country+ defaults to the marketplace's home country
|
|
130
|
+
# (amazon.com -> US), pinning the exit over the ISP pool. Pass either
|
|
131
|
+
# +asin+ (with +marketplace+) or a full product +url+.
|
|
132
|
+
def amazon_product(asin: nil, url: nil, marketplace: "amazon.com", proxy_country: nil)
|
|
133
|
+
post_json("/marketplace/amazon-product",
|
|
134
|
+
asin: asin, url: url, marketplace: marketplace,
|
|
135
|
+
proxy_country: proxy_country)
|
|
136
|
+
end
|
|
137
|
+
|
|
138
|
+
# Search Amazon and return the result cards as an Array of Hashes.
|
|
139
|
+
#
|
|
140
|
+
# Each card carries asin, title, numeric price and currency, list price,
|
|
141
|
+
# rating, review count, a clean product URL, image and the sponsored /
|
|
142
|
+
# prime flags. +sort+ is "featured" (default), "price_asc", "price_desc",
|
|
143
|
+
# "avg_review" or "newest". Prices are in the marketplace's own currency;
|
|
144
|
+
# +proxy_country+ defaults to the marketplace's home country.
|
|
145
|
+
def amazon_search(keyword, marketplace: "amazon.com", page: 1, sort: "featured",
|
|
146
|
+
min_price: nil, max_price: nil, proxy_country: nil)
|
|
147
|
+
post_json("/marketplace/amazon-search",
|
|
148
|
+
keyword: keyword, marketplace: marketplace, page: page,
|
|
149
|
+
sort: sort, min_price: min_price, max_price: max_price,
|
|
150
|
+
proxy_country: proxy_country)
|
|
151
|
+
end
|
|
152
|
+
|
|
96
153
|
# Fetch an image URL through the bypass chain and return its raw bytes.
|
|
97
154
|
def get_image(url, proxy_country: nil)
|
|
98
155
|
request("/getImage", url: url, proxy_country: proxy_country)[:body]
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: scrapeunblocker
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.9
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- ScrapeUnblocker
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-
|
|
11
|
+
date: 2026-08-28 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
|
|
14
14
|
plus Google SERP and Skyscanner flights/hotels/car-hire scraping as JSON.
|