scrapeunblocker 0.1.0 → 0.1.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +8 -0
- data/README.md +16 -0
- data/lib/scrapeunblocker/client.rb +23 -0
- data/lib/scrapeunblocker/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4d9a2366dce1f1f9586ccb5d9d0f5b9c667a1a9573f811093f017984118d138a
|
|
4
|
+
data.tar.gz: 527960e09c0159162825fe7ef7a75dcb071bafaf0872f847bcd25d6fd65df813
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e71d1368c7be85c86736955c25f39043457ae20e733d7ebee33416717981588eb8f35f82a16c10c9dd18b2d961f920121b984417e3dfb42aa450ecab5b2ace77
|
|
7
|
+
data.tar.gz: 58cb22b93d148eb3cdc342cf184268c4826f2567380909bb964cb8ebb37a26644995df0dcbcb183e8d1266d5f50fd1f0bf21ff075ddc69c3ef6b20ed5f7da39c
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,13 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.1.2 (2026-07-22)
|
|
4
|
+
|
|
5
|
+
- Added `oopbuy_search(keyword, ...)` for the new Oopbuy goods search plugin (`POST /goods/oopbuy-search`) - searches 1688, Taobao or the official channel and returns matched products (spu, title, price, monthSold, image, url) as a Hash.
|
|
6
|
+
|
|
7
|
+
## 0.1.1
|
|
8
|
+
|
|
9
|
+
- Added `google_local(keyword, ...)` for the new Google Local (Maps) plugin (`POST /maps/google-local`) - returns local business listings (name, rating, reviews, price, category, address, hours) as a Hash.
|
|
10
|
+
|
|
3
11
|
## 0.1.0
|
|
4
12
|
|
|
5
13
|
Initial release.
|
data/README.md
CHANGED
|
@@ -80,6 +80,22 @@ fresh = su.get_parsed(url, refresh_rules: true, rules_hint: "price is missing")
|
|
|
80
80
|
serp = su.serp("web scraping api", pages_to_check: 2, proxy_country: "US")
|
|
81
81
|
```
|
|
82
82
|
|
|
83
|
+
## Google Local (Maps)
|
|
84
|
+
|
|
85
|
+
```ruby
|
|
86
|
+
local = su.google_local("coffee shops in chicago", proxy_country: "US", gl: "us")
|
|
87
|
+
local["results"].each { |biz| puts "#{biz['name']} #{biz['rating']} #{biz['address']}" }
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
## Oopbuy goods search
|
|
91
|
+
|
|
92
|
+
```ruby
|
|
93
|
+
goods = su.oopbuy_search("running shoes", channel: "1688", page: 1, page_size: 20, sort: "default")
|
|
94
|
+
goods["results"].each { |item| puts "#{item['title']} #{item['price']} #{item['url']}" }
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
`channel` is one of `"1688"` (default), `"taobao"` or `"official"`. `sort` is one of `"default"`, `"price_asc"`, `"price_desc"` or `"best_selling"`. `page_size` max is 60. Brand keywords return HTTP 422.
|
|
98
|
+
|
|
83
99
|
## Cookies and the serving proxy
|
|
84
100
|
|
|
85
101
|
```ruby
|
|
@@ -70,6 +70,29 @@ module ScrapeUnblocker
|
|
|
70
70
|
captcha_pause: (captcha_pause.zero? ? nil : captcha_pause))
|
|
71
71
|
end
|
|
72
72
|
|
|
73
|
+
# Search Google Local (Maps) and return the businesses as a Hash.
|
|
74
|
+
#
|
|
75
|
+
# Returns up to ~20 businesses, each with name, rating, reviews, price,
|
|
76
|
+
# category, address, hours and a top review snippet. Local results are
|
|
77
|
+
# location-sensitive, so set +proxy_country+ (and optionally +gl+).
|
|
78
|
+
def google_local(keyword, proxy_country: nil, hl: nil, gl: nil)
|
|
79
|
+
post_json("/maps/google-local",
|
|
80
|
+
keyword: keyword, proxy_country: proxy_country, hl: hl, gl: gl)
|
|
81
|
+
end
|
|
82
|
+
|
|
83
|
+
# Search Oopbuy (1688, Taobao or official channel) and return the goods as a Hash.
|
|
84
|
+
#
|
|
85
|
+
# Returns matched products, each with spu, channel, title, titleCn, price,
|
|
86
|
+
# originalPrice, priceCny, monthSold, image and url. +channel+ is one of
|
|
87
|
+
# "1688" (default), "taobao" or "official"; +sort+ is one of "default",
|
|
88
|
+
# "price_asc", "price_desc" or "best_selling". +page_size+ max is 60.
|
|
89
|
+
# Brand keywords return HTTP 422.
|
|
90
|
+
def oopbuy_search(keyword, channel: "1688", page: 1, page_size: 20, sort: "default", proxy_country: nil)
|
|
91
|
+
post_json("/goods/oopbuy-search",
|
|
92
|
+
keyword: keyword, channel: channel, page: page,
|
|
93
|
+
page_size: page_size, sort: sort, proxy_country: proxy_country)
|
|
94
|
+
end
|
|
95
|
+
|
|
73
96
|
# Fetch an image URL through the bypass chain and return its raw bytes.
|
|
74
97
|
def get_image(url, proxy_country: nil)
|
|
75
98
|
request("/getImage", url: url, proxy_country: proxy_country)[:body]
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: scrapeunblocker
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.1.
|
|
4
|
+
version: 0.1.2
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- ScrapeUnblocker
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-07-
|
|
11
|
+
date: 2026-07-22 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
|
|
14
14
|
plus Google SERP and Skyscanner flights/hotels/car-hire scraping as JSON.
|