scrapeunblocker 0.11.0 → 0.11.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +5 -0
- data/README.md +4 -2
- data/lib/scrapeunblocker/client.rb +5 -1
- data/lib/scrapeunblocker/errors.rb +8 -7
- data/lib/scrapeunblocker/version.rb +1 -1
- metadata +2 -2
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 9bd6d663c941e4d49cc8c8b7446f15c2c4cff0ac7025e85144dd72706ac24220
|
|
4
|
+
data.tar.gz: 68949c187f57eb403d1f890a236fd9caa671beeaf395225c8100d4da22ce8fac
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: b6c58655e6ea4657b358e0edf1c147226d2fdbb5b55b64a18b88f3f5b5d821b58b78ef29ccb2f505554998c6ef9e8183afa45f5dcf01fcf9f9eb3fb5dea28d75
|
|
7
|
+
data.tar.gz: 8ab2b5635e0b7edc16bfd1202e92ed24e0a189a115d2f37ea26e2c21c9b7780e5f9f942a50edda29cdb6645e2b7a61228bcea014abe08c44f20f5565f31f1057
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,10 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.11.1 (2026-10-15)
|
|
4
|
+
|
|
5
|
+
- `get_image` docs: the API now returns the image's original bytes and format (JPEG, PNG, WebP, AVIF, GIF, SVG, ...) instead of a PNG re-encode. The method still returns a binary `String`.
|
|
6
|
+
- A missing image (the host's own 404/410) raises `TargetNotFoundError`; a URL that is not an image raises `UnsupportedContentError`. The 415 message no longer says "does not serve HTML", which was wrong for `get_image`.
|
|
7
|
+
|
|
3
8
|
## 0.11.0 (2026-10-13)
|
|
4
9
|
|
|
5
10
|
- Removed four plugin methods added in 0.10.0: `youtube_comments`, `telegram_channel`, `quora_question` and `quora_profile`. Those plugins are no longer offered through the SDKs; `plugin(path, params)` still reaches any endpoint your account can use.
|
data/README.md
CHANGED
|
@@ -242,6 +242,8 @@ bytes = su.get_image("https://example.com/photo.jpg")
|
|
|
242
242
|
File.binwrite("photo.jpg", bytes)
|
|
243
243
|
```
|
|
244
244
|
|
|
245
|
+
The bytes come back exactly as the image host serves them, in the original format (JPEG, PNG, WebP, AVIF, GIF, SVG, ...); nothing is re-encoded. Pass the image URL itself, not the page it sits on.
|
|
246
|
+
|
|
245
247
|
## Skyscanner plugins
|
|
246
248
|
|
|
247
249
|
Flights, hotels and car hire as JSON:
|
|
@@ -287,10 +289,10 @@ end
|
|
|
287
289
|
| `BudgetExceededError` | 402 | This billing period's spend reached the monthly budget limit you set |
|
|
288
290
|
| `PaymentFailedError` | 402 | A card payment was declined three times |
|
|
289
291
|
| `BlockedError` | 403 | Blocked by bot protection on every path |
|
|
290
|
-
| `NotFoundError` | 404 | What you asked for does not exist -
|
|
292
|
+
| `NotFoundError` | 404 | What you asked for does not exist - a plugin lookup found nothing |
|
|
291
293
|
| `TargetNotFoundError` | 404 / 410 | The target page itself does not exist; carries `origin_status`, `html`, `destination_url` (subclass of `NotFoundError`, billed) |
|
|
292
294
|
| `BrowserTimeoutError` | 408 | Our browser run timed out before the page was ready |
|
|
293
|
-
| `UnsupportedContentError` | 415 | The URL serves something other than HTML |
|
|
295
|
+
| `UnsupportedContentError` | 415 | The URL serves something other than HTML (`get_page_source`) or other than an image (`get_image`) |
|
|
294
296
|
| `ValidationError` | 422 | Missing or wrong-typed parameter; `body` holds the `detail` array |
|
|
295
297
|
| `RateLimitError` | 429 | Too many requests |
|
|
296
298
|
| `UpstreamOutageError` | 503 | The target origin is down |
|
|
@@ -215,7 +215,11 @@ module ScrapeUnblocker
|
|
|
215
215
|
proxy_country: proxy_country)
|
|
216
216
|
end
|
|
217
217
|
|
|
218
|
-
# Fetch an image
|
|
218
|
+
# Fetch an image and return its bytes exactly as the image host serves them -
|
|
219
|
+
# the original format (JPEG, PNG, WebP, AVIF, GIF, SVG, ...), nothing
|
|
220
|
+
# re-encoded. +url+ must point at the image itself: a page URL raises
|
|
221
|
+
# UnsupportedContentError, and an image the host answers 404 or 410 for
|
|
222
|
+
# raises TargetNotFoundError.
|
|
219
223
|
def get_image(url, proxy_country: nil)
|
|
220
224
|
request("/getImage", url: url, proxy_country: proxy_country)[:body]
|
|
221
225
|
end
|
|
@@ -94,16 +94,15 @@ module ScrapeUnblocker
|
|
|
94
94
|
|
|
95
95
|
# Something the call asked for does not exist (HTTP 404).
|
|
96
96
|
#
|
|
97
|
-
#
|
|
98
|
-
# plugin methods raise it when the item they look up does not exist. When the
|
|
97
|
+
# Plugin methods raise it when the item they look up does not exist. When the
|
|
99
98
|
# target page itself answered 404 or 410, the more specific
|
|
100
99
|
# TargetNotFoundError subclass is raised instead.
|
|
101
100
|
class NotFoundError < APIError; end
|
|
102
101
|
|
|
103
102
|
# The target page itself does not exist (HTTP 404 or 410).
|
|
104
103
|
#
|
|
105
|
-
# Raised by #get_page_source, #get_parsed
|
|
106
|
-
# site you asked for answered 404 or 410 on its own. The API passes that
|
|
104
|
+
# Raised by #get_page_source, #get_parsed, #get_page_with_cookies and
|
|
105
|
+
# #get_image when the site you asked for answered 404 or 410 on its own. The API passes that
|
|
107
106
|
# status through and marks it with the X-Origin-Status header, which is how
|
|
108
107
|
# this is told apart from an API-side 404. It is the target's final answer,
|
|
109
108
|
# not a block, so it is never retried - and the call is billed, because the
|
|
@@ -130,8 +129,10 @@ module ScrapeUnblocker
|
|
|
130
129
|
# the API answered - it just could not render the page in time.
|
|
131
130
|
class BrowserTimeoutError < APIError; end
|
|
132
131
|
|
|
133
|
-
# The URL serves
|
|
134
|
-
#
|
|
132
|
+
# The URL serves a content type the call cannot return (HTTP 415): something
|
|
133
|
+
# other than HTML for #get_page_source (for images, use #get_image), or
|
|
134
|
+
# something other than an image for #get_image. The message names the content
|
|
135
|
+
# type found.
|
|
135
136
|
class UnsupportedContentError < APIError; end
|
|
136
137
|
|
|
137
138
|
# A request parameter is missing or has the wrong type (HTTP 422).
|
|
@@ -183,7 +184,7 @@ module ScrapeUnblocker
|
|
|
183
184
|
403 => "Target blocked by bot protection on every bypass path",
|
|
184
185
|
404 => "Requested element not found on the page",
|
|
185
186
|
408 => "Browser run timed out before the page was ready",
|
|
186
|
-
415 => "URL
|
|
187
|
+
415 => "URL serves a content type this call cannot return",
|
|
187
188
|
422 => "Validation error - see the detail array in the response body",
|
|
188
189
|
429 => "Rate limited - too many requests",
|
|
189
190
|
503 => "Upstream origin returned a server-side outage page",
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: scrapeunblocker
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.11.
|
|
4
|
+
version: 0.11.1
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- ScrapeUnblocker
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-10-
|
|
11
|
+
date: 2026-10-15 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
|
|
14
14
|
plus Google SERP and 100+ ready-made site plugins (Amazon, Walmart, Zillow, Google
|