scrapeunblocker 0.11.0 → 0.12.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 506dfab9e7e8c63f8b2808a9f7cdf8b113a509fe94b1841c2e8970a33e2bc451
4
- data.tar.gz: b1bc717e7851b88651cf6960cabfb8d9298589d0494040be702a1c9ce5d953e4
3
+ metadata.gz: 9349609e20a6b8949d1cc3d77b919cf6c916b4b1917d181bb2d18ec441f46777
4
+ data.tar.gz: 55bf0f292146e6e166a502ecb43a33d15b2629261f808ed8754f4ca7a33d96fe
5
5
  SHA512:
6
- metadata.gz: 5f1901a2940ae350e8603fe8cb4776ecdb433173340010ec8c320ef063c6f4d906ebd135ca36359a3d708b2722826f1db77d56d785724ff7c5a7bbddef01db4b
7
- data.tar.gz: 0dce31734e4c853bae21c43581880ff84251990ba1ea8a8023002b4705d669206141b10ec8dea45db4fe825a384334a3d7cbe5b07d3feeba0bb5ef50c7b49f94
6
+ metadata.gz: 22da248af5842639758605b333930c888534405b796cd199f9c8a2c8bced4104e6dace474c62fbaa2aa878d321878aef3118121fb0563efc4f25a1d2c00451b6
7
+ data.tar.gz: 3fea75e8eb0602eb156978c019fea92fe7cfca41aab0714f9049412d26207e707899750ad8547b0caf57d3da368d347d2f2ead34e82d74ee2d4764de5f90afb9
data/CHANGELOG.md CHANGED
@@ -1,5 +1,16 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.12.0 (2026-10-15)
4
+
5
+ - `google_images` takes `time_range:` (`"hour"`, `"day"`, `"week"`, `"month"` or `"year"`) to return only the images Google found in that recent period, so a query scanned on a schedule returns only what is new. For a fixed window pass `date_from:` and/or `date_to:` (`"YYYY-MM-DD"`) instead; the API answers 400 when `time_range` is combined with either.
6
+
7
+ No breaking changes.
8
+
9
+ ## 0.11.1 (2026-10-15)
10
+
11
+ - `get_image` docs: the API now returns the image's original bytes and format (JPEG, PNG, WebP, AVIF, GIF, SVG, ...) instead of a PNG re-encode. The method still returns a binary `String`.
12
+ - A missing image (the host's own 404/410) raises `TargetNotFoundError`; a URL that is not an image raises `UnsupportedContentError`. The 415 message no longer says "does not serve HTML", which was wrong for `get_image`.
13
+
3
14
  ## 0.11.0 (2026-10-13)
4
15
 
5
16
  - Removed four plugin methods added in 0.10.0: `youtube_comments`, `telegram_channel`, `quora_question` and `quora_profile`. Those plugins are no longer offered through the SDKs; `plugin(path, params)` still reaches any endpoint your account can use.
data/README.md CHANGED
@@ -174,8 +174,15 @@ local["results"].each { |biz| puts "#{biz['name']} #{biz['rating']} #{biz['addre
174
174
  ```ruby
175
175
  images = su.google_images("golden retriever puppy", proxy_country: "US", pages: 3)
176
176
  images["results"].each { |img| puts "#{img['imageUrl']} #{img['sourceDomain']} #{img['title']}" }
177
+
178
+ # Only images Google found in the last day (for a daily scan of the same query)
179
+ new_images = su.google_images("eiffel tower", proxy_country: "US", time_range: "day")
180
+ # Or a fixed window: date_from / date_to (YYYY-MM-DD), each one optional
181
+ october = su.google_images("eiffel tower", date_from: "2026-10-01", date_to: "2026-10-07")
177
182
  ```
178
183
 
184
+ `time_range` takes `hour`, `day`, `week`, `month` or `year` and cannot be combined with `date_from` / `date_to`.
185
+
179
186
  ## Meta Ad Library
180
187
 
181
188
  ```ruby
@@ -242,6 +249,8 @@ bytes = su.get_image("https://example.com/photo.jpg")
242
249
  File.binwrite("photo.jpg", bytes)
243
250
  ```
244
251
 
252
+ The bytes come back exactly as the image host serves them, in the original format (JPEG, PNG, WebP, AVIF, GIF, SVG, ...); nothing is re-encoded. Pass the image URL itself, not the page it sits on.
253
+
245
254
  ## Skyscanner plugins
246
255
 
247
256
  Flights, hotels and car hire as JSON:
@@ -287,10 +296,10 @@ end
287
296
  | `BudgetExceededError` | 402 | This billing period's spend reached the monthly budget limit you set |
288
297
  | `PaymentFailedError` | 402 | A card payment was declined three times |
289
298
  | `BlockedError` | 403 | Blocked by bot protection on every path |
290
- | `NotFoundError` | 404 | What you asked for does not exist - no image on the page (`get_image`), or a plugin lookup found nothing |
299
+ | `NotFoundError` | 404 | What you asked for does not exist - a plugin lookup found nothing |
291
300
  | `TargetNotFoundError` | 404 / 410 | The target page itself does not exist; carries `origin_status`, `html`, `destination_url` (subclass of `NotFoundError`, billed) |
292
301
  | `BrowserTimeoutError` | 408 | Our browser run timed out before the page was ready |
293
- | `UnsupportedContentError` | 415 | The URL serves something other than HTML |
302
+ | `UnsupportedContentError` | 415 | The URL serves something other than HTML (`get_page_source`) or other than an image (`get_image`) |
294
303
  | `ValidationError` | 422 | Missing or wrong-typed parameter; `body` holds the `detail` array |
295
304
  | `RateLimitError` | 429 | Too many requests |
296
305
  | `UpstreamOutageError` | 503 | The target origin is down |
@@ -126,10 +126,16 @@ module ScrapeUnblocker
126
126
  # +proxy_country+ to target a market (the Google market follows it; +gl+
127
127
  # is an optional override), +pages+ (1-5, ~100 results each; each page
128
128
  # fetched is billed as one request, see +pagesFetched+) and +max_results+
129
- # (1-500) to cap the count.
130
- def google_images(q, proxy_country: nil, pages: nil, gl: nil, max_results: nil)
129
+ # (1-500) to cap the count. +time_range+ ("hour", "day", "week", "month",
130
+ # "year") keeps only the images Google found in that recent period, e.g.
131
+ # the new ones since the last scheduled scan; +date_from+ / +date_to+
132
+ # ("YYYY-MM-DD", each optional) set a fixed window instead. The two forms
133
+ # cannot be combined.
134
+ def google_images(q, proxy_country: nil, pages: nil, gl: nil, max_results: nil,
135
+ time_range: nil, date_from: nil, date_to: nil)
131
136
  post_json("/images/google-search",
132
- q: q, pages: pages, gl: gl, max_results: max_results, proxy_country: proxy_country)
137
+ q: q, pages: pages, gl: gl, max_results: max_results, proxy_country: proxy_country,
138
+ time_range: time_range, date_from: date_from, date_to: date_to)
133
139
  end
134
140
 
135
141
  # Fetch an advertiser's Meta (Facebook) Ad Library ads and return them as a Hash.
@@ -215,7 +221,11 @@ module ScrapeUnblocker
215
221
  proxy_country: proxy_country)
216
222
  end
217
223
 
218
- # Fetch an image URL through the bypass chain and return its raw bytes.
224
+ # Fetch an image and return its bytes exactly as the image host serves them -
225
+ # the original format (JPEG, PNG, WebP, AVIF, GIF, SVG, ...), nothing
226
+ # re-encoded. +url+ must point at the image itself: a page URL raises
227
+ # UnsupportedContentError, and an image the host answers 404 or 410 for
228
+ # raises TargetNotFoundError.
219
229
  def get_image(url, proxy_country: nil)
220
230
  request("/getImage", url: url, proxy_country: proxy_country)[:body]
221
231
  end
@@ -94,16 +94,15 @@ module ScrapeUnblocker
94
94
 
95
95
  # Something the call asked for does not exist (HTTP 404).
96
96
  #
97
- # #get_image raises it when the page rendered but held no <img> tag, and
98
- # plugin methods raise it when the item they look up does not exist. When the
97
+ # Plugin methods raise it when the item they look up does not exist. When the
99
98
  # target page itself answered 404 or 410, the more specific
100
99
  # TargetNotFoundError subclass is raised instead.
101
100
  class NotFoundError < APIError; end
102
101
 
103
102
  # The target page itself does not exist (HTTP 404 or 410).
104
103
  #
105
- # Raised by #get_page_source, #get_parsed and #get_page_with_cookies when the
106
- # site you asked for answered 404 or 410 on its own. The API passes that
104
+ # Raised by #get_page_source, #get_parsed, #get_page_with_cookies and
105
+ # #get_image when the site you asked for answered 404 or 410 on its own. The API passes that
107
106
  # status through and marks it with the X-Origin-Status header, which is how
108
107
  # this is told apart from an API-side 404. It is the target's final answer,
109
108
  # not a block, so it is never retried - and the call is billed, because the
@@ -130,8 +129,10 @@ module ScrapeUnblocker
130
129
  # the API answered - it just could not render the page in time.
131
130
  class BrowserTimeoutError < APIError; end
132
131
 
133
- # The URL serves something other than HTML (HTTP 415).
134
- # The message names the content type found. For images, use #get_image.
132
+ # The URL serves a content type the call cannot return (HTTP 415): something
133
+ # other than HTML for #get_page_source (for images, use #get_image), or
134
+ # something other than an image for #get_image. The message names the content
135
+ # type found.
135
136
  class UnsupportedContentError < APIError; end
136
137
 
137
138
  # A request parameter is missing or has the wrong type (HTTP 422).
@@ -183,7 +184,7 @@ module ScrapeUnblocker
183
184
  403 => "Target blocked by bot protection on every bypass path",
184
185
  404 => "Requested element not found on the page",
185
186
  408 => "Browser run timed out before the page was ready",
186
- 415 => "URL does not serve HTML",
187
+ 415 => "URL serves a content type this call cannot return",
187
188
  422 => "Validation error - see the detail array in the response body",
188
189
  429 => "Rate limited - too many requests",
189
190
  503 => "Upstream origin returned a server-side outage page",
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module ScrapeUnblocker
4
- VERSION = "0.11.0"
4
+ VERSION = "0.12.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: scrapeunblocker
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.11.0
4
+ version: 0.12.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - ScrapeUnblocker
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-10-13 00:00:00.000000000 Z
11
+ date: 2026-10-15 00:00:00.000000000 Z
12
12
  dependencies: []
13
13
  description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
14
14
  plus Google SERP and 100+ ready-made site plugins (Amazon, Walmart, Zillow, Google