scrapeunblocker 0.8.0 → 0.9.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: e8096ca55e9f93c01077d947e60e36ec71c595e7ee92d1405a413a0746c303f0
4
- data.tar.gz: 28c0667c78459edf026b26effd0209ea0635409eb078f152b8e4963be2312dd3
3
+ metadata.gz: f20529c692d1de01f3f02acca1accaf677cd5df2ec9d075bfb949b4b230272d2
4
+ data.tar.gz: b0eff5160bbd56e6e45700609ab17966d061d74537909c7a1a06df9242cddb7a
5
5
  SHA512:
6
- metadata.gz: f1b51dee641104d7d77fd600c5cb2299f30d6dcdf3e155686befc2d3024c981fb0c904d2aa8d5d1f360797a9cdbd331356ec2c99f1a3832907704198689306c6
7
- data.tar.gz: 8274485fa1644d6dfb1512bc33e93fc034f3c01a0248e450869315f3d0669244761ef146db9e11cac0ccfb3fe6361148d5b3ef64a4c706d967d825ef23fba9e6
6
+ metadata.gz: 2421c59b2bc7c32d769d01546b4f796b7f71237edf500af4ee21b56f0a1393532b6361cc4207264c6072c5510bd31dc840986adda94dfbc18e7d2fdbf03a1df8
7
+ data.tar.gz: 98532dd9a066ef4e52f1123a958e36f1f40df7f603a3055b78784337bedfb256a237343d011e086ba8d0feb94cd17100e2501d5317055b35bae15fc57bea27ea
data/CHANGELOG.md CHANGED
@@ -1,5 +1,11 @@
1
1
  # Changelog
2
2
 
3
+ ## 0.9.0 (2026-10-07)
4
+
5
+ - New `ScrapeUnblocker::BudgetExceededError` (a subclass of `PaymentRequiredError`): raised when the API answers 402 with `User set budget exceeded`, meaning this billing period's spend reached the monthly budget limit (EUR, excluding VAT) set in the [profile](https://app.scrapeunblocker.com/dashboard/profile). It clears at the start of the next billing period, or within about a minute after the limit is raised or removed. Like the other 402s it is never retried and not billed. Before, this body surfaced as a plain `PaymentRequiredError`.
6
+
7
+ No breaking changes: `rescue ScrapeUnblocker::PaymentRequiredError` still catches it.
8
+
3
9
  ## 0.8.0 (2026-10-06)
4
10
 
5
11
  - `get_parsed` on a page that rendered but held no structured data now returns a normal `ParsedPage` instead of raising `NoDataExtractedError`. The API answers this case with a billed 200: `data` is empty, the new `data_extracted?` is `false`, the new `detail` explains it and the new `html` carries the rendered page, so no second call is needed. On a successful parse `data_extracted?` is `true` and `html` / `detail` are `nil`.
data/README.md CHANGED
@@ -108,7 +108,6 @@ result["elements"].each { |el| p el }
108
108
  ```ruby
109
109
  result = su.get_parsed("https://www.walmart.com/ip/12345")
110
110
  puts result.page_type # e.g. "product"
111
- puts result.source # how it was extracted
112
111
  p result.data # the fields
113
112
 
114
113
  # If a parse ever comes back wrong, force a fresh set of rules:
@@ -259,7 +258,7 @@ begin
259
258
  rescue ScrapeUnblocker::BlockedError
260
259
  # 403: the target blocked every bypass path (not billed)
261
260
  rescue ScrapeUnblocker::PaymentRequiredError
262
- # 402: quota, credit limit, or a failed payment - fix billing
261
+ # 402: quota, credit limit, your budget limit, or a failed payment - fix billing
263
262
  rescue ScrapeUnblocker::RateLimitError
264
263
  # 429: slow down
265
264
  rescue ScrapeUnblocker::UpstreamOutageError
@@ -272,9 +271,10 @@ end
272
271
  | `InvalidRequestError` | 400 | Bad URL, unsupported scheme, or the API key header was not sent |
273
272
  | `AuthenticationError` | 401 | Key not recognised - typo, stray whitespace, or a rotated key |
274
273
  | `NoSubscriptionError` | 401 | Key is fine, but the account has no active plan |
275
- | `PaymentRequiredError` | 402 | Billing block - base class for the three below |
274
+ | `PaymentRequiredError` | 402 | Billing block - base class for the four below |
276
275
  | `QuotaExceededError` | 402 | The plan's requests for this period are used up |
277
276
  | `CreditLimitExceededError` | 402 | Unpaid balance is past the account's credit limit |
277
+ | `BudgetExceededError` | 402 | This billing period's spend reached the monthly budget limit you set |
278
278
  | `PaymentFailedError` | 402 | A card payment was declined three times |
279
279
  | `BlockedError` | 403 | Blocked by bot protection on every path |
280
280
  | `NotFoundError` | 404 | What you asked for does not exist - no image on the page (`get_image`), or a plugin lookup found nothing |
@@ -323,7 +323,7 @@ Transient failures (429, 502, 503, 504 and network errors) are retried automatic
323
323
 
324
324
  ### Billing errors (402)
325
325
 
326
- The three billing blocks share a status code and differ only in their message, so the client raises a dedicated error for each:
326
+ The four billing blocks share a status code and differ only in their message, so the client raises a dedicated error for each:
327
327
 
328
328
  ```ruby
329
329
  begin
@@ -332,12 +332,16 @@ rescue ScrapeUnblocker::QuotaExceededError
332
332
  # plan quota (plus any overage allowance) is used up for this period
333
333
  rescue ScrapeUnblocker::CreditLimitExceededError
334
334
  # unpaid balance passed the account credit limit
335
+ rescue ScrapeUnblocker::BudgetExceededError
336
+ # this period's spend reached the monthly budget limit set in your profile
335
337
  rescue ScrapeUnblocker::PaymentFailedError
336
338
  # card declined three times - update the payment method
337
339
  end
338
340
  ```
339
341
 
340
- When more than one applies, the most serious wins: failed payment outranks credit limit, which outranks quota. All three lift by themselves once the billing state changes - access returns within about a minute, and the API key stays the same. One catch worth knowing: subscribing to a new plan does **not** clear `PaymentFailedError`, because the old unpaid invoice stays open until it is paid.
342
+ When more than one applies, the most serious wins: failed payment outranks credit limit, which outranks quota, which outranks your own budget limit. All four lift by themselves once the billing state changes - access returns within about a minute, and the API key stays the same. One catch worth knowing: subscribing to a new plan does **not** clear `PaymentFailedError`, because the old unpaid invoice stays open until it is paid.
343
+
344
+ `BudgetExceededError` (body `User set budget exceeded`) means this billing period's spend reached the monthly budget limit you set in your [profile](https://app.scrapeunblocker.com/dashboard/profile?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk) (EUR, excluding VAT). The key works again at the start of the next billing period, or within about a minute after you raise or remove the limit.
341
345
 
342
346
  Full details for every status code: [docs.scrapeunblocker.com/errors](https://docs.scrapeunblocker.com/errors).
343
347
 
@@ -36,15 +36,16 @@ module ScrapeUnblocker
36
36
  # The account has a billing problem (HTTP 402).
37
37
  #
38
38
  # Credentials are fine - the request was stopped for a billing reason. There
39
- # are three, each raised as a dedicated subclass: QuotaExceededError,
40
- # CreditLimitExceededError and PaymentFailedError. Rescue this base class to
41
- # handle all three.
39
+ # are four, each raised as a dedicated subclass: QuotaExceededError,
40
+ # CreditLimitExceededError, BudgetExceededError and PaymentFailedError.
41
+ # Rescue this base class to handle all four.
42
42
  #
43
43
  # When more than one applies, the most serious wins: failed payment outranks
44
- # credit limit, which outranks quota. All three lift by themselves once the
45
- # billing state changes - access returns within roughly a minute, with no key
46
- # change needed. Like a 401, a 402 is refused before anything is scraped, so
47
- # it is never billed. Retrying is pointless; fix the billing state first.
44
+ # credit limit, which outranks quota, which outranks your own budget limit.
45
+ # All four lift by themselves once the billing state changes - access returns
46
+ # within roughly a minute, with no key change needed. Like a 401, a 402 is
47
+ # refused before anything is scraped, so it is never billed. Retrying is
48
+ # pointless; fix the billing state first.
48
49
  class PaymentRequiredError < APIError; end
49
50
 
50
51
  # Every request the plan allows this period has been used (HTTP 402).
@@ -64,6 +65,17 @@ module ScrapeUnblocker
64
65
  # usually clears itself within about a minute.
65
66
  class CreditLimitExceededError < PaymentRequiredError; end
66
67
 
68
+ # This billing period's spend reached the monthly budget limit you set
69
+ # (HTTP 402).
70
+ #
71
+ # The limit is set in your profile (EUR, excluding VAT), and spend is counted
72
+ # the way the invoice is: the plan's fixed monthly fee, if any, plus the
73
+ # requests billed on top of it. Requests paid from coupon credit do not count.
74
+ # The key works again at the start of the next billing period, or within
75
+ # about a minute after you raise or remove the limit at
76
+ # https://app.scrapeunblocker.com/dashboard/profile.
77
+ class BudgetExceededError < PaymentRequiredError; end
78
+
67
79
  # A card payment has been declined three times (HTTP 402).
68
80
  #
69
81
  # Those attempts are the payment provider's automatic retries spread over
@@ -167,7 +179,7 @@ module ScrapeUnblocker
167
179
  BASE_MESSAGES = {
168
180
  400 => "Invalid request (bad URL, unsupported scheme, or missing API key header)",
169
181
  401 => "Authentication failed - key not recognised, or account has no active plan",
170
- 402 => "Billing block - quota exceeded, credit limit exceeded, or a failed payment",
182
+ 402 => "Billing block - quota exceeded, credit limit exceeded, budget limit reached, or a failed payment",
171
183
  403 => "Target blocked by bot protection on every bypass path",
172
184
  404 => "Requested element not found on the page",
173
185
  408 => "Browser run timed out before the page was ready",
@@ -187,12 +199,13 @@ module ScrapeUnblocker
187
199
  end
188
200
  private_class_method :auth_error_class
189
201
 
190
- # The three billing blocks share a status code and differ only in their
202
+ # The four billing blocks share a status code and differ only in their
191
203
  # plain-text body. An unrecognised body falls back to PaymentRequiredError.
192
204
  def self.billing_error_class(body)
193
205
  text = (body || "").downcase
194
206
  return QuotaExceededError if text.include?("quota exceeded")
195
207
  return CreditLimitExceededError if text.include?("credit limit exceeded")
208
+ return BudgetExceededError if text.include?("user set budget exceeded")
196
209
  return PaymentFailedError if text.include?("payment failed")
197
210
 
198
211
  PaymentRequiredError
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module ScrapeUnblocker
4
- VERSION = "0.8.0"
4
+ VERSION = "0.9.0"
5
5
  end
metadata CHANGED
@@ -1,14 +1,14 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: scrapeunblocker
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.8.0
4
+ version: 0.9.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - ScrapeUnblocker
8
8
  autorequire:
9
9
  bindir: bin
10
10
  cert_chain: []
11
- date: 2026-10-06 00:00:00.000000000 Z
11
+ date: 2026-10-07 00:00:00.000000000 Z
12
12
  dependencies: []
13
13
  description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
14
14
  plus Google SERP and Skyscanner flights/hotels/car-hire scraping as JSON.