scrapeunblocker 0.1.9 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 28108195aac1cc524421665dfd8eafcf8c41e0419f0e692172b72f55ab3912c8
-  data.tar.gz: 9f80e41ef6047c3b12e32338739c9edd3a0d796211933a09f47b5dd3b70708e8
+  metadata.gz: 52fcc1782b1ae470a517b26515674f570071abefa9183613f772f54c002cd050
+  data.tar.gz: 688d1524ef190967afcd2272e7ca6cc17fc0a36417fec75a839f0cc2b607fde7
 SHA512:
-  metadata.gz: 04040b3a41778d5a98307082d147ca90be55992453dd64409e09b94a7cbee9b5c767585fccb3150d426505ba3f6f533215dafdf63a2e675ce024f6909fe2d925
-  data.tar.gz: 65a01b202aea9d3e94126692df4de80d35312821ee6a14425c6cca8f63e2cf59a0d8f51c38cd1e54e972cbe844e80e0b1a461b38948ce8855682d5ad6a7d67c3
+  metadata.gz: f60f1a37fcf62e63fe8195904a50464d7a06f6a1fcc4aac3cffe435fe5a9af0c7fa6de4a89ea0dc0658ed7e7c527f7b731bcb14389e5581913d1173f893f505d
+  data.tar.gz: c044299c381b979ad904cf1e0ecb67687e5e93ed91b44f4bc06af06bcf11cd11bda3e7bbabe08c3a0e63cac3474b24b5384141fe2999c9e5f06b548905f84e77

data/CHANGELOG.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## 0.2.1 (2026-09-02)
+
+- Added `meta_ad_library(advertiser, ...)` for the new Meta Ad Library plugin (`POST /ads/meta-ad-library`) - returns an advertiser's Meta (Facebook) Ad Library ads as a Hash. `advertiser` is required; optional filters `country`, `active_status`, `media_type` and `max_ads` are dropped when unset so the API applies its own defaults.
+
+No breaking changes.
+
+## 0.2.0 (2026-08-29)
+
+- `get_page_source` now accepts `steps:` - an ordered Array of browser-action Hashes the API runs in the real browser after the page loads (`wait_for`, `wait_for_text`, `wait`, `click`, `type`, `select`, `press_key`, `scroll`). The array is JSON-encoded into the `steps` query parameter. Steps run once and are not idempotent; a failed step returns HTTP 422 and raises `ScrapeUnblocker::ValidationError`, whose `body` names the failed step (`step_index`, `action`, `reason`, `selector`, `html`).
+- `get_page_source` now accepts `list_elements:` - pass `true` to get a JSON summary of the matched elements (`{"url", "count", "elements"}`) instead of HTML. When set, the method returns the parsed Hash rather than an HTML String, mirroring `get_parsed`.
+
+No breaking changes.
+
 ## 0.1.9 (2026-08-28)
 
 - Added `amazon_product` and `amazon_search` for the new Amazon plugin. `amazon_product(asin:/url:)` returns one product - title, brand, numeric price and currency, list price and savings, availability, rating, review count, seller, feature bullets, categories and images. `amazon_search(keyword, ...)` returns a keyword search's cards - asin, title, price, list price, rating, review count, a clean product URL, image and the sponsored/prime flags - on any of 20 regional marketplaces.

data/README.md

@@ -62,6 +62,47 @@ html = su.get_page_source(
 )
 ```
 
+## Browser steps
+
+Drive the page after it loads with an ordered list of browser actions - fill a form, click through, wait for content, then capture the rendered result. Steps run once in the real browser, in order.
+
+```ruby
+html = su.get_page_source(
+  "https://example.com/search",
+  steps: [
+    { action: "type", selector: "input#q", value: "web scraping", clear: true },
+    { action: "press_key", value: "Enter" },
+    { action: "wait_for", selector: "#results" },
+    { action: "scroll", value: "bottom" }
+  ]
+)
+```
+
+Supported actions and their fields:
+
+| Action | Fields |
+|---|---|
+| `wait_for` | `selector`, `selector_type` (`css` default, `xPath`, `className`, `tagName`), `timeout_ms` |
+| `wait_for_text` | `value` (text to await), `timeout_ms` |
+| `wait` | `value` (milliseconds) |
+| `click` | `selector`, `selector_type`, `timeout_ms` |
+| `type` | `selector`, `selector_type`, `value`, `clear` (bool), `timeout_ms` - types like a human |
+| `select` | `selector`, `selector_type`, `value`, `timeout_ms` |
+| `press_key` | `value` - one of `Enter`, `Tab`, `Escape`, `Backspace`, `Delete`, `Space`, `ArrowUp`, `ArrowDown`, `ArrowLeft`, `ArrowRight`, `Home`, `End`, `PageUp`, `PageDown` |
+| `scroll` | `value` - `"bottom"` or a pixel count |
+
+Steps are not idempotent (they submit forms, click buttons), so a call is not safe to blindly retry. If a step fails, the API answers HTTP 422 and the client raises `ScrapeUnblocker::ValidationError`; its `body` is the JSON describing which step failed (`error: "step_failed"`, `step_index`, `action`, `reason`, `selector`, `html`).
+
+## List elements
+
+Pass `list_elements: true` to get a JSON summary of the matched elements instead of the full HTML. The method then returns a parsed Hash (`{"url", "count", "elements"}`) rather than an HTML String.
+
+```ruby
+result = su.get_page_source("https://example.com", list_elements: true)
+puts result["count"]
+result["elements"].each { |el| p el }
+```
+
 ## Get parsed JSON
 
 ```ruby
@@ -87,6 +128,15 @@ local = su.google_local("coffee shops in chicago", proxy_country: "US", gl: "us"
 local["results"].each { |biz| puts "#{biz['name']} #{biz['rating']} #{biz['address']}" }
 ```
 
+## Meta Ad Library
+
+```ruby
+ads = su.meta_ad_library("Nike", country: "US")
+ads["results"].each { |ad| puts "#{ad['id']} #{ad['media_type']}" }
+```
+
+`advertiser` is required. Optional filters: `country`, `active_status`, `media_type` and `max_ads`. Unset filters are dropped and the API applies its own defaults.
+
 ## Oopbuy goods search
 
 ```ruby

data/lib/scrapeunblocker/client.rb

@@ -37,10 +37,31 @@ module ScrapeUnblocker
     end
 
     # Fetch a URL and return the fully rendered HTML.
-    def get_page_source(url, proxy_country: nil, time_sleep: nil, method: nil, value: nil, method_timeout: nil)
-      request("/getPageSource",
-              url: url, proxy_country: proxy_country, time_sleep: time_sleep,
-              method: method, value: value, method_timeout: method_timeout)[:body]
+    #
+    # +steps+ is an ordered Array of browser-action Hashes the API runs in the
+    # real browser after the page loads (each Hash carries an +action+ and its
+    # fields): +wait_for+ {selector, selector_type?, timeout_ms?}, +wait_for_text+
+    # {value, timeout_ms?}, +wait+ {value}, +click+ {selector, selector_type?,
+    # timeout_ms?}, +type+ {selector, selector_type?, value, clear?, timeout_ms?},
+    # +select+ {selector, selector_type?, value, timeout_ms?}, +press_key+ {value},
+    # +scroll+ {value}. +selector_type+ is one of "css" (default), "xPath",
+    # "className" or "tagName". The steps run once and are not idempotent; if a
+    # step fails the API answers HTTP 422 with a JSON body naming the failed step,
+    # which surfaces here as a ScrapeUnblocker::ValidationError.
+    #
+    # +list_elements+, when true, makes the API return a JSON summary of the
+    # matched elements ({"url", "count", "elements"}) instead of HTML. This method
+    # then returns that parsed Hash rather than an HTML String.
+    def get_page_source(url, proxy_country: nil, time_sleep: nil, method: nil, value: nil, method_timeout: nil,
+                        steps: nil, list_elements: nil)
+      body = request("/getPageSource",
+                     url: url, proxy_country: proxy_country, time_sleep: time_sleep,
+                     method: method, value: value, method_timeout: method_timeout,
+                     steps: (steps ? JSON.generate(steps) : nil),
+                     list_elements: (list_elements ? true : nil))[:body]
+      return JSON.parse(body) if list_elements
+
+      body
     end
 
     # Fetch a URL and return structured JSON instead of HTML.
@@ -80,6 +101,19 @@ module ScrapeUnblocker
                 keyword: keyword, proxy_country: proxy_country, hl: hl, gl: gl)
     end
 
+    # Fetch an advertiser's Meta (Facebook) Ad Library ads and return them as a Hash.
+    #
+    # +advertiser+ is the advertiser name or page to look up. Optional filters:
+    # +country+ (the Ad Library region), +active_status+ (active or inactive
+    # ads), +media_type+ (image, video, etc.) and +max_ads+ (a cap on how many
+    # ads to return). Unset filters are dropped from the request and the API
+    # applies its own defaults.
+    def meta_ad_library(advertiser, country: nil, active_status: nil, media_type: nil, max_ads: nil)
+      post_json("/ads/meta-ad-library",
+                advertiser: advertiser, country: country, active_status: active_status,
+                media_type: media_type, max_ads: max_ads)
+    end
+
     # Search Oopbuy (1688, Taobao or official channel) and return the goods as a Hash.
     #
     # Returns matched products, each with spu, channel, title, titleCn, price,

data/lib/scrapeunblocker/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ScrapeUnblocker
-  VERSION = "0.1.9"
+  VERSION = "0.2.1"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: scrapeunblocker
 version: !ruby/object:Gem::Version
-  version: 0.1.9
+  version: 0.2.1
 platform: ruby
 authors:
 - ScrapeUnblocker
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-08-28 00:00:00.000000000 Z
+date: 2026-09-02 00:00:00.000000000 Z
 dependencies: []
 description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
   plus Google SERP and Skyscanner flights/hotels/car-hire scraping as JSON.