scrapeunblocker 0.2.1 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 52fcc1782b1ae470a517b26515674f570071abefa9183613f772f54c002cd050
-  data.tar.gz: 688d1524ef190967afcd2272e7ca6cc17fc0a36417fec75a839f0cc2b607fde7
+  metadata.gz: fbe2fc1ce79fbf6b9293fe17403a6293599bf6cb04fe775d3596537a40fd5acb
+  data.tar.gz: 4b9dce4731d710996fe0796766bc67829d1cdd5753e6a812f3fc74796cdf7c48
 SHA512:
-  metadata.gz: f60f1a37fcf62e63fe8195904a50464d7a06f6a1fcc4aac3cffe435fe5a9af0c7fa6de4a89ea0dc0658ed7e7c527f7b731bcb14389e5581913d1173f893f505d
-  data.tar.gz: c044299c381b979ad904cf1e0ecb67687e5e93ed91b44f4bc06af06bcf11cd11bda3e7bbabe08c3a0e63cac3474b24b5384141fe2999c9e5f06b548905f84e77
+  metadata.gz: bca9218a5cee0b1bff9b9160860cc2d1cdd69e0b115abf3db35d449d005828e1f52099395f9d9ff0072ee4b509155cdc9088690d947a964179a3cb57823fffe2
+  data.tar.gz: 14a9b61b2f89d2bd23d2356a7fdc35fe16c075862251173aa29cb2463633705132f19f4f87a8e76e7dd637d7f955f23078151389007c0a6860f8b21bddc6a3a0

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## 0.4.0 (2026-09-16)
+
+- Added `southwest.flights(origin:, dest:, depart_date:, ...)` for the new Southwest Airlines plugin (`POST /flights/southwest-quotes`), reached through the new `su.southwest` namespace and mirroring `su.skyscanner`. `origin` and `dest` are IATA airport codes; `depart_date` and the optional `return_date` are `YYYY-MM-DD` (omit `return_date` for a one-way search). Optional `adults` (1-8, default 1), `fare_type` (`"dollars"` default or `"points"`), `proxy_country` (default `"US"`) and `max_attempts` (1-5, default 3). Returns the raw booking / shopping JSON as a Hash.
+
+No breaking changes.
+
+## 0.3.0 (2026-09-08)
+
+- Added `tiktok_profile`, `tiktok_video`, `tiktok_hashtag`, `tiktok_search` and `tiktok_comments` for the new TikTok plugin: a creator's exact follower / like / video counts with their newest videos (up to 200), any video or photo post with exact plays, likes, comments, shares, saves and reposts, hashtags, music, play / download URLs, subtitle tracks and an optional transcript, a hashtag's total views and videos with its videos, keyword search in TikTok's own ranking, and the comments of any post. No login.
+
 ## 0.2.1 (2026-09-02)
 
 - Added `meta_ad_library(advertiser, ...)` for the new Meta Ad Library plugin (`POST /ads/meta-ad-library`) - returns an advertiser's Meta (Facebook) Ad Library ads as a Hash. `advertiser` is required; optional filters `country`, `active_status`, `media_type` and `max_ads` are dropped when unset so the API applies its own defaults.

data/README.md

@@ -178,6 +178,19 @@ end
 
 `marketplace` is any of the 19 regional eBay hosts (`ebay.com` default). `condition` is one of `"new"`, `"open_box"`, `"refurbished"`, `"used"` or `"for_parts"`; `sort` is one of `"best_match"` (default), `"newly_listed"`, `"ending_soon"`, `"price_asc"` or `"price_desc"`; `page_size` is 60, 120 or 240. `exactMatches` is `false` when eBay found nothing for the keyword and answered with its own loosely-related suggestions instead, so check it before using the listings.
 
+### TikTok
+
+```ruby
+profile = su.tiktok_profile("nasa", max_videos: 5)      # exact stats + newest videos
+video = su.tiktok_video("https://www.tiktok.com/@nasa/video/7665075736742530317", include_transcript: true)
+tag = su.tiktok_hashtag("nasa", max_videos: 10)
+results = su.tiktok_search("space telescope", max_results: 25)   # TikTok's own ranking
+comments = su.tiktok_comments("https://www.tiktok.com/@nasa/video/7665075736742530317", max_comments: 40)
+puts profile["stats"]["followers"], video["stats"]["plays"], tag["stats"]["views"], comments["totalComments"]
+```
+
+Profiles and hashtags list up to 10 videos in a couple of seconds from TikTok's server-rendered widget; ask for more (up to 200) and the real grid is scrolled in a browser session.
+
 ## Cookies and the serving proxy
 
 ```ruby
@@ -210,6 +223,25 @@ hotels = su.skyscanner.hotels(destination: "Madrid", checkin: "2026-09-01", chec
 cars = su.skyscanner.carhire(pickup: "Madrid", pickup_datetime: "2026-09-01T10:00", dropoff_datetime: "2026-09-03T10:00")
 ```
 
+## Southwest plugin
+
+Southwest Airlines flight quotes as JSON. `origin` and `dest` are IATA airport codes; omit `return_date` for a one-way search.
+
+```ruby
+flights = su.southwest.flights(
+  origin: "DAL", dest: "HOU",
+  depart_date: "2026-10-20", return_date: "2026-10-27"
+)
+
+# One-way, priced in Rapid Rewards points
+oneway = su.southwest.flights(
+  origin: "DAL", dest: "HOU",
+  depart_date: "2026-10-20", adults: 2, fare_type: "points"
+)
+```
+
+`adults` is 1-8 (default 1), `fare_type` is `"dollars"` (default) or `"points"`, `proxy_country` defaults to `"US"` and `max_attempts` is 1-5 (default 3). The call returns the raw booking / shopping JSON.
+
 ## Error handling
 
 Non-2xx responses raise typed errors, all subclasses of `ScrapeUnblocker::Error`.
@@ -268,7 +300,7 @@ end
 
 When more than one applies, the most serious wins: failed payment outranks credit limit, which outranks quota. All three lift by themselves once the billing state changes - access returns within about a minute, and the API key stays the same. One catch worth knowing: subscribing to a new plan does **not** clear `PaymentFailedError`, because the old unpaid invoice stays open until it is paid.
 
-Full details for every status code: [developers.scrapeunblocker.com/errors](https://developers.scrapeunblocker.com/errors).
+Full details for every status code: [docs.scrapeunblocker.com/errors](https://docs.scrapeunblocker.com/errors).
 
 ## Configuration
 
@@ -283,7 +315,7 @@ ScrapeUnblocker::Client.new(
 
 ## Links
 
-- Documentation: [developers.scrapeunblocker.com](https://developers.scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk)
+- Documentation: [docs.scrapeunblocker.com](https://docs.scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk)
 - Website: [scrapeunblocker.com](https://scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk)
 - Dashboard: [app.scrapeunblocker.com](https://app.scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk)
 

data/lib/scrapeunblocker/client.rb

@@ -7,6 +7,7 @@ require "json"
 require_relative "errors"
 require_relative "parsed_page"
 require_relative "skyscanner"
+require_relative "southwest"
 require_relative "version"
 
 module ScrapeUnblocker
@@ -22,6 +23,9 @@ module ScrapeUnblocker
     # @return [Skyscanner] the Skyscanner plugin endpoints
     attr_reader :skyscanner
 
+    # @return [Southwest] the Southwest Airlines plugin endpoints
+    attr_reader :southwest
+
     def initialize(api_key: nil, base_url: DEFAULT_BASE_URL, timeout: 180, max_retries: 2, transport: nil)
       @api_key = api_key || ENV["SCRAPEUNBLOCKER_KEY"]
       if @api_key.nil? || @api_key.empty?
@@ -34,6 +38,7 @@ module ScrapeUnblocker
       @max_retries = max_retries
       @transport = transport || method(:net_http_transport)
       @skyscanner = Skyscanner.new(self)
+      @southwest = Southwest.new(self)
     end
 
     # Fetch a URL and return the fully rendered HTML.
@@ -190,6 +195,53 @@ module ScrapeUnblocker
     end
 
     # @api private
+    # Scrape a public TikTok creator profile and its newest videos.
+    #
+    # Returns the exact follower, following, like and video counts, bio, bio
+    # link, verified / private / organization / seller flags, avatar and a
+    # +videos+ array of the creator's newest posts (up to 200), each in the
+    # full #tiktok_video shape. No login.
+    def tiktok_profile(username, max_videos: 10, video_details: true, proxy_country: nil)
+      post_json("/social/tiktok-profile",
+                username: username, max_videos: max_videos,
+                video_details: video_details ? nil : false,
+                proxy_country: proxy_country)
+    end
+
+    # Scrape one TikTok video or photo post: exact plays, likes, comments,
+    # shares, saves and reposts, hashtags, mentions, author, music, play /
+    # download URLs, subtitle tracks and, with +include_transcript+, the
+    # transcript as text.
+    def tiktok_video(url, include_transcript: false, transcript_language: nil, proxy_country: nil)
+      post_json("/social/tiktok-video",
+                url: url, include_transcript: include_transcript ? true : nil,
+                transcript_language: transcript_language, proxy_country: proxy_country)
+    end
+
+    # Scrape a TikTok hashtag: total views and videos plus its videos (up to 200).
+    def tiktok_hashtag(hashtag, max_videos: 10, video_details: true, proxy_country: nil)
+      post_json("/social/tiktok-hashtag",
+                hashtag: hashtag, max_videos: max_videos,
+                video_details: video_details ? nil : false,
+                proxy_country: proxy_country)
+    end
+
+    # Search TikTok videos by keyword. Runs in a browser session that clears
+    # TikTok's captcha, so +results+ carry TikTok's own ranking (region via
+    # +proxy_country+), each in the full #tiktok_video shape. 20-45 s.
+    def tiktok_search(query, max_results: 20, proxy_country: nil)
+      post_json("/social/tiktok-search",
+                query: query, max_results: max_results, proxy_country: proxy_country)
+    end
+
+    # Scrape the comments of a TikTok post (text, date, likes, reply count,
+    # author, creator flags, preloaded replies) plus +totalComments+ and
+    # +hasMore+. Runs in a browser session that clears TikTok's captcha. 20-45 s.
+    def tiktok_comments(url, max_comments: 50, proxy_country: nil)
+      post_json("/social/tiktok-comments",
+                url: url, max_comments: max_comments, proxy_country: proxy_country)
+    end
+
     def post_json(path, params)
       JSON.parse(request(path, params)[:body])
     end

data/lib/scrapeunblocker/southwest.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+module ScrapeUnblocker
+  # Southwest Airlines plugin endpoints (flights).
+  class Southwest
+    # @api private
+    def initialize(client)
+      @client = client
+    end
+
+    # Fetch Southwest Airlines flight quotes and return the raw booking JSON.
+    #
+    # +origin+ and +dest+ are IATA airport codes (e.g. "DAL", "HOU").
+    # +depart_date+ (and the optional +return_date+) are "YYYY-MM-DD"; omit
+    # +return_date+ for a one-way search. +adults+ is 1-8. +fare_type+ is
+    # "dollars" (default) or "points". +proxy_country+ pins the exit (default
+    # "US"). +max_attempts+ is 1-5.
+    def flights(origin:, dest:, depart_date:, return_date: nil, adults: 1,
+                fare_type: "dollars", proxy_country: "US", max_attempts: 3)
+      @client.post_json("/flights/southwest-quotes",
+                        origin: origin, dest: dest, depart_date: depart_date,
+                        return_date: return_date, adults: adults,
+                        fare_type: fare_type, proxy_country: proxy_country,
+                        max_attempts: max_attempts)
+    end
+  end
+end

data/lib/scrapeunblocker/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module ScrapeUnblocker
-  VERSION = "0.2.1"
+  VERSION = "0.4.0"
 end

data/lib/scrapeunblocker.rb

@@ -4,6 +4,7 @@ require_relative "scrapeunblocker/version"
 require_relative "scrapeunblocker/errors"
 require_relative "scrapeunblocker/parsed_page"
 require_relative "scrapeunblocker/skyscanner"
+require_relative "scrapeunblocker/southwest"
 require_relative "scrapeunblocker/client"
 
 # Official Ruby client for the ScrapeUnblocker web scraping API.

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: scrapeunblocker
 version: !ruby/object:Gem::Version
-  version: 0.2.1
+  version: 0.4.0
 platform: ruby
 authors:
 - ScrapeUnblocker
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-09-02 00:00:00.000000000 Z
+date: 2026-09-16 00:00:00.000000000 Z
 dependencies: []
 description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
   plus Google SERP and Skyscanner flights/hotels/car-hire scraping as JSON.
@@ -26,6 +26,7 @@ files:
 - lib/scrapeunblocker/errors.rb
 - lib/scrapeunblocker/parsed_page.rb
 - lib/scrapeunblocker/skyscanner.rb
+- lib/scrapeunblocker/southwest.rb
 - lib/scrapeunblocker/version.rb
 homepage: https://scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk
 licenses:
@@ -33,7 +34,7 @@ licenses:
 metadata:
   homepage_uri: https://scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk
   source_code_uri: https://github.com/ScrapeUnblocker/scrapeunblocker-ruby
-  documentation_uri: https://developers.scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk
+  documentation_uri: https://docs.scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk
   changelog_uri: https://github.com/ScrapeUnblocker/scrapeunblocker-ruby/blob/main/CHANGELOG.md
   bug_tracker_uri: https://github.com/ScrapeUnblocker/scrapeunblocker-ruby/issues
   rubygems_mfa_required: 'true'