scrapeunblocker 0.2.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +15 -2
- data/lib/scrapeunblocker/client.rb +47 -0
- data/lib/scrapeunblocker/version.rb +1 -1
- metadata +3 -3
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 4542500ec21bf7ce42e644394bf9302b74399a5447f6f5717e3f6abeaf8e7cb4
|
|
4
|
+
data.tar.gz: 92470e75374896eb308d3f7f939f39dcecb210aeaff4fa151aa8b0df8c516a95
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 672b0d182dc35e5650102171465dadf60f6bd22890a3d000b6e0a82a5e9b137b8857d1fac1231013af24eefd4b782e883db83423fca436b47aec1183af4c6b45
|
|
7
|
+
data.tar.gz: 75f9249d01b7d1b6aa6e2ef0c373ec386cee41d5cd7e42fefceff65aa74d4992aaa036067e67f24be0e11e2ee89f705942975a863c8f0dba707cc5bf4f01b1a3
|
data/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,9 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.3.0 (2026-09-08)
|
|
4
|
+
|
|
5
|
+
- Added `tiktok_profile`, `tiktok_video`, `tiktok_hashtag`, `tiktok_search` and `tiktok_comments` for the new TikTok plugin: a creator's exact follower / like / video counts with their newest videos (up to 200), any video or photo post with exact plays, likes, comments, shares, saves and reposts, hashtags, music, play / download URLs, subtitle tracks and an optional transcript, a hashtag's total views and videos with its videos, keyword search in TikTok's own ranking, and the comments of any post. No login.
|
|
6
|
+
|
|
3
7
|
## 0.2.1 (2026-09-02)
|
|
4
8
|
|
|
5
9
|
- Added `meta_ad_library(advertiser, ...)` for the new Meta Ad Library plugin (`POST /ads/meta-ad-library`) - returns an advertiser's Meta (Facebook) Ad Library ads as a Hash. `advertiser` is required; optional filters `country`, `active_status`, `media_type` and `max_ads` are dropped when unset so the API applies its own defaults.
|
data/README.md
CHANGED
|
@@ -178,6 +178,19 @@ end
|
|
|
178
178
|
|
|
179
179
|
`marketplace` is any of the 19 regional eBay hosts (`ebay.com` default). `condition` is one of `"new"`, `"open_box"`, `"refurbished"`, `"used"` or `"for_parts"`; `sort` is one of `"best_match"` (default), `"newly_listed"`, `"ending_soon"`, `"price_asc"` or `"price_desc"`; `page_size` is 60, 120 or 240. `exactMatches` is `false` when eBay found nothing for the keyword and answered with its own loosely-related suggestions instead, so check it before using the listings.
|
|
180
180
|
|
|
181
|
+
### TikTok
|
|
182
|
+
|
|
183
|
+
```ruby
|
|
184
|
+
profile = su.tiktok_profile("nasa", max_videos: 5) # exact stats + newest videos
|
|
185
|
+
video = su.tiktok_video("https://www.tiktok.com/@nasa/video/7665075736742530317", include_transcript: true)
|
|
186
|
+
tag = su.tiktok_hashtag("nasa", max_videos: 10)
|
|
187
|
+
results = su.tiktok_search("space telescope", max_results: 25) # TikTok's own ranking
|
|
188
|
+
comments = su.tiktok_comments("https://www.tiktok.com/@nasa/video/7665075736742530317", max_comments: 40)
|
|
189
|
+
puts profile["stats"]["followers"], video["stats"]["plays"], tag["stats"]["views"], comments["totalComments"]
|
|
190
|
+
```
|
|
191
|
+
|
|
192
|
+
Profiles and hashtags list up to 10 videos in a couple of seconds from TikTok's server-rendered widget; ask for more (up to 200) and the real grid is scrolled in a browser session.
|
|
193
|
+
|
|
181
194
|
## Cookies and the serving proxy
|
|
182
195
|
|
|
183
196
|
```ruby
|
|
@@ -268,7 +281,7 @@ end
|
|
|
268
281
|
|
|
269
282
|
When more than one applies, the most serious wins: failed payment outranks credit limit, which outranks quota. All three lift by themselves once the billing state changes - access returns within about a minute, and the API key stays the same. One catch worth knowing: subscribing to a new plan does **not** clear `PaymentFailedError`, because the old unpaid invoice stays open until it is paid.
|
|
270
283
|
|
|
271
|
-
Full details for every status code: [
|
|
284
|
+
Full details for every status code: [docs.scrapeunblocker.com/errors](https://docs.scrapeunblocker.com/errors).
|
|
272
285
|
|
|
273
286
|
## Configuration
|
|
274
287
|
|
|
@@ -283,7 +296,7 @@ ScrapeUnblocker::Client.new(
|
|
|
283
296
|
|
|
284
297
|
## Links
|
|
285
298
|
|
|
286
|
-
- Documentation: [
|
|
299
|
+
- Documentation: [docs.scrapeunblocker.com](https://docs.scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk)
|
|
287
300
|
- Website: [scrapeunblocker.com](https://scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk)
|
|
288
301
|
- Dashboard: [app.scrapeunblocker.com](https://app.scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk)
|
|
289
302
|
|
|
@@ -190,6 +190,53 @@ module ScrapeUnblocker
|
|
|
190
190
|
end
|
|
191
191
|
|
|
192
192
|
# @api private
|
|
193
|
+
# Scrape a public TikTok creator profile and its newest videos.
|
|
194
|
+
#
|
|
195
|
+
# Returns the exact follower, following, like and video counts, bio, bio
|
|
196
|
+
# link, verified / private / organization / seller flags, avatar and a
|
|
197
|
+
# +videos+ array of the creator's newest posts (up to 200), each in the
|
|
198
|
+
# full #tiktok_video shape. No login.
|
|
199
|
+
def tiktok_profile(username, max_videos: 10, video_details: true, proxy_country: nil)
|
|
200
|
+
post_json("/social/tiktok-profile",
|
|
201
|
+
username: username, max_videos: max_videos,
|
|
202
|
+
video_details: video_details ? nil : false,
|
|
203
|
+
proxy_country: proxy_country)
|
|
204
|
+
end
|
|
205
|
+
|
|
206
|
+
# Scrape one TikTok video or photo post: exact plays, likes, comments,
|
|
207
|
+
# shares, saves and reposts, hashtags, mentions, author, music, play /
|
|
208
|
+
# download URLs, subtitle tracks and, with +include_transcript+, the
|
|
209
|
+
# transcript as text.
|
|
210
|
+
def tiktok_video(url, include_transcript: false, transcript_language: nil, proxy_country: nil)
|
|
211
|
+
post_json("/social/tiktok-video",
|
|
212
|
+
url: url, include_transcript: include_transcript ? true : nil,
|
|
213
|
+
transcript_language: transcript_language, proxy_country: proxy_country)
|
|
214
|
+
end
|
|
215
|
+
|
|
216
|
+
# Scrape a TikTok hashtag: total views and videos plus its videos (up to 200).
|
|
217
|
+
def tiktok_hashtag(hashtag, max_videos: 10, video_details: true, proxy_country: nil)
|
|
218
|
+
post_json("/social/tiktok-hashtag",
|
|
219
|
+
hashtag: hashtag, max_videos: max_videos,
|
|
220
|
+
video_details: video_details ? nil : false,
|
|
221
|
+
proxy_country: proxy_country)
|
|
222
|
+
end
|
|
223
|
+
|
|
224
|
+
# Search TikTok videos by keyword. Runs in a browser session that clears
|
|
225
|
+
# TikTok's captcha, so +results+ carry TikTok's own ranking (region via
|
|
226
|
+
# +proxy_country+), each in the full #tiktok_video shape. 20-45 s.
|
|
227
|
+
def tiktok_search(query, max_results: 20, proxy_country: nil)
|
|
228
|
+
post_json("/social/tiktok-search",
|
|
229
|
+
query: query, max_results: max_results, proxy_country: proxy_country)
|
|
230
|
+
end
|
|
231
|
+
|
|
232
|
+
# Scrape the comments of a TikTok post (text, date, likes, reply count,
|
|
233
|
+
# author, creator flags, preloaded replies) plus +totalComments+ and
|
|
234
|
+
# +hasMore+. Runs in a browser session that clears TikTok's captcha. 20-45 s.
|
|
235
|
+
def tiktok_comments(url, max_comments: 50, proxy_country: nil)
|
|
236
|
+
post_json("/social/tiktok-comments",
|
|
237
|
+
url: url, max_comments: max_comments, proxy_country: proxy_country)
|
|
238
|
+
end
|
|
239
|
+
|
|
193
240
|
def post_json(path, params)
|
|
194
241
|
JSON.parse(request(path, params)[:body])
|
|
195
242
|
end
|
metadata
CHANGED
|
@@ -1,14 +1,14 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: scrapeunblocker
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- ScrapeUnblocker
|
|
8
8
|
autorequire:
|
|
9
9
|
bindir: bin
|
|
10
10
|
cert_chain: []
|
|
11
|
-
date: 2026-09-
|
|
11
|
+
date: 2026-09-08 00:00:00.000000000 Z
|
|
12
12
|
dependencies: []
|
|
13
13
|
description: JS-rendered pages that bypass Cloudflare, DataDome, PerimeterX and Akamai,
|
|
14
14
|
plus Google SERP and Skyscanner flights/hotels/car-hire scraping as JSON.
|
|
@@ -33,7 +33,7 @@ licenses:
|
|
|
33
33
|
metadata:
|
|
34
34
|
homepage_uri: https://scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk
|
|
35
35
|
source_code_uri: https://github.com/ScrapeUnblocker/scrapeunblocker-ruby
|
|
36
|
-
documentation_uri: https://
|
|
36
|
+
documentation_uri: https://docs.scrapeunblocker.com?utm_source=rubygems&utm_medium=integration&utm_campaign=ruby-sdk
|
|
37
37
|
changelog_uri: https://github.com/ScrapeUnblocker/scrapeunblocker-ruby/blob/main/CHANGELOG.md
|
|
38
38
|
bug_tracker_uri: https://github.com/ScrapeUnblocker/scrapeunblocker-ruby/issues
|
|
39
39
|
rubygems_mfa_required: 'true'
|