pinterest-dl 1.0.1 → 2.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +40 -0
- data/LICENSE +13 -0
- data/README.md +187 -57
- data/bin/pinterest-dl +107 -0
- data/lib/pinterest-dl.rb +17 -135
- data/lib/pinterest_dl/api.rb +84 -0
- data/lib/pinterest_dl/board.rb +106 -0
- data/lib/pinterest_dl/client.rb +111 -0
- data/lib/pinterest_dl/configuration.rb +46 -0
- data/lib/pinterest_dl/downloader.rb +57 -0
- data/lib/pinterest_dl/errors.rb +49 -0
- data/lib/pinterest_dl/extractors/image.rb +47 -0
- data/lib/pinterest_dl/extractors/video.rb +61 -0
- data/lib/pinterest_dl/progress_bar.rb +21 -0
- data/lib/pinterest_dl/search.rb +142 -0
- data/lib/pinterest_dl/version.rb +6 -0
- metadata +28 -8
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 6c0b3841a1ae58f40462a3b5e3545a7f72c8e3e99457e34a046a78a06598ad5c
|
|
4
|
+
data.tar.gz: 220569a87bab3bfe8776169a97cf373278402a8ad49190deb761e749010f5798
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: aa67c66ab2f7bbfb5c58de173ec73b1b077ea29bdb316623f1a2d9abf4a6f3a2c26388127a167a4955fa0f5ac63dbb7705963cad9d4bbf018aaa70ec24e53cad
|
|
7
|
+
data.tar.gz: 0ea15400b28788771709772a9d5058ca392231a2c27270b310ee634847b075e209531182bfb5eec1873c4d4b058d6b96567a8bf0503a7e533c1b6df4d6dc6972
|
data/CHANGELOG.md
ADDED
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
All notable changes to this project are documented here.
|
|
4
|
+
This project follows [Semantic Versioning](https://semver.org/).
|
|
5
|
+
|
|
6
|
+
## [2.0.0]
|
|
7
|
+
|
|
8
|
+
### Added
|
|
9
|
+
- `PinterestDL.download` / `PinterestDL.download_batch` for real file downloads,
|
|
10
|
+
the latter with per-file progress callbacks and a `PinterestDL::ProgressBar` helper.
|
|
11
|
+
- `PinterestDL.board` for fetching pins from a board or board section, with pagination.
|
|
12
|
+
- `PinterestDL.configure` for global settings: cookies/session, user agent,
|
|
13
|
+
timeouts, retry count, request rate limiting, and default image quality.
|
|
14
|
+
- `PinterestDL.get_media` to fetch image + video URLs for a pin in one request.
|
|
15
|
+
- Pagination support in `PinterestDL.search` (`pages:` option).
|
|
16
|
+
- `creator` field on pin metadata returned by `search` and `board`.
|
|
17
|
+
- Meaningful exception hierarchy under `PinterestDL::Error` (`NetworkError`,
|
|
18
|
+
`NotFoundError`, `RateLimitError`, `AuthenticationError`, `InvalidURLError`,
|
|
19
|
+
`DownloadError`) replacing silent `nil` returns and `puts`-based error reporting.
|
|
20
|
+
- Real CLI: `pinterest-dl image|video|search|board ...`.
|
|
21
|
+
- Configurable timeouts and retry/backoff on all HTTP requests.
|
|
22
|
+
- Small in-memory response cache and optional request rate limiting.
|
|
23
|
+
- Test suite (Minitest) and GitHub Actions CI matrix (Ruby 3.0–3.3).
|
|
24
|
+
- RuboCop configuration.
|
|
25
|
+
|
|
26
|
+
### Changed
|
|
27
|
+
- User-Agent updated to a current Chrome UA string.
|
|
28
|
+
- All `puts` debug/status output removed from library code in favor of a
|
|
29
|
+
configurable `Logger` (`PinterestDL.config.logger`).
|
|
30
|
+
- Internals reorganized into `lib/pinterest_dl/{client,configuration,errors,
|
|
31
|
+
extractors,search,board,downloader,progress_bar,api}.rb`; `PinterestDL::VERSION`
|
|
32
|
+
is now the single source of truth for the gem version.
|
|
33
|
+
|
|
34
|
+
### Fixed
|
|
35
|
+
- The gem no longer prints anythign on `require`.
|
|
36
|
+
|
|
37
|
+
## [1.0.3]
|
|
38
|
+
|
|
39
|
+
- Initial public release: `get_image_url`, `get_video_url`, `search`,
|
|
40
|
+
`search_json`, `search_images`, `search_videos`.
|
data/LICENSE
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
|
|
2
|
+
Version 2, December 2004
|
|
3
|
+
|
|
4
|
+
Copyright (C) 2004 Sam Hocevar <sam@hocevar.net>
|
|
5
|
+
|
|
6
|
+
Everyone is permitted to copy and distribute verbatim or modified
|
|
7
|
+
copies of this license document, and changing it is allowed as long
|
|
8
|
+
as the name is changed.
|
|
9
|
+
|
|
10
|
+
DO WHAT THE FUCK YOU WANT TO PUBLIC LICENSE
|
|
11
|
+
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION
|
|
12
|
+
|
|
13
|
+
0. You just DO WHAT THE FUCK YOU WANT TO.
|
data/README.md
CHANGED
|
@@ -1,57 +1,187 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
[](https://rubygems.org/gems/pinterest-dl)
|
|
6
|
+
[](https://rubygems.org/gems/pinterest-dl)
|
|
7
|
+
[](https://github.com/monji024/pinterest-dl/actions/workflows/ci.yml)
|
|
8
|
+
[](LICENSE)
|
|
9
|
+
[](https://www.ruby-lang.org/)
|
|
10
|
+
<img src="https://img.icons8.com/color/96/000000/pinterest--v1.png" width="80">
|
|
11
|
+
|
|
12
|
+
---
|
|
13
|
+
|
|
14
|
+
> **Note:** Users in some regions (e.g. Iran) may need a fu*king VPN to reach Pinterest.
|
|
15
|
+
> (Yeah, because even in 2026 Pinterest still gives you the middle finger on clean APIs.)
|
|
16
|
+
|
|
17
|
+
Pinterest doesn't give you a clean, official API for getting a direct media URL out of a pin, walking a board, or downloading a batch of images. `pinterest-dl` does that: point it at a pin, a search query, or a board, and get back real `i.pinimg.com` / `v.pinimg.com` URLs — or the fucking files themselves, already saved to disk.
|
|
18
|
+
|
|
19
|
+
## What it does
|
|
20
|
+
|
|
21
|
+
- Pulls the **direct image or video URL** out of a single pin page.
|
|
22
|
+
- **Searches** Pinterest and returns pin metadata (id, url, title, description, creator, image/video URL), with pagination.
|
|
23
|
+
- Fetches all pins from a **board** (or a specific board section).
|
|
24
|
+
- **Downloads** a URL — or a whole batch of them — straight to disk, with progress reporting and per-file error handling.
|
|
25
|
+
- Supports **authenticated requests** via cookies, for boards/pins that require a logged-in session.
|
|
26
|
+
- Raises **specific, catchable exceptions** (`NotFoundError`, `RateLimitError`, `NetworkError`, ...) instead of returning `nil` or printing to stdout.
|
|
27
|
+
- Ships a **CLI** (`pinterest-dl`) for one-off use from the shell.
|
|
28
|
+
|
|
29
|
+
## Installation
|
|
30
|
+
|
|
31
|
+
```bash
|
|
32
|
+
gem install pinterest-dl
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
Or add it to your Gemfile:
|
|
36
|
+
|
|
37
|
+
```ruby
|
|
38
|
+
gem 'pinterest-dl'
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
## Quick start
|
|
42
|
+
|
|
43
|
+
```ruby
|
|
44
|
+
require 'pinterest-dl'
|
|
45
|
+
# Direct image URL
|
|
46
|
+
PinterestDL.get_image_url('https://www.pinterest.com/pin/1234567890123456789/') # => "https://i.pinimg.com/originals/aa/bb/cc/aabbcc...jpg"
|
|
47
|
+
# Direct video URL (raises PinterestDL::NotFoundError if the pin has no video)
|
|
48
|
+
PinterestDL.get_video_url('https://www.pinterest.com/pin/1234567890123456789/') # => "https://v.pinimg.com/videos/mc/720p/aa/bb/cc/aabbcc....mp4"
|
|
49
|
+
# Download it
|
|
50
|
+
PinterestDL.download(
|
|
51
|
+
PinterestDL.get_image_url('https://www.pinterest.com/pin/1234567890123456789/'),
|
|
52
|
+
path: 'downloads/pin.jpg'
|
|
53
|
+
)
|
|
54
|
+
```
|
|
55
|
+
|
|
56
|
+
## Usage
|
|
57
|
+
|
|
58
|
+
### Get image / video URLs
|
|
59
|
+
|
|
60
|
+
```ruby
|
|
61
|
+
url = PinterestDL.get_image_url('https://www.pinterest.com/pin/example/') # Pick a specific size bucket
|
|
62
|
+
url = PinterestDL.get_image_url('https://www.pinterest.com/pin/example/', quality: :"474x") # quality can be :originals, :"736x", :"474x", :"236x"
|
|
63
|
+
url = PinterestDL.get_video_url('https://www.pinterest.com/pin/example/')
|
|
64
|
+
# Both at once
|
|
65
|
+
media = PinterestDL.get_media('https://www.pinterest.com/pin/example/') # => { image_url: "https://i.pinimg.com/...", video_url: nil }
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
### Search
|
|
69
|
+
|
|
70
|
+
```ruby
|
|
71
|
+
# Search for pins with a specific query
|
|
72
|
+
results = PinterestDL.search('nature landscape', limit: 10) # => [{ id: "...", pin_url: "...", title: "...", description: "...", creator: "...", image_url: "...", video_url: nil }, ...]
|
|
73
|
+
# Fetch more than one page of results (25 pins/page from Pinterest)
|
|
74
|
+
results = PinterestDL.search('nature landscape', limit: 100, pages: 10)
|
|
75
|
+
# Get only image URLs from search results
|
|
76
|
+
images = PinterestDL.search_images('sunset', limit: 5) # => ["https://i.pinimg.com/originals/...", ...]
|
|
77
|
+
# Get only video URLs from search results
|
|
78
|
+
videos = PinterestDL.search_videos('tutorial', limit: 3) # => ["https://v.pinimg.com/videos/...", ...]
|
|
79
|
+
# Get search results as a JSON string
|
|
80
|
+
json_results = PinterestDL.search_json('anime art', limit: 20)
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
### Boards & sections
|
|
84
|
+
|
|
85
|
+
```ruby
|
|
86
|
+
# All pins on a board
|
|
87
|
+
pins = PinterestDL.board('https://www.pinterest.com/username/board-name/', limit: 50)
|
|
88
|
+
# A specific section within a board
|
|
89
|
+
pins = PinterestDL.board('https://www.pinterest.com/username/board-name/section-slug/', limit: 50)
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### Downloading
|
|
93
|
+
|
|
94
|
+
```ruby
|
|
95
|
+
# Single file
|
|
96
|
+
PinterestDL.download('https://i.pinimg.com/originals/aa/bb/cc/img.jpg', path: 'out/img.jpg')
|
|
97
|
+
# Batch download (returns { succeeded: [...], failed: [{ url:, error: }, ...] })
|
|
98
|
+
urls = PinterestDL.search_images('mountains', limit: 20)
|
|
99
|
+
result = PinterestDL.download_batch(urls, dir: './downloads')
|
|
100
|
+
# With progress reporting
|
|
101
|
+
bar = PinterestDL::ProgressBar.new(total: urls.size)
|
|
102
|
+
PinterestDL.download_batch(urls, dir: './downloads') do |done, total, url|
|
|
103
|
+
bar.update(done)
|
|
104
|
+
end
|
|
105
|
+
```
|
|
106
|
+
|
|
107
|
+
### Cookies / authenticated sessions
|
|
108
|
+
|
|
109
|
+
Some boards and pins require a logged-in session to view. Pass your browser's Pinterest cookies through global configuration:
|
|
110
|
+
|
|
111
|
+
```ruby
|
|
112
|
+
PinterestDL.configure do |c|
|
|
113
|
+
c.cookies = 'sessionid=...; csrftoken=...' # yeah, steal your own fucking session if you have to
|
|
114
|
+
c.user_agent = 'Mozilla/5.0 ...' # optional override
|
|
115
|
+
end
|
|
116
|
+
```
|
|
117
|
+
|
|
118
|
+
### Other configuration
|
|
119
|
+
|
|
120
|
+
```ruby
|
|
121
|
+
PinterestDL.configure do |c|
|
|
122
|
+
c.open_timeout = 10 # seconds, connection open timeout
|
|
123
|
+
c.read_timeout = 20 # seconds, response read timeout
|
|
124
|
+
c.max_retries = 3 # retries on transient network failures
|
|
125
|
+
c.retry_wait = 1.0 # base seconds between retries (backs off linearly)
|
|
126
|
+
c.rate_limit_interval = 0.5 # minimum seconds between outgoing requests
|
|
127
|
+
c.quality = :originals # default image quality for get_image_url
|
|
128
|
+
c.logger.level = Logger::INFO # library uses this Logger instead of puts
|
|
129
|
+
end
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
### Error handling
|
|
133
|
+
|
|
134
|
+
Every method raises a subclass of `PinterestDL::Error` instead of returning `nil` or printing to the console:
|
|
135
|
+
|
|
136
|
+
```ruby
|
|
137
|
+
begin
|
|
138
|
+
PinterestDL.get_image_url('https://www.pinterest.com/pin/does-not-exist/')
|
|
139
|
+
rescue PinterestDL::NotFoundError
|
|
140
|
+
# pin/board/search had nothing to extract
|
|
141
|
+
rescue PinterestDL::RateLimitError => e
|
|
142
|
+
# Pinterest throttled us; e.retry_after may hold a hint
|
|
143
|
+
rescue PinterestDL::AuthenticationError
|
|
144
|
+
# the resource needs c.cookies to be set
|
|
145
|
+
rescue PinterestDL::NetworkError => e
|
|
146
|
+
# DNS/timeout/connection-level failure; e.cause_error has the original exception
|
|
147
|
+
rescue PinterestDL::InvalidURLError
|
|
148
|
+
# the given string isn't a Pinterest pin/board URL
|
|
149
|
+
rescue PinterestDL::Error
|
|
150
|
+
# catch-all for anything above
|
|
151
|
+
end
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
### CLI
|
|
155
|
+
|
|
156
|
+
```bash
|
|
157
|
+
pinterest-dl image https://www.pinterest.com/pin/example/
|
|
158
|
+
pinterest-dl image https://www.pinterest.com/pin/example/ -o pin.jpg
|
|
159
|
+
pinterest-dl video https://www.pinterest.com/pin/example/ -o pin.mp4
|
|
160
|
+
pinterest-dl search "cyberpunk city" -n 20
|
|
161
|
+
pinterest-dl search "cyberpunk city" -n 20 -o ./downloads
|
|
162
|
+
pinterest-dl board https://www.pinterest.com/username/board-name/ -n 50 -o ./downloads
|
|
163
|
+
pinterest-dl --help
|
|
164
|
+
```
|
|
165
|
+
|
|
166
|
+
Flags: `-o/--output`, `-n/--limit`, `-q/--quality`, `-c/--cookies`, `-v/--verbose`.
|
|
167
|
+
|
|
168
|
+
|
|
169
|
+
## Development
|
|
170
|
+
|
|
171
|
+
```bash
|
|
172
|
+
bundle install
|
|
173
|
+
bundle exec rake test # run the test suite
|
|
174
|
+
bundle exec rake rubocop # lint
|
|
175
|
+
bundle exec rake # both (default task)
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
## Links
|
|
179
|
+
|
|
180
|
+
- [RubyGems](https://rubygems.org/gems/pinterest-dl)
|
|
181
|
+
- [GitHub](https://github.com/monji024/pinterest-dl)
|
|
182
|
+
- [Changelog](CHANGELOG.md)
|
|
183
|
+
- [Issue tracker](https://github.com/monji024/pinterest-dl/issues)
|
|
184
|
+
|
|
185
|
+
## License
|
|
186
|
+
|
|
187
|
+
WTFPL © [monji024](https://github.com/monji024)
|
data/bin/pinterest-dl
ADDED
|
@@ -0,0 +1,107 @@
|
|
|
1
|
+
#!/usr/bin/env ruby
|
|
2
|
+
# Monji024
|
|
3
|
+
require_relative '../lib/pinterest-dl'
|
|
4
|
+
require 'optparse'
|
|
5
|
+
|
|
6
|
+
def usage
|
|
7
|
+
<<~USAGE
|
|
8
|
+
pinterest-dl v#{PinterestDL::VERSION}
|
|
9
|
+
|
|
10
|
+
Usage:
|
|
11
|
+
pinterest-dl image URL [-o FILE]
|
|
12
|
+
pinterest-dl video URL [-o FILE]
|
|
13
|
+
pinterest-dl search QUERY [-n LIMIT] [-o DIR]
|
|
14
|
+
pinterest-dl board URL [-n LIMIT] [-o DIR]
|
|
15
|
+
|
|
16
|
+
Options:
|
|
17
|
+
-o, --output PATH File (image/video) or directory (search/board) to save to
|
|
18
|
+
-n, --limit N Max number of results (search/board), default 25
|
|
19
|
+
-q, --quality Q Image quality: originals, 736x, 474x, 236x (default originals)
|
|
20
|
+
-c, --cookies STR Cookie string for authenticated requests
|
|
21
|
+
-v, --verbose Print debug logging
|
|
22
|
+
-h, --help Show this help
|
|
23
|
+
USAGE
|
|
24
|
+
end
|
|
25
|
+
|
|
26
|
+
options = { limit: 25, quality: :originals }
|
|
27
|
+
parser = OptionParser.new do |opts|
|
|
28
|
+
opts.on('-o PATH', '--output PATH') { |v| options[:output] = v }
|
|
29
|
+
opts.on('-n N', '--limit N', Integer) { |v| options[:limit] = v }
|
|
30
|
+
opts.on('-q Q', '--quality Q') { |v| options[:quality] = v.to_sym }
|
|
31
|
+
opts.on('-c STR', '--cookies STR') { |v| options[:cookies] = v }
|
|
32
|
+
opts.on('-v', '--verbose') { options[:verbose] = true }
|
|
33
|
+
opts.on('-h', '--help') do
|
|
34
|
+
puts usage
|
|
35
|
+
exit 0
|
|
36
|
+
end
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
begin
|
|
40
|
+
parser.parse!(ARGV)
|
|
41
|
+
rescue OptionParser::ParseError => e
|
|
42
|
+
warn "pinterest-dl: #{e.message}"
|
|
43
|
+
exit 1
|
|
44
|
+
end
|
|
45
|
+
|
|
46
|
+
command = ARGV.shift
|
|
47
|
+
target = ARGV.shift
|
|
48
|
+
|
|
49
|
+
if command.nil? || target.nil?
|
|
50
|
+
puts usage
|
|
51
|
+
exit(command.nil? ? 0 : 1)
|
|
52
|
+
end
|
|
53
|
+
|
|
54
|
+
PinterestDL.configure do |c|
|
|
55
|
+
c.cookies = options[:cookies] if options[:cookies]
|
|
56
|
+
c.logger.level = Logger::DEBUG if options[:verbose]
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
def die(message)
|
|
60
|
+
warn "pinterest-dl: #{message}"
|
|
61
|
+
exit 1
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
begin
|
|
65
|
+
case command
|
|
66
|
+
when 'image'
|
|
67
|
+
url = PinterestDL.get_image_url(target, quality: options[:quality])
|
|
68
|
+
if options[:output]
|
|
69
|
+
PinterestDL.download(url, path: options[:output])
|
|
70
|
+
puts "Saved to #{options[:output]}"
|
|
71
|
+
else
|
|
72
|
+
puts url
|
|
73
|
+
end
|
|
74
|
+
when 'video'
|
|
75
|
+
url = PinterestDL.get_video_url(target)
|
|
76
|
+
if options[:output]
|
|
77
|
+
PinterestDL.download(url, path: options[:output])
|
|
78
|
+
puts "Saved to #{options[:output]}"
|
|
79
|
+
else
|
|
80
|
+
puts url
|
|
81
|
+
end
|
|
82
|
+
when 'search'
|
|
83
|
+
pins = PinterestDL.search(target, limit: options[:limit])
|
|
84
|
+
if options[:output]
|
|
85
|
+
urls = pins.filter_map { |p| p[:image_url] }
|
|
86
|
+
bar = PinterestDL::ProgressBar.new(total: urls.size)
|
|
87
|
+
result = PinterestDL.download_batch(urls, dir: options[:output]) { |done, total, _| bar.update(done) }
|
|
88
|
+
puts "Downloaded #{result[:succeeded].size}/#{urls.size} images to #{options[:output]}"
|
|
89
|
+
else
|
|
90
|
+
puts JSON.pretty_generate(pins)
|
|
91
|
+
end
|
|
92
|
+
when 'board'
|
|
93
|
+
pins = PinterestDL.board(target, limit: options[:limit])
|
|
94
|
+
if options[:output]
|
|
95
|
+
urls = pins.filter_map { |p| p[:image_url] }
|
|
96
|
+
bar = PinterestDL::ProgressBar.new(total: urls.size)
|
|
97
|
+
result = PinterestDL.download_batch(urls, dir: options[:output]) { |done, total, _| bar.update(done) }
|
|
98
|
+
puts "Downloaded #{result[:succeeded].size}/#{urls.size} images to #{options[:output]}"
|
|
99
|
+
else
|
|
100
|
+
puts JSON.pretty_generate(pins)
|
|
101
|
+
end
|
|
102
|
+
else
|
|
103
|
+
die "unknown command '#{command}'\n\n#{usage}"
|
|
104
|
+
end
|
|
105
|
+
rescue PinterestDL::Error => e
|
|
106
|
+
die e.message
|
|
107
|
+
end
|
data/lib/pinterest-dl.rb
CHANGED
|
@@ -1,135 +1,17 @@
|
|
|
1
|
-
|
|
2
|
-
|
|
3
|
-
#
|
|
4
|
-
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
|
|
9
|
-
|
|
10
|
-
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
puts "pinterest-dl v#{ver} (#{github})\n\n"
|
|
19
|
-
|
|
20
|
-
USER_AGENT = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.124 Safari/537.36'.freeze
|
|
21
|
-
|
|
22
|
-
class << self
|
|
23
|
-
def get_image_url(pin_url)
|
|
24
|
-
html = fetch_html(pin_url)
|
|
25
|
-
return nil unless html
|
|
26
|
-
|
|
27
|
-
# 1 -> json-ld
|
|
28
|
-
url = extract_json_ld(html, 'ImageObject')
|
|
29
|
-
return url if url
|
|
30
|
-
|
|
31
|
-
# 2 -> open graph
|
|
32
|
-
match = html.match(%r{<meta[^>]*property="og:image"[^>]*content="([^"]+)"})
|
|
33
|
-
if match
|
|
34
|
-
url = match[1]
|
|
35
|
-
puts "image url: #{url}"
|
|
36
|
-
return url
|
|
37
|
-
end
|
|
38
|
-
|
|
39
|
-
# 3 -> direct i.pinimg.com patterns
|
|
40
|
-
match = html.match(%r{https://i\.pinimg\.com/(?:originals|474x|236x|736x)/[^"\s]+})
|
|
41
|
-
if match
|
|
42
|
-
url = match[0]
|
|
43
|
-
reset = "\e[0m"
|
|
44
|
-
green = "\e[32m"
|
|
45
|
-
puts "#{green}image url :#{reset} #{url}"
|
|
46
|
-
return url
|
|
47
|
-
end
|
|
48
|
-
|
|
49
|
-
puts 'No found'
|
|
50
|
-
nil
|
|
51
|
-
end
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
def get_video_url(pin_url)
|
|
55
|
-
html = fetch_html(pin_url)
|
|
56
|
-
return nil unless html
|
|
57
|
-
|
|
58
|
-
# 1 -> json-ld
|
|
59
|
-
url = extract_json_ld(html, 'VideoObject')
|
|
60
|
-
if url
|
|
61
|
-
url = url.gsub('\\u002F', '/').gsub('\\', '')
|
|
62
|
-
puts "ok : #{url}"
|
|
63
|
-
return url
|
|
64
|
-
end
|
|
65
|
-
|
|
66
|
-
# 2 -> Open Graph video
|
|
67
|
-
match = html.match(%r{<meta[^>]*property="og:video"[^>]*content="([^"]+)"})
|
|
68
|
-
if match
|
|
69
|
-
url = match[1].gsub('\\u002F', '/').gsub('\\', '')
|
|
70
|
-
puts "video url: #{url}"
|
|
71
|
-
return url
|
|
72
|
-
end
|
|
73
|
-
|
|
74
|
-
# 3 -> contentUrl in page source
|
|
75
|
-
match = html.match(/"contentUrl"\s*:\s*"([^"]+\.mp4[^"]*)"/)
|
|
76
|
-
if match
|
|
77
|
-
url = match[1].gsub('\\u002F', '/').gsub('\\', '')
|
|
78
|
-
green = "\e[32m"
|
|
79
|
-
reset = "\e[0m"
|
|
80
|
-
puts "#{green}video url :#{reset} #{url}"
|
|
81
|
-
return url
|
|
82
|
-
end
|
|
83
|
-
|
|
84
|
-
puts 'No found'
|
|
85
|
-
nil
|
|
86
|
-
end
|
|
87
|
-
|
|
88
|
-
private
|
|
89
|
-
|
|
90
|
-
def fetch_html(url)
|
|
91
|
-
uri = URI(url)
|
|
92
|
-
http = Net::HTTP.new(uri.host, uri.port)
|
|
93
|
-
http.use_ssl = (uri.scheme == 'https')
|
|
94
|
-
http.max_retries = 2
|
|
95
|
-
http.open_timeout = 10
|
|
96
|
-
http.read_timeout = 20
|
|
97
|
-
|
|
98
|
-
resp = http.request(Net::HTTP::Head.new(uri, 'User-Agent' => USER_AGENT))
|
|
99
|
-
|
|
100
|
-
5.times do
|
|
101
|
-
break unless resp.is_a?(Net::HTTPRedirection)
|
|
102
|
-
location = resp['location']
|
|
103
|
-
location = URI.join(uri.to_s, location) if location.start_with?('/')
|
|
104
|
-
uri = URI(location)
|
|
105
|
-
http = Net::HTTP.new(uri.host, uri.port)
|
|
106
|
-
http.use_ssl = (uri.scheme == 'https')
|
|
107
|
-
resp = http.request(Net::HTTP::Head.new(uri, 'User-Agent' => USER_AGENT))
|
|
108
|
-
end
|
|
109
|
-
|
|
110
|
-
# get final page
|
|
111
|
-
req = Net::HTTP::Get.new(uri, 'User-Agent' => USER_AGENT)
|
|
112
|
-
resp = http.request(req)
|
|
113
|
-
|
|
114
|
-
return resp.body if resp.is_a?(Net::HTTPSuccess)
|
|
115
|
-
nil
|
|
116
|
-
rescue StandardError => e
|
|
117
|
-
red = "\e[31m"
|
|
118
|
-
reset = "\e[0m"
|
|
119
|
-
puts "#{red}http err!:#{reset} #{e.message}"
|
|
120
|
-
nil
|
|
121
|
-
end
|
|
122
|
-
def extract_json_ld(html, type)
|
|
123
|
-
match = html.match(%r{<script[^>]*type="application/ld\+json"[^>]*>(.*?)</script>}m)
|
|
124
|
-
return nil unless match
|
|
125
|
-
|
|
126
|
-
begin
|
|
127
|
-
data = JSON.parse(match[1])
|
|
128
|
-
data = data[0] if data.is_a?(Array) && !data.empty?
|
|
129
|
-
return data['contentUrl'] if data['@type'] == type && data['contentUrl']
|
|
130
|
-
rescue JSON::ParserError
|
|
131
|
-
end
|
|
132
|
-
nil
|
|
133
|
-
end
|
|
134
|
-
end
|
|
135
|
-
end
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Monji024
|
|
4
|
+
require_relative 'pinterest_dl/version'
|
|
5
|
+
require_relative 'pinterest_dl/errors'
|
|
6
|
+
require_relative 'pinterest_dl/configuration'
|
|
7
|
+
require_relative 'pinterest_dl/client'
|
|
8
|
+
require_relative 'pinterest_dl/extractors/image'
|
|
9
|
+
require_relative 'pinterest_dl/extractors/video'
|
|
10
|
+
require_relative 'pinterest_dl/search'
|
|
11
|
+
require_relative 'pinterest_dl/board'
|
|
12
|
+
require_relative 'pinterest_dl/downloader'
|
|
13
|
+
require_relative 'pinterest_dl/progress_bar'
|
|
14
|
+
require_relative 'pinterest_dl/api'
|
|
15
|
+
|
|
16
|
+
module PinterestDL
|
|
17
|
+
end
|
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
# Monji024
|
|
4
|
+
module PinterestDL
|
|
5
|
+
PIN_URL_PATTERN = %r{pinterest\.[a-z.]+/pin/}i.freeze
|
|
6
|
+
|
|
7
|
+
class << self
|
|
8
|
+
def config
|
|
9
|
+
@config ||= Configuration.new
|
|
10
|
+
end
|
|
11
|
+
|
|
12
|
+
def configure
|
|
13
|
+
yield config
|
|
14
|
+
config
|
|
15
|
+
end
|
|
16
|
+
|
|
17
|
+
def logger
|
|
18
|
+
config.logger
|
|
19
|
+
end
|
|
20
|
+
|
|
21
|
+
def get_image_url(pin_url, quality: config.quality, client: Client.new(config: config))
|
|
22
|
+
validate_pin_url!(pin_url)
|
|
23
|
+
html = client.get(pin_url)
|
|
24
|
+
url = Extractors::Image.extract(html, quality: quality)
|
|
25
|
+
raise NotFoundError, "No image found at #{pin_url}" unless url
|
|
26
|
+
|
|
27
|
+
url
|
|
28
|
+
end
|
|
29
|
+
|
|
30
|
+
def get_video_url(pin_url, client: Client.new(config: config))
|
|
31
|
+
validate_pin_url!(pin_url)
|
|
32
|
+
html = client.get(pin_url)
|
|
33
|
+
url = Extractors::Video.extract(html)
|
|
34
|
+
raise NotFoundError, "No video found at #{pin_url}" unless url
|
|
35
|
+
|
|
36
|
+
url
|
|
37
|
+
end
|
|
38
|
+
|
|
39
|
+
def get_media(pin_url, quality: config.quality)
|
|
40
|
+
client = Client.new(config: config)
|
|
41
|
+
html = client.get(pin_url)
|
|
42
|
+
{
|
|
43
|
+
image_url: Extractors::Image.extract(html, quality: quality),
|
|
44
|
+
video_url: Extractors::Video.extract(html)
|
|
45
|
+
}
|
|
46
|
+
end
|
|
47
|
+
|
|
48
|
+
def download(url, path:)
|
|
49
|
+
Downloader.new(config: config).download(url, path: path)
|
|
50
|
+
end
|
|
51
|
+
|
|
52
|
+
def download_batch(urls, dir:, filename: nil, &progress)
|
|
53
|
+
Downloader.new(config: config).download_batch(urls, dir: dir, filename: filename, &progress)
|
|
54
|
+
end
|
|
55
|
+
|
|
56
|
+
def search(query, limit: 25, pages: 5)
|
|
57
|
+
Search.new(config: config).call(query, limit: limit, pages: pages)
|
|
58
|
+
end
|
|
59
|
+
|
|
60
|
+
def search_json(query, limit: 25)
|
|
61
|
+
JSON.pretty_generate(search(query, limit: limit))
|
|
62
|
+
end
|
|
63
|
+
|
|
64
|
+
def search_images(query, limit: 25)
|
|
65
|
+
search(query, limit: limit).filter_map { |p| p[:image_url] }
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
def search_videos(query, limit: 25)
|
|
69
|
+
search(query, limit: limit).filter_map { |p| p[:video_url] }
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
def board(board_url, limit: 50, pages: 10)
|
|
73
|
+
Board.new(config: config).call(board_url, limit: limit, pages: pages)
|
|
74
|
+
end
|
|
75
|
+
|
|
76
|
+
private
|
|
77
|
+
|
|
78
|
+
def validate_pin_url!(pin_url)
|
|
79
|
+
return if pin_url.to_s.match?(PIN_URL_PATTERN)
|
|
80
|
+
|
|
81
|
+
raise InvalidURLError, "Not a Pinterest pin URL: #{pin_url.inspect}"
|
|
82
|
+
end
|
|
83
|
+
end
|
|
84
|
+
end
|