patreon-scraper-api 0.0.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- patreon_scraper_api-0.0.1/LICENSE +21 -0
- patreon_scraper_api-0.0.1/MANIFEST.in +2 -0
- patreon_scraper_api-0.0.1/PKG-INFO +214 -0
- patreon_scraper_api-0.0.1/README.md +184 -0
- patreon_scraper_api-0.0.1/pyproject.toml +38 -0
- patreon_scraper_api-0.0.1/setup.cfg +4 -0
- patreon_scraper_api-0.0.1/src/patreon_scraper_api/__init__.py +6 -0
- patreon_scraper_api-0.0.1/src/patreon_scraper_api/client.py +260 -0
- patreon_scraper_api-0.0.1/src/patreon_scraper_api.egg-info/PKG-INFO +214 -0
- patreon_scraper_api-0.0.1/src/patreon_scraper_api.egg-info/SOURCES.txt +11 -0
- patreon_scraper_api-0.0.1/src/patreon_scraper_api.egg-info/dependency_links.txt +1 -0
- patreon_scraper_api-0.0.1/src/patreon_scraper_api.egg-info/requires.txt +1 -0
- patreon_scraper_api-0.0.1/src/patreon_scraper_api.egg-info/top_level.txt +1 -0
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 wordstotech
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: patreon-scraper-api
|
|
3
|
+
Version: 0.0.1
|
|
4
|
+
Summary: Patreon scraper API client: public creator pages, entry tiers, about text and structured profile data via ScrapingBee.
|
|
5
|
+
Author: wordstotech
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/ScrapingBee/patreon-api
|
|
8
|
+
Project-URL: Repository, https://github.com/ScrapingBee/patreon-api
|
|
9
|
+
Project-URL: Documentation, https://www.scrapingbee.com/documentation/
|
|
10
|
+
Keywords: patreon scraper,patreon api,scraper for patreon,creator economy,web scraping,scrapingbee
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
22
|
+
Classifier: Topic :: Internet :: WWW/HTTP
|
|
23
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
24
|
+
Classifier: Topic :: Text Processing :: Markup :: HTML
|
|
25
|
+
Requires-Python: >=3.8
|
|
26
|
+
Description-Content-Type: text/markdown
|
|
27
|
+
License-File: LICENSE
|
|
28
|
+
Requires-Dist: requests>=2.25.0
|
|
29
|
+
Dynamic: license-file
|
|
30
|
+
|
|
31
|
+
# patreon-scraper-api
|
|
32
|
+
|
|
33
|
+
A Python client for scraping public Patreon creator pages through ScrapingBee. Four extraction routes, each with a documented cost and a documented failure mode.
|
|
34
|
+
|
|
35
|
+
**Verified against `https://www.patreon.com/kurzgesagt` on 2026-09-10.** Every field name, credit figure and return value below came from a real call. Two of the four routes have caveats that only a live run exposes, and both are written down rather than smoothed over.
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
pip install patreon-scraper-api
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Requires Python 3.8 or newer and `requests`.
|
|
42
|
+
|
|
43
|
+
## What is in scope
|
|
44
|
+
|
|
45
|
+
Public creator pages, read anonymously.
|
|
46
|
+
|
|
47
|
+
**Reachable:** creator display name, campaign tagline, the full about text, avatar and banner URLs, canonical URL, the complete tier list with prices and per tier post counts, and the entry price teaser.
|
|
48
|
+
|
|
49
|
+
**Not reachable, by design:** patron only posts, attachment downloads, member lists, individual pledge amounts and direct messages. Those require a signed in session, and scraping under login credentials is prohibited by ScrapingBee's terms of service. No parameter in this package changes that.
|
|
50
|
+
|
|
51
|
+
## Authentication
|
|
52
|
+
|
|
53
|
+
```python
|
|
54
|
+
from patreon_scraper_api import PatreonScraper
|
|
55
|
+
|
|
56
|
+
bee = PatreonScraper("YOUR_API_KEY")
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Sent as `Authorization: Bearer YOUR_API_KEY` on every request. The `api_key` query parameter still answers but the current documentation marks it deprecated.
|
|
60
|
+
|
|
61
|
+
Key and 1,000 free credits: [ScrapingBee](https://www.scrapingbee.com/). Landing page for this target: [Patreon scraper API](https://www.scrapingbee.com/scrapers/patreon-api/).
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## Method reference
|
|
66
|
+
|
|
67
|
+
### `creator(slug)`
|
|
68
|
+
|
|
69
|
+
**5 credits.** Meta tags, via `extract_rules`. The route that does not break, because these tags exist for social preview cards rather than for the application.
|
|
70
|
+
|
|
71
|
+
```python
|
|
72
|
+
bee.creator("kurzgesagt")
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
```python
|
|
76
|
+
{
|
|
77
|
+
'creator': 'Kurzgesagt – In a Nutshell',
|
|
78
|
+
'title': 'Kurzgesagt – In a Nutshell — Creating Science Animation Videos | Patreon',
|
|
79
|
+
'og_title': 'Kurzgesagt – In a Nutshell — Creating Science Animation Videos',
|
|
80
|
+
'og_desc': 'Get more from Kurzgesagt – In a Nutshell on Patreon. Creating Science '
|
|
81
|
+
'Animation Videos. Support Kurzgesagt – In a Nutshell and get exclusive '
|
|
82
|
+
'access to their work.',
|
|
83
|
+
'campaign_tagline': 'Creating Science Animation Videos',
|
|
84
|
+
}
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
`campaign_tagline` is derived, not scraped. Patreon joins the creator name and the tagline with an em dash, while a creator name may itself contain an en dash, so splitting on the em dash is unambiguous. The client does that split for you.
|
|
88
|
+
|
|
89
|
+
`mode=auto` walks the proxy ladder and bills only the rung that worked, which was the JavaScript rung at 5 credits. `spb-initial-status-code` reported `308`, which is Patreon's canonical redirect rather than a block.
|
|
90
|
+
|
|
91
|
+
### `profile(slug)`
|
|
92
|
+
|
|
93
|
+
**5 credits.** The `application/ld+json` `ProfilePage` block, parsed after the fetch.
|
|
94
|
+
|
|
95
|
+
```python
|
|
96
|
+
bee.profile("kurzgesagt")
|
|
97
|
+
# {'name': 'Kurzgesagt – In a Nutshell',
|
|
98
|
+
# 'alternate_name': 'Kurzgesagt',
|
|
99
|
+
# 'url': 'https://www.patreon.com/Kurzgesagt',
|
|
100
|
+
# 'about': '<4,476 characters>',
|
|
101
|
+
# 'image': 'https://c10.patreonusercontent.com/...',
|
|
102
|
+
# 'thumbnail': 'https://c10.patreonusercontent.com/...'}
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
Use this over `creator()` when you need the complete about copy, for search indexing or embeddings. The meta description truncates it. The live page carried 4,476 characters here against roughly 160 in `og_desc`.
|
|
106
|
+
|
|
107
|
+
**Why this parses HTML instead of using `extract_rules`:** because `extract_rules` cannot reach script tag contents. A rule of `{"jsonld": {"selector": "script[type=\"application/ld+json\"]", "output": "@text"}}` returns `None`. That was tested directly, not assumed. The client fetches the page and parses the block itself.
|
|
108
|
+
|
|
109
|
+
The live page carried six such blocks, and their order is not stable, so the client matches on `@type`. One of them is an `Organization` block describing Patreon rather than the creator, which is exactly the row an index based parser would pick up by mistake.
|
|
110
|
+
|
|
111
|
+
### `tiers(slug)`
|
|
112
|
+
|
|
113
|
+
**5 credits.** The complete tier list, from Patreon's own bootstrap payload.
|
|
114
|
+
|
|
115
|
+
```python
|
|
116
|
+
result = bee.tiers("kurzgesagt")
|
|
117
|
+
result["matched"] # True
|
|
118
|
+
result["count"] # 4
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Live output, sorted cheapest first:
|
|
122
|
+
|
|
123
|
+
| `title` | `amount_cents` | `is_free_tier` | `post_count` | `declined_patron_count` |
|
|
124
|
+
|---|---|---|---|---|
|
|
125
|
+
| Free | 0 | True | 8 | 2 |
|
|
126
|
+
| Trainee Producer | 314 | False | 1 | 31 |
|
|
127
|
+
| Producer | 1500 | False | 62 | 14 |
|
|
128
|
+
| Senior Producer | 4200 | False | 58 | 0 |
|
|
129
|
+
|
|
130
|
+
Every key on a tier object: `title`, `amount_cents`, `currency`, `description`, `url`, `image_url`, `is_free_tier`, `post_count`, `published`, `requires_shipping`, `declined_patron_count`, `patron_amount_cents`, `patron_currency`, `discord_role_ids`, `remaining`, `user_limit`.
|
|
131
|
+
|
|
132
|
+
**`declined_patron_count` is not a patron count.** It counts declined payments. It reads 2, 31, 14 and 0 across those tiers, which is nothing like a membership figure for a creator of that size. Patreon does not publish a per tier patron count on the public page, so this package passes the field through under its real name rather than relabelling it as something more useful.
|
|
133
|
+
|
|
134
|
+
`amount_cents` with `currency` is the tier price and is the pair to trust. `patron_amount_cents` and `patron_currency` also appear, reading `800` and `DKK` on the free tier, which does not correspond to the tier price, so they are returned raw without interpretation.
|
|
135
|
+
|
|
136
|
+
`remaining` and `user_limit` were both `None` on every tier here. They carry real values on creators who cap a tier.
|
|
137
|
+
|
|
138
|
+
This route reads a private application format Patreon has no obligation to keep stable. When the pattern stops matching, the client returns `{"tiers": [], "matched": False, "fallback": <creator() result>}` rather than an empty list that reads like a creator with no tiers.
|
|
139
|
+
|
|
140
|
+
### `price_ladder(slug)` and `entry_tier(slug)`
|
|
141
|
+
|
|
142
|
+
Convenience wrappers over `tiers()`, same 5 credits.
|
|
143
|
+
|
|
144
|
+
```python
|
|
145
|
+
bee.price_ladder("kurzgesagt")
|
|
146
|
+
# [{'title': 'Free', 'price': 0.0, 'currency': 'USD', 'free': True, 'posts': 8},
|
|
147
|
+
# {'title': 'Trainee Producer', 'price': 3.14, 'currency': 'USD', 'free': False, 'posts': 1},
|
|
148
|
+
# {'title': 'Producer', 'price': 15.0, 'currency': 'USD', 'free': False, 'posts': 62},
|
|
149
|
+
# {'title': 'Senior Producer', 'price': 42.0, 'currency': 'USD', 'free': False, 'posts': 58}]
|
|
150
|
+
|
|
151
|
+
bee.entry_tier("kurzgesagt")["title"] # 'Trainee Producer'
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
`entry_tier()` skips the free tier, because `amount_cents` of 0 is not an entry price.
|
|
155
|
+
|
|
156
|
+
### `entry_price(slug)`
|
|
157
|
+
|
|
158
|
+
**30 credits.** Premium proxy plus JavaScript at 25, plus 5 for the AI query. Uses `ai_extract_rules`, so there is no parser to maintain.
|
|
159
|
+
|
|
160
|
+
```python
|
|
161
|
+
bee.entry_price("kurzgesagt")
|
|
162
|
+
# {'creator_name': 'Kurzgesagt – In a Nutshell',
|
|
163
|
+
# 'about': '<full about text>',
|
|
164
|
+
# 'membership_tiers': ['Access exclusive benefits starting at $3.14/month']}
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
`creator_name` and `about` come back complete and correct. `membership_tiers` comes back as **a single teaser string, not a tier array**, because the tier cards are mounted by a component that has not rendered at capture time. The model described what was actually on the page, which is the honest result rather than a broken one.
|
|
168
|
+
|
|
169
|
+
That said, `$3.14` matches the `amount_cents` value of `314` that `tiers()` reads out of the payload, so the two routes independently agree on the entry price. Use `entry_price()` when you want that one number and no parser. Use `tiers()` when you want the whole ladder.
|
|
170
|
+
|
|
171
|
+
### `usage()`
|
|
172
|
+
|
|
173
|
+
Free. Account credits, concurrency and renewal date.
|
|
174
|
+
|
|
175
|
+
---
|
|
176
|
+
|
|
177
|
+
## Choosing a route
|
|
178
|
+
|
|
179
|
+
| You need | Method | Credits |
|
|
180
|
+
|---|---|---|
|
|
181
|
+
| Name and tagline for a directory | `creator` | 5 |
|
|
182
|
+
| Full about copy for search or embeddings | `profile` | 5 |
|
|
183
|
+
| Every tier with its price and post count | `tiers` | 5 |
|
|
184
|
+
| A price ladder ready to display | `price_ladder` | 5 |
|
|
185
|
+
| Entry price only, no parser to own | `entry_price` | 30 |
|
|
186
|
+
|
|
187
|
+
`creator()` and `profile()` read the same fetched page, so if you want both, fetch raw once and run both parsers locally for a single 5 credit charge.
|
|
188
|
+
|
|
189
|
+
## Credit cost
|
|
190
|
+
|
|
191
|
+
Measured from `spb-cost` headers. Available on `bee.last_cost` after every call.
|
|
192
|
+
|
|
193
|
+
| Configuration | Credits |
|
|
194
|
+
|---|---|
|
|
195
|
+
| `mode=auto` on a creator page, settled at the JavaScript rung | 5 |
|
|
196
|
+
| `render_js` plus `premium_proxy` | 25 |
|
|
197
|
+
| The same plus `ai_extract_rules` | 30 |
|
|
198
|
+
| Validation error | 0 |
|
|
199
|
+
|
|
200
|
+
Auto mode is the right default here: it charges only for the configuration that worked, and nothing at all if every rung fails. It is incompatible with `render_js`, `premium_proxy` and `stealth_proxy`, and sending both returns HTTP 400 while billing nothing, which fails quietly if you do not read status codes.
|
|
201
|
+
|
|
202
|
+
At 5 credits per creator, 250,000 credits covers 50,000 creator checks. Plan tiers: [ScrapingBee pricing](https://www.scrapingbee.com/pricing).
|
|
203
|
+
|
|
204
|
+
## Related
|
|
205
|
+
|
|
206
|
+
Landing pages for adjacent creator platforms: [Substack scraper API](https://www.scrapingbee.com/scrapers/substack-scraper-api/), [Twitch API](https://www.scrapingbee.com/scrapers/twitch-api/), [TikTok API](https://www.scrapingbee.com/scrapers/tiktok-api/), [TikTok follower API](https://www.scrapingbee.com/scrapers/tiktok-follower/), [YouTube video scraper API](https://www.scrapingbee.com/scrapers/youtube-video-scraper-api/), [YouTube transcript scraper API](https://www.scrapingbee.com/scrapers/youtube-transcript-scraper-api/), [YouTube comment scraper API](https://www.scrapingbee.com/scrapers/youtube-comment-scraper-api/), [Snapchat scraper API](https://www.scrapingbee.com/scrapers/snapchat-scraper-api/).
|
|
207
|
+
|
|
208
|
+
Features: [AI web scraping](https://www.scrapingbee.com/features/ai-web-scraping-api/), [data extraction](https://www.scrapingbee.com/features/data-extraction/), [markdown scraper](https://www.scrapingbee.com/features/markdown-scraper/), [screenshots](https://www.scrapingbee.com/features/screenshot/), [JavaScript scenario](https://www.scrapingbee.com/features/javascript-scenario/), [n8n integration](https://www.scrapingbee.com/features/n8n/).
|
|
209
|
+
|
|
210
|
+
Reference: [extraction rules documentation](https://www.scrapingbee.com/documentation/data-extraction/). Guide version with the full route walkthrough: [github.com/ScrapingBee/patreon-api](https://github.com/ScrapingBee/patreon-api).
|
|
211
|
+
|
|
212
|
+
## License
|
|
213
|
+
|
|
214
|
+
MIT
|
|
@@ -0,0 +1,184 @@
|
|
|
1
|
+
# patreon-scraper-api
|
|
2
|
+
|
|
3
|
+
A Python client for scraping public Patreon creator pages through ScrapingBee. Four extraction routes, each with a documented cost and a documented failure mode.
|
|
4
|
+
|
|
5
|
+
**Verified against `https://www.patreon.com/kurzgesagt` on 2026-09-10.** Every field name, credit figure and return value below came from a real call. Two of the four routes have caveats that only a live run exposes, and both are written down rather than smoothed over.
|
|
6
|
+
|
|
7
|
+
```bash
|
|
8
|
+
pip install patreon-scraper-api
|
|
9
|
+
```
|
|
10
|
+
|
|
11
|
+
Requires Python 3.8 or newer and `requests`.
|
|
12
|
+
|
|
13
|
+
## What is in scope
|
|
14
|
+
|
|
15
|
+
Public creator pages, read anonymously.
|
|
16
|
+
|
|
17
|
+
**Reachable:** creator display name, campaign tagline, the full about text, avatar and banner URLs, canonical URL, the complete tier list with prices and per tier post counts, and the entry price teaser.
|
|
18
|
+
|
|
19
|
+
**Not reachable, by design:** patron only posts, attachment downloads, member lists, individual pledge amounts and direct messages. Those require a signed in session, and scraping under login credentials is prohibited by ScrapingBee's terms of service. No parameter in this package changes that.
|
|
20
|
+
|
|
21
|
+
## Authentication
|
|
22
|
+
|
|
23
|
+
```python
|
|
24
|
+
from patreon_scraper_api import PatreonScraper
|
|
25
|
+
|
|
26
|
+
bee = PatreonScraper("YOUR_API_KEY")
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Sent as `Authorization: Bearer YOUR_API_KEY` on every request. The `api_key` query parameter still answers but the current documentation marks it deprecated.
|
|
30
|
+
|
|
31
|
+
Key and 1,000 free credits: [ScrapingBee](https://www.scrapingbee.com/). Landing page for this target: [Patreon scraper API](https://www.scrapingbee.com/scrapers/patreon-api/).
|
|
32
|
+
|
|
33
|
+
---
|
|
34
|
+
|
|
35
|
+
## Method reference
|
|
36
|
+
|
|
37
|
+
### `creator(slug)`
|
|
38
|
+
|
|
39
|
+
**5 credits.** Meta tags, via `extract_rules`. The route that does not break, because these tags exist for social preview cards rather than for the application.
|
|
40
|
+
|
|
41
|
+
```python
|
|
42
|
+
bee.creator("kurzgesagt")
|
|
43
|
+
```
|
|
44
|
+
|
|
45
|
+
```python
|
|
46
|
+
{
|
|
47
|
+
'creator': 'Kurzgesagt – In a Nutshell',
|
|
48
|
+
'title': 'Kurzgesagt – In a Nutshell — Creating Science Animation Videos | Patreon',
|
|
49
|
+
'og_title': 'Kurzgesagt – In a Nutshell — Creating Science Animation Videos',
|
|
50
|
+
'og_desc': 'Get more from Kurzgesagt – In a Nutshell on Patreon. Creating Science '
|
|
51
|
+
'Animation Videos. Support Kurzgesagt – In a Nutshell and get exclusive '
|
|
52
|
+
'access to their work.',
|
|
53
|
+
'campaign_tagline': 'Creating Science Animation Videos',
|
|
54
|
+
}
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
`campaign_tagline` is derived, not scraped. Patreon joins the creator name and the tagline with an em dash, while a creator name may itself contain an en dash, so splitting on the em dash is unambiguous. The client does that split for you.
|
|
58
|
+
|
|
59
|
+
`mode=auto` walks the proxy ladder and bills only the rung that worked, which was the JavaScript rung at 5 credits. `spb-initial-status-code` reported `308`, which is Patreon's canonical redirect rather than a block.
|
|
60
|
+
|
|
61
|
+
### `profile(slug)`
|
|
62
|
+
|
|
63
|
+
**5 credits.** The `application/ld+json` `ProfilePage` block, parsed after the fetch.
|
|
64
|
+
|
|
65
|
+
```python
|
|
66
|
+
bee.profile("kurzgesagt")
|
|
67
|
+
# {'name': 'Kurzgesagt – In a Nutshell',
|
|
68
|
+
# 'alternate_name': 'Kurzgesagt',
|
|
69
|
+
# 'url': 'https://www.patreon.com/Kurzgesagt',
|
|
70
|
+
# 'about': '<4,476 characters>',
|
|
71
|
+
# 'image': 'https://c10.patreonusercontent.com/...',
|
|
72
|
+
# 'thumbnail': 'https://c10.patreonusercontent.com/...'}
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
Use this over `creator()` when you need the complete about copy, for search indexing or embeddings. The meta description truncates it. The live page carried 4,476 characters here against roughly 160 in `og_desc`.
|
|
76
|
+
|
|
77
|
+
**Why this parses HTML instead of using `extract_rules`:** because `extract_rules` cannot reach script tag contents. A rule of `{"jsonld": {"selector": "script[type=\"application/ld+json\"]", "output": "@text"}}` returns `None`. That was tested directly, not assumed. The client fetches the page and parses the block itself.
|
|
78
|
+
|
|
79
|
+
The live page carried six such blocks, and their order is not stable, so the client matches on `@type`. One of them is an `Organization` block describing Patreon rather than the creator, which is exactly the row an index based parser would pick up by mistake.
|
|
80
|
+
|
|
81
|
+
### `tiers(slug)`
|
|
82
|
+
|
|
83
|
+
**5 credits.** The complete tier list, from Patreon's own bootstrap payload.
|
|
84
|
+
|
|
85
|
+
```python
|
|
86
|
+
result = bee.tiers("kurzgesagt")
|
|
87
|
+
result["matched"] # True
|
|
88
|
+
result["count"] # 4
|
|
89
|
+
```
|
|
90
|
+
|
|
91
|
+
Live output, sorted cheapest first:
|
|
92
|
+
|
|
93
|
+
| `title` | `amount_cents` | `is_free_tier` | `post_count` | `declined_patron_count` |
|
|
94
|
+
|---|---|---|---|---|
|
|
95
|
+
| Free | 0 | True | 8 | 2 |
|
|
96
|
+
| Trainee Producer | 314 | False | 1 | 31 |
|
|
97
|
+
| Producer | 1500 | False | 62 | 14 |
|
|
98
|
+
| Senior Producer | 4200 | False | 58 | 0 |
|
|
99
|
+
|
|
100
|
+
Every key on a tier object: `title`, `amount_cents`, `currency`, `description`, `url`, `image_url`, `is_free_tier`, `post_count`, `published`, `requires_shipping`, `declined_patron_count`, `patron_amount_cents`, `patron_currency`, `discord_role_ids`, `remaining`, `user_limit`.
|
|
101
|
+
|
|
102
|
+
**`declined_patron_count` is not a patron count.** It counts declined payments. It reads 2, 31, 14 and 0 across those tiers, which is nothing like a membership figure for a creator of that size. Patreon does not publish a per tier patron count on the public page, so this package passes the field through under its real name rather than relabelling it as something more useful.
|
|
103
|
+
|
|
104
|
+
`amount_cents` with `currency` is the tier price and is the pair to trust. `patron_amount_cents` and `patron_currency` also appear, reading `800` and `DKK` on the free tier, which does not correspond to the tier price, so they are returned raw without interpretation.
|
|
105
|
+
|
|
106
|
+
`remaining` and `user_limit` were both `None` on every tier here. They carry real values on creators who cap a tier.
|
|
107
|
+
|
|
108
|
+
This route reads a private application format Patreon has no obligation to keep stable. When the pattern stops matching, the client returns `{"tiers": [], "matched": False, "fallback": <creator() result>}` rather than an empty list that reads like a creator with no tiers.
|
|
109
|
+
|
|
110
|
+
### `price_ladder(slug)` and `entry_tier(slug)`
|
|
111
|
+
|
|
112
|
+
Convenience wrappers over `tiers()`, same 5 credits.
|
|
113
|
+
|
|
114
|
+
```python
|
|
115
|
+
bee.price_ladder("kurzgesagt")
|
|
116
|
+
# [{'title': 'Free', 'price': 0.0, 'currency': 'USD', 'free': True, 'posts': 8},
|
|
117
|
+
# {'title': 'Trainee Producer', 'price': 3.14, 'currency': 'USD', 'free': False, 'posts': 1},
|
|
118
|
+
# {'title': 'Producer', 'price': 15.0, 'currency': 'USD', 'free': False, 'posts': 62},
|
|
119
|
+
# {'title': 'Senior Producer', 'price': 42.0, 'currency': 'USD', 'free': False, 'posts': 58}]
|
|
120
|
+
|
|
121
|
+
bee.entry_tier("kurzgesagt")["title"] # 'Trainee Producer'
|
|
122
|
+
```
|
|
123
|
+
|
|
124
|
+
`entry_tier()` skips the free tier, because `amount_cents` of 0 is not an entry price.
|
|
125
|
+
|
|
126
|
+
### `entry_price(slug)`
|
|
127
|
+
|
|
128
|
+
**30 credits.** Premium proxy plus JavaScript at 25, plus 5 for the AI query. Uses `ai_extract_rules`, so there is no parser to maintain.
|
|
129
|
+
|
|
130
|
+
```python
|
|
131
|
+
bee.entry_price("kurzgesagt")
|
|
132
|
+
# {'creator_name': 'Kurzgesagt – In a Nutshell',
|
|
133
|
+
# 'about': '<full about text>',
|
|
134
|
+
# 'membership_tiers': ['Access exclusive benefits starting at $3.14/month']}
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
`creator_name` and `about` come back complete and correct. `membership_tiers` comes back as **a single teaser string, not a tier array**, because the tier cards are mounted by a component that has not rendered at capture time. The model described what was actually on the page, which is the honest result rather than a broken one.
|
|
138
|
+
|
|
139
|
+
That said, `$3.14` matches the `amount_cents` value of `314` that `tiers()` reads out of the payload, so the two routes independently agree on the entry price. Use `entry_price()` when you want that one number and no parser. Use `tiers()` when you want the whole ladder.
|
|
140
|
+
|
|
141
|
+
### `usage()`
|
|
142
|
+
|
|
143
|
+
Free. Account credits, concurrency and renewal date.
|
|
144
|
+
|
|
145
|
+
---
|
|
146
|
+
|
|
147
|
+
## Choosing a route
|
|
148
|
+
|
|
149
|
+
| You need | Method | Credits |
|
|
150
|
+
|---|---|---|
|
|
151
|
+
| Name and tagline for a directory | `creator` | 5 |
|
|
152
|
+
| Full about copy for search or embeddings | `profile` | 5 |
|
|
153
|
+
| Every tier with its price and post count | `tiers` | 5 |
|
|
154
|
+
| A price ladder ready to display | `price_ladder` | 5 |
|
|
155
|
+
| Entry price only, no parser to own | `entry_price` | 30 |
|
|
156
|
+
|
|
157
|
+
`creator()` and `profile()` read the same fetched page, so if you want both, fetch raw once and run both parsers locally for a single 5 credit charge.
|
|
158
|
+
|
|
159
|
+
## Credit cost
|
|
160
|
+
|
|
161
|
+
Measured from `spb-cost` headers. Available on `bee.last_cost` after every call.
|
|
162
|
+
|
|
163
|
+
| Configuration | Credits |
|
|
164
|
+
|---|---|
|
|
165
|
+
| `mode=auto` on a creator page, settled at the JavaScript rung | 5 |
|
|
166
|
+
| `render_js` plus `premium_proxy` | 25 |
|
|
167
|
+
| The same plus `ai_extract_rules` | 30 |
|
|
168
|
+
| Validation error | 0 |
|
|
169
|
+
|
|
170
|
+
Auto mode is the right default here: it charges only for the configuration that worked, and nothing at all if every rung fails. It is incompatible with `render_js`, `premium_proxy` and `stealth_proxy`, and sending both returns HTTP 400 while billing nothing, which fails quietly if you do not read status codes.
|
|
171
|
+
|
|
172
|
+
At 5 credits per creator, 250,000 credits covers 50,000 creator checks. Plan tiers: [ScrapingBee pricing](https://www.scrapingbee.com/pricing).
|
|
173
|
+
|
|
174
|
+
## Related
|
|
175
|
+
|
|
176
|
+
Landing pages for adjacent creator platforms: [Substack scraper API](https://www.scrapingbee.com/scrapers/substack-scraper-api/), [Twitch API](https://www.scrapingbee.com/scrapers/twitch-api/), [TikTok API](https://www.scrapingbee.com/scrapers/tiktok-api/), [TikTok follower API](https://www.scrapingbee.com/scrapers/tiktok-follower/), [YouTube video scraper API](https://www.scrapingbee.com/scrapers/youtube-video-scraper-api/), [YouTube transcript scraper API](https://www.scrapingbee.com/scrapers/youtube-transcript-scraper-api/), [YouTube comment scraper API](https://www.scrapingbee.com/scrapers/youtube-comment-scraper-api/), [Snapchat scraper API](https://www.scrapingbee.com/scrapers/snapchat-scraper-api/).
|
|
177
|
+
|
|
178
|
+
Features: [AI web scraping](https://www.scrapingbee.com/features/ai-web-scraping-api/), [data extraction](https://www.scrapingbee.com/features/data-extraction/), [markdown scraper](https://www.scrapingbee.com/features/markdown-scraper/), [screenshots](https://www.scrapingbee.com/features/screenshot/), [JavaScript scenario](https://www.scrapingbee.com/features/javascript-scenario/), [n8n integration](https://www.scrapingbee.com/features/n8n/).
|
|
179
|
+
|
|
180
|
+
Reference: [extraction rules documentation](https://www.scrapingbee.com/documentation/data-extraction/). Guide version with the full route walkthrough: [github.com/ScrapingBee/patreon-api](https://github.com/ScrapingBee/patreon-api).
|
|
181
|
+
|
|
182
|
+
## License
|
|
183
|
+
|
|
184
|
+
MIT
|
|
@@ -0,0 +1,38 @@
|
|
|
1
|
+
[build-system]
|
|
2
|
+
requires = ["setuptools>=68", "wheel"]
|
|
3
|
+
build-backend = "setuptools.build_meta"
|
|
4
|
+
|
|
5
|
+
[project]
|
|
6
|
+
name = "patreon-scraper-api"
|
|
7
|
+
version = "0.0.1"
|
|
8
|
+
description = "Patreon scraper API client: public creator pages, entry tiers, about text and structured profile data via ScrapingBee."
|
|
9
|
+
readme = "README.md"
|
|
10
|
+
requires-python = ">=3.8"
|
|
11
|
+
license = { text = "MIT" }
|
|
12
|
+
authors = [{ name = "wordstotech" }]
|
|
13
|
+
keywords = ["patreon scraper", "patreon api", "scraper for patreon", "creator economy", "web scraping", "scrapingbee"]
|
|
14
|
+
dependencies = ["requests>=2.25.0"]
|
|
15
|
+
classifiers = [
|
|
16
|
+
"Development Status :: 4 - Beta",
|
|
17
|
+
"Intended Audience :: Developers",
|
|
18
|
+
"License :: OSI Approved :: MIT License",
|
|
19
|
+
"Operating System :: OS Independent",
|
|
20
|
+
"Programming Language :: Python :: 3",
|
|
21
|
+
"Programming Language :: Python :: 3.8",
|
|
22
|
+
"Programming Language :: Python :: 3.9",
|
|
23
|
+
"Programming Language :: Python :: 3.10",
|
|
24
|
+
"Programming Language :: Python :: 3.11",
|
|
25
|
+
"Programming Language :: Python :: 3.12",
|
|
26
|
+
"Programming Language :: Python :: 3.13",
|
|
27
|
+
"Topic :: Internet :: WWW/HTTP",
|
|
28
|
+
"Topic :: Software Development :: Libraries :: Python Modules",
|
|
29
|
+
"Topic :: Text Processing :: Markup :: HTML",
|
|
30
|
+
]
|
|
31
|
+
|
|
32
|
+
[project.urls]
|
|
33
|
+
Homepage = "https://github.com/ScrapingBee/patreon-api"
|
|
34
|
+
Repository = "https://github.com/ScrapingBee/patreon-api"
|
|
35
|
+
Documentation = "https://www.scrapingbee.com/documentation/"
|
|
36
|
+
|
|
37
|
+
[tool.setuptools.packages.find]
|
|
38
|
+
where = ["src"]
|
|
@@ -0,0 +1,260 @@
|
|
|
1
|
+
"""ScrapingBee based Patreon scraper for public creator pages.
|
|
2
|
+
|
|
3
|
+
Verified against https://www.patreon.com/kurzgesagt on 2026-09-10.
|
|
4
|
+
|
|
5
|
+
Scope: public creator pages only. Patron only posts, member lists,
|
|
6
|
+
individual pledge amounts and anything behind a sign in are out of reach,
|
|
7
|
+
and scraping under login credentials is prohibited by ScrapingBee's terms
|
|
8
|
+
of service.
|
|
9
|
+
"""
|
|
10
|
+
|
|
11
|
+
from __future__ import annotations
|
|
12
|
+
|
|
13
|
+
import json
|
|
14
|
+
import re
|
|
15
|
+
from typing import Any, Dict, List, Optional
|
|
16
|
+
|
|
17
|
+
import requests
|
|
18
|
+
|
|
19
|
+
__all__ = ["PatreonScraper", "ScrapingBeeError"]
|
|
20
|
+
|
|
21
|
+
BASE_URL = "https://app.scrapingbee.com/api/v1/"
|
|
22
|
+
|
|
23
|
+
# Meta tag rules. The durable route: these tags exist for social previews,
|
|
24
|
+
# so Patreon cannot move them without breaking every share link.
|
|
25
|
+
_META_RULES = {
|
|
26
|
+
"creator": {"selector": "h1", "output": "text"},
|
|
27
|
+
"title": "title",
|
|
28
|
+
"og_title": {"selector": 'meta[property="og:title"]', "output": "@content"},
|
|
29
|
+
"og_desc": {"selector": 'meta[name="description"]', "output": "@content"},
|
|
30
|
+
}
|
|
31
|
+
|
|
32
|
+
_LD_BLOCK = re.compile(
|
|
33
|
+
r"<script[^>]*application/ld\+json[^>]*>(.*?)</script>", re.S
|
|
34
|
+
)
|
|
35
|
+
# Tier objects live in an "attributes" block inside Patreon's own bootstrap
|
|
36
|
+
# payload. Brace match them rather than regexing individual fields: the
|
|
37
|
+
# only count-shaped key in there is declined_patron_count, which is
|
|
38
|
+
# declined payments, NOT the tier's patron count. Patreon does not publish
|
|
39
|
+
# a per tier patron count on the public page.
|
|
40
|
+
_ATTRS = re.compile(r'"attributes":\{')
|
|
41
|
+
|
|
42
|
+
|
|
43
|
+
class ScrapingBeeError(RuntimeError):
|
|
44
|
+
def __init__(self, message: str, status: Optional[int] = None,
|
|
45
|
+
payload: Any = None) -> None:
|
|
46
|
+
super().__init__(message)
|
|
47
|
+
self.status = status
|
|
48
|
+
self.payload = payload
|
|
49
|
+
|
|
50
|
+
|
|
51
|
+
class PatreonScraper:
|
|
52
|
+
"""Four routes to public Patreon creator data.
|
|
53
|
+
|
|
54
|
+
>>> bee = PatreonScraper("YOUR_API_KEY")
|
|
55
|
+
>>> bee.creator("kurzgesagt")["creator"]
|
|
56
|
+
'Kurzgesagt – In a Nutshell'
|
|
57
|
+
|
|
58
|
+
Credit cost, measured live: 5 for ``creator``, ``profile`` and
|
|
59
|
+
``tiers`` (auto mode settles on the JavaScript rung), 30 for
|
|
60
|
+
``entry_price`` (premium plus JavaScript at 25, plus 5 for the AI
|
|
61
|
+
query), 0 for a rejected request.
|
|
62
|
+
"""
|
|
63
|
+
|
|
64
|
+
def __init__(self, api_key: str, base_url: str = BASE_URL,
|
|
65
|
+
timeout: int = 120,
|
|
66
|
+
session: Optional[requests.Session] = None) -> None:
|
|
67
|
+
if not api_key:
|
|
68
|
+
raise ValueError("api_key is required")
|
|
69
|
+
self.api_key = api_key
|
|
70
|
+
self.base_url = base_url
|
|
71
|
+
self.timeout = timeout
|
|
72
|
+
self._session = session or requests.Session()
|
|
73
|
+
self.last_cost: Optional[int] = None
|
|
74
|
+
|
|
75
|
+
# ------------------------------------------------------------------ core
|
|
76
|
+
|
|
77
|
+
def _fetch(self, slug: str, **params: Any) -> requests.Response:
|
|
78
|
+
url = slug if slug.startswith("http") else f"https://www.patreon.com/{slug}"
|
|
79
|
+
clean = {k: v for k, v in params.items() if v is not None}
|
|
80
|
+
for key, value in list(clean.items()):
|
|
81
|
+
if isinstance(value, bool):
|
|
82
|
+
clean[key] = "true" if value else "false"
|
|
83
|
+
|
|
84
|
+
response = self._session.get(
|
|
85
|
+
self.base_url,
|
|
86
|
+
headers={"Authorization": f"Bearer {self.api_key}"},
|
|
87
|
+
params={"url": url, **clean},
|
|
88
|
+
timeout=self.timeout,
|
|
89
|
+
)
|
|
90
|
+
cost = response.headers.get("spb-cost")
|
|
91
|
+
self.last_cost = int(cost) if cost and cost.isdigit() else None
|
|
92
|
+
|
|
93
|
+
if response.status_code == 400:
|
|
94
|
+
raise ScrapingBeeError(
|
|
95
|
+
"API rejected the request", 400, response.text[:500]
|
|
96
|
+
)
|
|
97
|
+
response.raise_for_status()
|
|
98
|
+
return response
|
|
99
|
+
|
|
100
|
+
# ---------------------------------------------------------------- routes
|
|
101
|
+
|
|
102
|
+
def creator(self, slug: str) -> Dict[str, Any]:
|
|
103
|
+
"""Route 1: meta tags. 5 credits. The route that does not break.
|
|
104
|
+
|
|
105
|
+
Splits the page title into creator name and campaign tagline.
|
|
106
|
+
Patreon joins the two with an em dash, while the creator name may
|
|
107
|
+
itself contain an en dash, so the split is unambiguous.
|
|
108
|
+
"""
|
|
109
|
+
data = self._fetch(
|
|
110
|
+
slug, extract_rules=json.dumps(_META_RULES), mode="auto"
|
|
111
|
+
).json()
|
|
112
|
+
title = data.get("title") or ""
|
|
113
|
+
if "—" in title:
|
|
114
|
+
_, _, tail = title.partition("—")
|
|
115
|
+
data["campaign_tagline"] = tail.split("|")[0].strip()
|
|
116
|
+
return data
|
|
117
|
+
|
|
118
|
+
def profile(self, slug: str) -> Optional[Dict[str, Any]]:
|
|
119
|
+
"""Route 2: the application/ld+json ProfilePage block. 5 credits.
|
|
120
|
+
|
|
121
|
+
Carries the full about text, which the meta description truncates,
|
|
122
|
+
plus both image sizes.
|
|
123
|
+
|
|
124
|
+
Note that ``extract_rules`` cannot read script tag contents. A rule
|
|
125
|
+
selecting ``script[type="application/ld+json"]`` returns None, so
|
|
126
|
+
this fetches the page and parses it instead. Match on ``@type``,
|
|
127
|
+
never on block index: the live page carried six blocks, including an
|
|
128
|
+
Organization block describing Patreon itself.
|
|
129
|
+
"""
|
|
130
|
+
html = self._fetch(slug, mode="auto").text
|
|
131
|
+
for block in _LD_BLOCK.findall(html):
|
|
132
|
+
try:
|
|
133
|
+
obj = json.loads(block)
|
|
134
|
+
except json.JSONDecodeError:
|
|
135
|
+
continue
|
|
136
|
+
if isinstance(obj, dict) and obj.get("@type") == "ProfilePage":
|
|
137
|
+
person = obj.get("mainEntity") or {}
|
|
138
|
+
image = person.get("image") or {}
|
|
139
|
+
return {
|
|
140
|
+
"name": person.get("name"),
|
|
141
|
+
"alternate_name": person.get("alternateName"),
|
|
142
|
+
"url": person.get("url"),
|
|
143
|
+
"about": person.get("description"),
|
|
144
|
+
"image": image.get("contentUrl"),
|
|
145
|
+
"thumbnail": image.get("thumbnailUrl"),
|
|
146
|
+
}
|
|
147
|
+
return None
|
|
148
|
+
|
|
149
|
+
def entry_price(self, slug: str) -> Dict[str, Any]:
|
|
150
|
+
"""Route 3: ai_extract_rules. 30 credits.
|
|
151
|
+
|
|
152
|
+
Returns the entry price teaser Patreon renders on the campaign
|
|
153
|
+
page, for example "Access exclusive benefits starting at
|
|
154
|
+
$3.14/month". It does not return a structured tier array, because
|
|
155
|
+
the tier cards are mounted by a component that has not rendered at
|
|
156
|
+
capture time. Use :meth:`tiers` when you need every tier.
|
|
157
|
+
"""
|
|
158
|
+
rules = {
|
|
159
|
+
"creator_name": "the creator display name",
|
|
160
|
+
"about": "the about text",
|
|
161
|
+
"membership_tiers": {
|
|
162
|
+
"type": "list",
|
|
163
|
+
"description": "each tier with name and monthly price",
|
|
164
|
+
},
|
|
165
|
+
}
|
|
166
|
+
return self._fetch(
|
|
167
|
+
slug,
|
|
168
|
+
ai_extract_rules=json.dumps(rules),
|
|
169
|
+
render_js=True,
|
|
170
|
+
premium_proxy=True,
|
|
171
|
+
).json()
|
|
172
|
+
|
|
173
|
+
def tiers(self, slug: str) -> Dict[str, Any]:
|
|
174
|
+
"""Route 4: the bootstrap payload. 5 credits. Brittle by design.
|
|
175
|
+
|
|
176
|
+
Patreon ships its own application state into the page as double
|
|
177
|
+
escaped JSON, and the full tier list lives there. Live call on
|
|
178
|
+
``kurzgesagt`` returned four tiers: Free, Trainee Producer at
|
|
179
|
+
``amount_cents`` 314, Producer at 1500, Senior Producer at 4200.
|
|
180
|
+
That 314 cross checks against the ``$3.14/month`` teaser
|
|
181
|
+
:meth:`entry_price` reads off the rendered page.
|
|
182
|
+
|
|
183
|
+
Each tier carries ``title``, ``amount_cents``, ``currency``,
|
|
184
|
+
``is_free_tier``, ``post_count``, ``published``,
|
|
185
|
+
``requires_shipping``, ``description``, ``url``, ``image_url``,
|
|
186
|
+
``discord_role_ids``, ``remaining`` and ``user_limit``.
|
|
187
|
+
|
|
188
|
+
There is no per tier patron count. The only count-shaped field is
|
|
189
|
+
``declined_patron_count``, which counts declined payments and is
|
|
190
|
+
passed through unchanged rather than relabelled.
|
|
191
|
+
|
|
192
|
+
This reads a private format Patreon has no obligation to keep
|
|
193
|
+
stable, so it falls back to :meth:`creator` when nothing matches
|
|
194
|
+
rather than returning silently empty.
|
|
195
|
+
"""
|
|
196
|
+
html = self._fetch(slug, mode="auto").text
|
|
197
|
+
raw = html.encode("utf-8", "ignore").decode("unicode_escape", "ignore")
|
|
198
|
+
|
|
199
|
+
tiers: List[Dict[str, Any]] = []
|
|
200
|
+
for match in _ATTRS.finditer(raw):
|
|
201
|
+
start = match.end() - 1
|
|
202
|
+
depth = 0
|
|
203
|
+
for i in range(start, min(start + 8000, len(raw))):
|
|
204
|
+
if raw[i] == "{":
|
|
205
|
+
depth += 1
|
|
206
|
+
elif raw[i] == "}":
|
|
207
|
+
depth -= 1
|
|
208
|
+
if depth == 0:
|
|
209
|
+
blob = raw[start:i + 1]
|
|
210
|
+
if "declined_patron_count" in blob:
|
|
211
|
+
try:
|
|
212
|
+
tiers.append(json.loads(blob))
|
|
213
|
+
except json.JSONDecodeError:
|
|
214
|
+
pass
|
|
215
|
+
break
|
|
216
|
+
|
|
217
|
+
if not tiers:
|
|
218
|
+
return {"tiers": [], "matched": False, "fallback": self.creator(slug)}
|
|
219
|
+
|
|
220
|
+
tiers.sort(key=lambda t: t.get("amount_cents") or 0)
|
|
221
|
+
return {"tiers": tiers, "matched": True, "count": len(tiers)}
|
|
222
|
+
|
|
223
|
+
# -------------------------------------------------------------- helpers
|
|
224
|
+
|
|
225
|
+
def usage(self) -> Dict[str, Any]:
|
|
226
|
+
"""Account credit usage. Free."""
|
|
227
|
+
response = self._session.get(
|
|
228
|
+
"https://app.scrapingbee.com/api/v1/usage",
|
|
229
|
+
headers={"Authorization": f"Bearer {self.api_key}"},
|
|
230
|
+
timeout=self.timeout,
|
|
231
|
+
)
|
|
232
|
+
response.raise_for_status()
|
|
233
|
+
return response.json()
|
|
234
|
+
|
|
235
|
+
def entry_tier(self, slug: str) -> Optional[Dict[str, Any]]:
|
|
236
|
+
"""The cheapest paid tier, or None when route 4 did not match.
|
|
237
|
+
|
|
238
|
+
Skips the free tier, since ``amount_cents`` 0 is not an entry price.
|
|
239
|
+
"""
|
|
240
|
+
result = self.tiers(slug)
|
|
241
|
+
if not result.get("matched"):
|
|
242
|
+
return None
|
|
243
|
+
paid = [t for t in result["tiers"] if not t.get("is_free_tier")]
|
|
244
|
+
return paid[0] if paid else None
|
|
245
|
+
|
|
246
|
+
def price_ladder(self, slug: str) -> List[Dict[str, Any]]:
|
|
247
|
+
"""Tier titles with prices in major units, cheapest first."""
|
|
248
|
+
result = self.tiers(slug)
|
|
249
|
+
if not result.get("matched"):
|
|
250
|
+
return []
|
|
251
|
+
return [
|
|
252
|
+
{
|
|
253
|
+
"title": t.get("title"),
|
|
254
|
+
"price": (t.get("amount_cents") or 0) / 100,
|
|
255
|
+
"currency": t.get("currency"),
|
|
256
|
+
"free": bool(t.get("is_free_tier")),
|
|
257
|
+
"posts": t.get("post_count"),
|
|
258
|
+
}
|
|
259
|
+
for t in result["tiers"]
|
|
260
|
+
]
|
|
@@ -0,0 +1,214 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: patreon-scraper-api
|
|
3
|
+
Version: 0.0.1
|
|
4
|
+
Summary: Patreon scraper API client: public creator pages, entry tiers, about text and structured profile data via ScrapingBee.
|
|
5
|
+
Author: wordstotech
|
|
6
|
+
License: MIT
|
|
7
|
+
Project-URL: Homepage, https://github.com/ScrapingBee/patreon-api
|
|
8
|
+
Project-URL: Repository, https://github.com/ScrapingBee/patreon-api
|
|
9
|
+
Project-URL: Documentation, https://www.scrapingbee.com/documentation/
|
|
10
|
+
Keywords: patreon scraper,patreon api,scraper for patreon,creator economy,web scraping,scrapingbee
|
|
11
|
+
Classifier: Development Status :: 4 - Beta
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
14
|
+
Classifier: Operating System :: OS Independent
|
|
15
|
+
Classifier: Programming Language :: Python :: 3
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
21
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
22
|
+
Classifier: Topic :: Internet :: WWW/HTTP
|
|
23
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
24
|
+
Classifier: Topic :: Text Processing :: Markup :: HTML
|
|
25
|
+
Requires-Python: >=3.8
|
|
26
|
+
Description-Content-Type: text/markdown
|
|
27
|
+
License-File: LICENSE
|
|
28
|
+
Requires-Dist: requests>=2.25.0
|
|
29
|
+
Dynamic: license-file
|
|
30
|
+
|
|
31
|
+
# patreon-scraper-api
|
|
32
|
+
|
|
33
|
+
A Python client for scraping public Patreon creator pages through ScrapingBee. Four extraction routes, each with a documented cost and a documented failure mode.
|
|
34
|
+
|
|
35
|
+
**Verified against `https://www.patreon.com/kurzgesagt` on 2026-09-10.** Every field name, credit figure and return value below came from a real call. Two of the four routes have caveats that only a live run exposes, and both are written down rather than smoothed over.
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
pip install patreon-scraper-api
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
Requires Python 3.8 or newer and `requests`.
|
|
42
|
+
|
|
43
|
+
## What is in scope
|
|
44
|
+
|
|
45
|
+
Public creator pages, read anonymously.
|
|
46
|
+
|
|
47
|
+
**Reachable:** creator display name, campaign tagline, the full about text, avatar and banner URLs, canonical URL, the complete tier list with prices and per tier post counts, and the entry price teaser.
|
|
48
|
+
|
|
49
|
+
**Not reachable, by design:** patron only posts, attachment downloads, member lists, individual pledge amounts and direct messages. Those require a signed in session, and scraping under login credentials is prohibited by ScrapingBee's terms of service. No parameter in this package changes that.
|
|
50
|
+
|
|
51
|
+
## Authentication
|
|
52
|
+
|
|
53
|
+
```python
|
|
54
|
+
from patreon_scraper_api import PatreonScraper
|
|
55
|
+
|
|
56
|
+
bee = PatreonScraper("YOUR_API_KEY")
|
|
57
|
+
```
|
|
58
|
+
|
|
59
|
+
Sent as `Authorization: Bearer YOUR_API_KEY` on every request. The `api_key` query parameter still answers but the current documentation marks it deprecated.
|
|
60
|
+
|
|
61
|
+
Key and 1,000 free credits: [ScrapingBee](https://www.scrapingbee.com/). Landing page for this target: [Patreon scraper API](https://www.scrapingbee.com/scrapers/patreon-api/).
|
|
62
|
+
|
|
63
|
+
---
|
|
64
|
+
|
|
65
|
+
## Method reference
|
|
66
|
+
|
|
67
|
+
### `creator(slug)`
|
|
68
|
+
|
|
69
|
+
**5 credits.** Meta tags, via `extract_rules`. The route that does not break, because these tags exist for social preview cards rather than for the application.
|
|
70
|
+
|
|
71
|
+
```python
|
|
72
|
+
bee.creator("kurzgesagt")
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
```python
|
|
76
|
+
{
|
|
77
|
+
'creator': 'Kurzgesagt – In a Nutshell',
|
|
78
|
+
'title': 'Kurzgesagt – In a Nutshell — Creating Science Animation Videos | Patreon',
|
|
79
|
+
'og_title': 'Kurzgesagt – In a Nutshell — Creating Science Animation Videos',
|
|
80
|
+
'og_desc': 'Get more from Kurzgesagt – In a Nutshell on Patreon. Creating Science '
|
|
81
|
+
'Animation Videos. Support Kurzgesagt – In a Nutshell and get exclusive '
|
|
82
|
+
'access to their work.',
|
|
83
|
+
'campaign_tagline': 'Creating Science Animation Videos',
|
|
84
|
+
}
|
|
85
|
+
```
|
|
86
|
+
|
|
87
|
+
`campaign_tagline` is derived, not scraped. Patreon joins the creator name and the tagline with an em dash, while a creator name may itself contain an en dash, so splitting on the em dash is unambiguous. The client does that split for you.
|
|
88
|
+
|
|
89
|
+
`mode=auto` walks the proxy ladder and bills only the rung that worked, which was the JavaScript rung at 5 credits. `spb-initial-status-code` reported `308`, which is Patreon's canonical redirect rather than a block.
|
|
90
|
+
|
|
91
|
+
### `profile(slug)`
|
|
92
|
+
|
|
93
|
+
**5 credits.** The `application/ld+json` `ProfilePage` block, parsed after the fetch.
|
|
94
|
+
|
|
95
|
+
```python
|
|
96
|
+
bee.profile("kurzgesagt")
|
|
97
|
+
# {'name': 'Kurzgesagt – In a Nutshell',
|
|
98
|
+
# 'alternate_name': 'Kurzgesagt',
|
|
99
|
+
# 'url': 'https://www.patreon.com/Kurzgesagt',
|
|
100
|
+
# 'about': '<4,476 characters>',
|
|
101
|
+
# 'image': 'https://c10.patreonusercontent.com/...',
|
|
102
|
+
# 'thumbnail': 'https://c10.patreonusercontent.com/...'}
|
|
103
|
+
```
|
|
104
|
+
|
|
105
|
+
Use this over `creator()` when you need the complete about copy, for search indexing or embeddings. The meta description truncates it. The live page carried 4,476 characters here against roughly 160 in `og_desc`.
|
|
106
|
+
|
|
107
|
+
**Why this parses HTML instead of using `extract_rules`:** because `extract_rules` cannot reach script tag contents. A rule of `{"jsonld": {"selector": "script[type=\"application/ld+json\"]", "output": "@text"}}` returns `None`. That was tested directly, not assumed. The client fetches the page and parses the block itself.
|
|
108
|
+
|
|
109
|
+
The live page carried six such blocks, and their order is not stable, so the client matches on `@type`. One of them is an `Organization` block describing Patreon rather than the creator, which is exactly the row an index based parser would pick up by mistake.
|
|
110
|
+
|
|
111
|
+
### `tiers(slug)`
|
|
112
|
+
|
|
113
|
+
**5 credits.** The complete tier list, from Patreon's own bootstrap payload.
|
|
114
|
+
|
|
115
|
+
```python
|
|
116
|
+
result = bee.tiers("kurzgesagt")
|
|
117
|
+
result["matched"] # True
|
|
118
|
+
result["count"] # 4
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Live output, sorted cheapest first:
|
|
122
|
+
|
|
123
|
+
| `title` | `amount_cents` | `is_free_tier` | `post_count` | `declined_patron_count` |
|
|
124
|
+
|---|---|---|---|---|
|
|
125
|
+
| Free | 0 | True | 8 | 2 |
|
|
126
|
+
| Trainee Producer | 314 | False | 1 | 31 |
|
|
127
|
+
| Producer | 1500 | False | 62 | 14 |
|
|
128
|
+
| Senior Producer | 4200 | False | 58 | 0 |
|
|
129
|
+
|
|
130
|
+
Every key on a tier object: `title`, `amount_cents`, `currency`, `description`, `url`, `image_url`, `is_free_tier`, `post_count`, `published`, `requires_shipping`, `declined_patron_count`, `patron_amount_cents`, `patron_currency`, `discord_role_ids`, `remaining`, `user_limit`.
|
|
131
|
+
|
|
132
|
+
**`declined_patron_count` is not a patron count.** It counts declined payments. It reads 2, 31, 14 and 0 across those tiers, which is nothing like a membership figure for a creator of that size. Patreon does not publish a per tier patron count on the public page, so this package passes the field through under its real name rather than relabelling it as something more useful.
|
|
133
|
+
|
|
134
|
+
`amount_cents` with `currency` is the tier price and is the pair to trust. `patron_amount_cents` and `patron_currency` also appear, reading `800` and `DKK` on the free tier, which does not correspond to the tier price, so they are returned raw without interpretation.
|
|
135
|
+
|
|
136
|
+
`remaining` and `user_limit` were both `None` on every tier here. They carry real values on creators who cap a tier.
|
|
137
|
+
|
|
138
|
+
This route reads a private application format Patreon has no obligation to keep stable. When the pattern stops matching, the client returns `{"tiers": [], "matched": False, "fallback": <creator() result>}` rather than an empty list that reads like a creator with no tiers.
|
|
139
|
+
|
|
140
|
+
### `price_ladder(slug)` and `entry_tier(slug)`
|
|
141
|
+
|
|
142
|
+
Convenience wrappers over `tiers()`, same 5 credits.
|
|
143
|
+
|
|
144
|
+
```python
|
|
145
|
+
bee.price_ladder("kurzgesagt")
|
|
146
|
+
# [{'title': 'Free', 'price': 0.0, 'currency': 'USD', 'free': True, 'posts': 8},
|
|
147
|
+
# {'title': 'Trainee Producer', 'price': 3.14, 'currency': 'USD', 'free': False, 'posts': 1},
|
|
148
|
+
# {'title': 'Producer', 'price': 15.0, 'currency': 'USD', 'free': False, 'posts': 62},
|
|
149
|
+
# {'title': 'Senior Producer', 'price': 42.0, 'currency': 'USD', 'free': False, 'posts': 58}]
|
|
150
|
+
|
|
151
|
+
bee.entry_tier("kurzgesagt")["title"] # 'Trainee Producer'
|
|
152
|
+
```
|
|
153
|
+
|
|
154
|
+
`entry_tier()` skips the free tier, because `amount_cents` of 0 is not an entry price.
|
|
155
|
+
|
|
156
|
+
### `entry_price(slug)`
|
|
157
|
+
|
|
158
|
+
**30 credits.** Premium proxy plus JavaScript at 25, plus 5 for the AI query. Uses `ai_extract_rules`, so there is no parser to maintain.
|
|
159
|
+
|
|
160
|
+
```python
|
|
161
|
+
bee.entry_price("kurzgesagt")
|
|
162
|
+
# {'creator_name': 'Kurzgesagt – In a Nutshell',
|
|
163
|
+
# 'about': '<full about text>',
|
|
164
|
+
# 'membership_tiers': ['Access exclusive benefits starting at $3.14/month']}
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
`creator_name` and `about` come back complete and correct. `membership_tiers` comes back as **a single teaser string, not a tier array**, because the tier cards are mounted by a component that has not rendered at capture time. The model described what was actually on the page, which is the honest result rather than a broken one.
|
|
168
|
+
|
|
169
|
+
That said, `$3.14` matches the `amount_cents` value of `314` that `tiers()` reads out of the payload, so the two routes independently agree on the entry price. Use `entry_price()` when you want that one number and no parser. Use `tiers()` when you want the whole ladder.
|
|
170
|
+
|
|
171
|
+
### `usage()`
|
|
172
|
+
|
|
173
|
+
Free. Account credits, concurrency and renewal date.
|
|
174
|
+
|
|
175
|
+
---
|
|
176
|
+
|
|
177
|
+
## Choosing a route
|
|
178
|
+
|
|
179
|
+
| You need | Method | Credits |
|
|
180
|
+
|---|---|---|
|
|
181
|
+
| Name and tagline for a directory | `creator` | 5 |
|
|
182
|
+
| Full about copy for search or embeddings | `profile` | 5 |
|
|
183
|
+
| Every tier with its price and post count | `tiers` | 5 |
|
|
184
|
+
| A price ladder ready to display | `price_ladder` | 5 |
|
|
185
|
+
| Entry price only, no parser to own | `entry_price` | 30 |
|
|
186
|
+
|
|
187
|
+
`creator()` and `profile()` read the same fetched page, so if you want both, fetch raw once and run both parsers locally for a single 5 credit charge.
|
|
188
|
+
|
|
189
|
+
## Credit cost
|
|
190
|
+
|
|
191
|
+
Measured from `spb-cost` headers. Available on `bee.last_cost` after every call.
|
|
192
|
+
|
|
193
|
+
| Configuration | Credits |
|
|
194
|
+
|---|---|
|
|
195
|
+
| `mode=auto` on a creator page, settled at the JavaScript rung | 5 |
|
|
196
|
+
| `render_js` plus `premium_proxy` | 25 |
|
|
197
|
+
| The same plus `ai_extract_rules` | 30 |
|
|
198
|
+
| Validation error | 0 |
|
|
199
|
+
|
|
200
|
+
Auto mode is the right default here: it charges only for the configuration that worked, and nothing at all if every rung fails. It is incompatible with `render_js`, `premium_proxy` and `stealth_proxy`, and sending both returns HTTP 400 while billing nothing, which fails quietly if you do not read status codes.
|
|
201
|
+
|
|
202
|
+
At 5 credits per creator, 250,000 credits covers 50,000 creator checks. Plan tiers: [ScrapingBee pricing](https://www.scrapingbee.com/pricing).
|
|
203
|
+
|
|
204
|
+
## Related
|
|
205
|
+
|
|
206
|
+
Landing pages for adjacent creator platforms: [Substack scraper API](https://www.scrapingbee.com/scrapers/substack-scraper-api/), [Twitch API](https://www.scrapingbee.com/scrapers/twitch-api/), [TikTok API](https://www.scrapingbee.com/scrapers/tiktok-api/), [TikTok follower API](https://www.scrapingbee.com/scrapers/tiktok-follower/), [YouTube video scraper API](https://www.scrapingbee.com/scrapers/youtube-video-scraper-api/), [YouTube transcript scraper API](https://www.scrapingbee.com/scrapers/youtube-transcript-scraper-api/), [YouTube comment scraper API](https://www.scrapingbee.com/scrapers/youtube-comment-scraper-api/), [Snapchat scraper API](https://www.scrapingbee.com/scrapers/snapchat-scraper-api/).
|
|
207
|
+
|
|
208
|
+
Features: [AI web scraping](https://www.scrapingbee.com/features/ai-web-scraping-api/), [data extraction](https://www.scrapingbee.com/features/data-extraction/), [markdown scraper](https://www.scrapingbee.com/features/markdown-scraper/), [screenshots](https://www.scrapingbee.com/features/screenshot/), [JavaScript scenario](https://www.scrapingbee.com/features/javascript-scenario/), [n8n integration](https://www.scrapingbee.com/features/n8n/).
|
|
209
|
+
|
|
210
|
+
Reference: [extraction rules documentation](https://www.scrapingbee.com/documentation/data-extraction/). Guide version with the full route walkthrough: [github.com/ScrapingBee/patreon-api](https://github.com/ScrapingBee/patreon-api).
|
|
211
|
+
|
|
212
|
+
## License
|
|
213
|
+
|
|
214
|
+
MIT
|
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
LICENSE
|
|
2
|
+
MANIFEST.in
|
|
3
|
+
README.md
|
|
4
|
+
pyproject.toml
|
|
5
|
+
src/patreon_scraper_api/__init__.py
|
|
6
|
+
src/patreon_scraper_api/client.py
|
|
7
|
+
src/patreon_scraper_api.egg-info/PKG-INFO
|
|
8
|
+
src/patreon_scraper_api.egg-info/SOURCES.txt
|
|
9
|
+
src/patreon_scraper_api.egg-info/dependency_links.txt
|
|
10
|
+
src/patreon_scraper_api.egg-info/requires.txt
|
|
11
|
+
src/patreon_scraper_api.egg-info/top_level.txt
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
requests>=2.25.0
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
patreon_scraper_api
|