devart-dl 4.2.0__tar.gz → 4.3.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {devart_dl-4.2.0/devart_dl.egg-info → devart_dl-4.3.1}/PKG-INFO +20 -5
- devart_dl-4.3.1/README.en.md +87 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/README.md +19 -4
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/__init__.py +4 -2
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/cli.py +38 -2
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/download.py +68 -12
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/models.py +97 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/official_api.py +5 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/provider.py +60 -1
- {devart_dl-4.2.0 → devart_dl-4.3.1/devart_dl.egg-info}/PKG-INFO +20 -5
- {devart_dl-4.2.0 → devart_dl-4.3.1}/devart_dl.egg-info/SOURCES.txt +1 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/pyproject.toml +1 -1
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_cli.py +26 -0
- devart_dl-4.3.1/tests/test_conformance.py +27 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_download.py +76 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_official_provider.py +40 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_provider.py +102 -2
- devart_dl-4.2.0/README.en.md +0 -282
- {devart_dl-4.2.0 → devart_dl-4.3.1}/.env.example +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/LICENSE +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/MANIFEST.in +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/bin/devart-dl +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/bin/install.sh +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/api.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/archive.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/auth.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/da_cookie_login.mjs +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/errors.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/http.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/manager.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/oauth.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/path.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/da_downloader/targets.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/devart_dl.egg-info/dependency_links.txt +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/devart_dl.egg-info/entry_points.txt +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/devart_dl.egg-info/requires.txt +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/devart_dl.egg-info/top_level.txt +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/requirements.txt +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/setup.cfg +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/setup.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_api.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_archive.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_http_downloader.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_manager.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_path.py +0 -0
- {devart_dl-4.2.0 → devart_dl-4.3.1}/tests/test_targets.py +0 -0
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: devart-dl
|
|
3
|
-
Version: 4.
|
|
3
|
+
Version: 4.3.1
|
|
4
4
|
Summary: 专业的 DeviantArt 作品批量下载工具 - Professional DeviantArt batch downloader with anti-ban features
|
|
5
5
|
Author: DeviantArt Downloader Team
|
|
6
6
|
License-Expression: MIT
|
|
@@ -34,9 +34,11 @@ Dynamic: license-file
|
|
|
34
34
|
|
|
35
35
|
# DeviantArt Downloader
|
|
36
36
|
|
|
37
|
-
|
|
37
|
+
**语言 / Language:** 中文 · [English](README.en.md)
|
|
38
38
|
|
|
39
|
-
|
|
39
|
+
> **一个可靠、聚焦的 DeviantArt 下载 / 归档命令行工具。**
|
|
40
|
+
|
|
41
|
+
[完整文档](https://redtidev1918.github.io/deviantart-downloader/) · [更新日志](CHANGELOG.md)
|
|
40
42
|
|
|
41
43
|
[](https://github.com/redtidev1918/deviantart-downloader/actions/workflows/ci.yml)
|
|
42
44
|
[](https://pypi.org/project/devart-dl/)
|
|
@@ -61,6 +63,9 @@ devart-dl https://www.deviantart.com/username/gallery
|
|
|
61
63
|
# 收藏夹 / 标签
|
|
62
64
|
devart-dl https://www.deviantart.com/username/favourites
|
|
63
65
|
devart-dl https://www.deviantart.com/tag/landscape
|
|
66
|
+
|
|
67
|
+
# 解析为 JSON 描述(不下载文件)
|
|
68
|
+
devart-dl resolve https://www.deviantart.com/username/art/title-123456
|
|
64
69
|
```
|
|
65
70
|
|
|
66
71
|
首次下载前建议使用 OAuth 登录:
|
|
@@ -88,10 +93,20 @@ devart-dl login browser # 打开你的 Chrome 登录,自动保存 Cookie,
|
|
|
88
93
|
- **下载档案**:`--archive` 用 SQLite 记住已下载,跨会话自动跳过。
|
|
89
94
|
- **路径模板 + 元数据 sidecar**:`--directory`/`--filename` 模板与 `--write-info-json`。
|
|
90
95
|
- **代理与降级**:支持 `--proxy`;未登录 OAuth 时自动回退 Cookie。
|
|
96
|
+
- **Resolve 模式**:`devart-dl resolve URL` 输出 `ResolvedDeviation`/`MediaAsset` 的 JSON 契约,不写文件。
|
|
97
|
+
|
|
98
|
+
## 文档
|
|
99
|
+
|
|
100
|
+
| 你想做什么 | 文档 |
|
|
101
|
+
| --- | --- |
|
|
102
|
+
| 下载安装包 | [下载页](https://redtidev1918.github.io/deviantart-downloader/download.md) |
|
|
103
|
+
| 登录与认证 | [登录与认证](docs/LOGIN.md) |
|
|
104
|
+
| 命令、配置与 Python 调用 | [文档站](https://redtidev1918.github.io/deviantart-downloader/) |
|
|
91
105
|
|
|
92
|
-
|
|
106
|
+
## 致谢
|
|
93
107
|
|
|
94
|
-
https://
|
|
108
|
+
- [requests](https://github.com/psf/requests):HTTP 客户端。
|
|
109
|
+
- 登录与归档目录的实现与 [gallery-dl](https://github.com/mikf/gallery-dl) 交叉核对过。
|
|
95
110
|
|
|
96
111
|
## 许可证与免责声明
|
|
97
112
|
|
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# DeviantArt Downloader
|
|
2
|
+
|
|
3
|
+
**Language / 语言:** [中文](README.md) · English
|
|
4
|
+
|
|
5
|
+
> **A reliable, focused DeviantArt downloader and archival CLI.**
|
|
6
|
+
|
|
7
|
+
📖 [Full documentation](https://redtidev1918.github.io/deviantart-downloader/) · [Changelog](CHANGELOG.md)
|
|
8
|
+
|
|
9
|
+
[](https://github.com/redtidev1918/deviantart-downloader/actions/workflows/ci.yml)
|
|
10
|
+
[](https://pypi.org/project/devart-dl/)
|
|
11
|
+
[](https://pypi.org/project/devart-dl/)
|
|
12
|
+
[](https://pypi.org/project/devart-dl/)
|
|
13
|
+
[](LICENSE)
|
|
14
|
+
|
|
15
|
+
See [CHANGELOG.md](CHANGELOG.md) for release notes.
|
|
16
|
+
|
|
17
|
+
---
|
|
18
|
+
|
|
19
|
+
## Core Features
|
|
20
|
+
|
|
21
|
+
- **Official API first (OAuth)** — log in once to download originals; no cookie export, no anti-ban delays.
|
|
22
|
+
- **URL-first** — `devart-dl URL` downloads artworks / galleries / favourites / tags / fav.me links directly.
|
|
23
|
+
- **Reliable downloads** — HTTP Range resume, automatic retry, 429 backoff, HTML/empty-file validation, atomic finalize.
|
|
24
|
+
- **Download archive** — `--archive` remembers what you've downloaded in SQLite, skipping it across sessions.
|
|
25
|
+
- **Path templates** — `--directory` / `--filename` support `{id}` `{title}` `{author}` `{published}` `{ext}` and more, with safe sanitization.
|
|
26
|
+
- **Metadata sidecars** — `--write-info-json` writes a `.json` next to each file.
|
|
27
|
+
- **Proxy & fallback** — `--proxy` or proxy environment variables; falls back to cookies when not logged in via OAuth.
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
## Quick Start
|
|
32
|
+
|
|
33
|
+
### Installation
|
|
34
|
+
|
|
35
|
+
Requires Python 3.10 or newer.
|
|
36
|
+
|
|
37
|
+
```bash
|
|
38
|
+
pip install devart-dl
|
|
39
|
+
```
|
|
40
|
+
|
|
41
|
+
### Basic Usage (URL-first)
|
|
42
|
+
|
|
43
|
+
```bash
|
|
44
|
+
# Single artwork (also fav.me links and bare artwork ids)
|
|
45
|
+
devart-dl https://www.deviantart.com/username/art/title-123456
|
|
46
|
+
|
|
47
|
+
# Gallery / all works by an artist
|
|
48
|
+
devart-dl https://www.deviantart.com/username/gallery
|
|
49
|
+
|
|
50
|
+
# Favourites
|
|
51
|
+
devart-dl https://www.deviantart.com/username/favourites
|
|
52
|
+
|
|
53
|
+
# Tag
|
|
54
|
+
devart-dl https://www.deviantart.com/tag/landscape
|
|
55
|
+
|
|
56
|
+
# Resolve to JSON descriptors (download nothing)
|
|
57
|
+
devart-dl resolve https://www.deviantart.com/username/art/title-123456
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
### Login (OAuth recommended)
|
|
61
|
+
|
|
62
|
+
```bash
|
|
63
|
+
devart-dl login oauth --client-id YOUR_PUBLIC_CLIENT_ID
|
|
64
|
+
devart-dl whoami # verify login
|
|
65
|
+
devart-dl logout # revoke the token
|
|
66
|
+
```
|
|
67
|
+
|
|
68
|
+
Register a **Public** OAuth app at [deviantart.com/developers](https://www.deviantart.com/developers/) and whitelist `http://127.0.0.1:8765/callback`. Login happens in the browser — no password or `client_secret` is ever given to the CLI.
|
|
69
|
+
|
|
70
|
+
---
|
|
71
|
+
|
|
72
|
+
## Documentation
|
|
73
|
+
|
|
74
|
+
The full install, sign-in, download modes, configuration, command reference, Python API and FAQ live on the [docs site](https://redtidev1918.github.io/deviantart-downloader/).
|
|
75
|
+
|
|
76
|
+
## Acknowledgements
|
|
77
|
+
|
|
78
|
+
- [requests](https://github.com/psf/requests) — the HTTP client.
|
|
79
|
+
- DevTools-free login and archive layout were cross-checked against [gallery-dl](https://github.com/mikf/gallery-dl).
|
|
80
|
+
|
|
81
|
+
## License
|
|
82
|
+
|
|
83
|
+
MIT License — see [LICENSE](LICENSE).
|
|
84
|
+
|
|
85
|
+
**Important disclaimer:** this tool is for personal learning and research only.
|
|
86
|
+
Respect DeviantArt's terms of service and copyright. Do not use it commercially
|
|
87
|
+
or to overload their servers. The author is not responsible for any misuse.
|
|
@@ -1,8 +1,10 @@
|
|
|
1
1
|
# DeviantArt Downloader
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
**语言 / Language:** 中文 · [English](README.en.md)
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
> **一个可靠、聚焦的 DeviantArt 下载 / 归档命令行工具。**
|
|
6
|
+
|
|
7
|
+
[完整文档](https://redtidev1918.github.io/deviantart-downloader/) · [更新日志](CHANGELOG.md)
|
|
6
8
|
|
|
7
9
|
[](https://github.com/redtidev1918/deviantart-downloader/actions/workflows/ci.yml)
|
|
8
10
|
[](https://pypi.org/project/devart-dl/)
|
|
@@ -27,6 +29,9 @@ devart-dl https://www.deviantart.com/username/gallery
|
|
|
27
29
|
# 收藏夹 / 标签
|
|
28
30
|
devart-dl https://www.deviantart.com/username/favourites
|
|
29
31
|
devart-dl https://www.deviantart.com/tag/landscape
|
|
32
|
+
|
|
33
|
+
# 解析为 JSON 描述(不下载文件)
|
|
34
|
+
devart-dl resolve https://www.deviantart.com/username/art/title-123456
|
|
30
35
|
```
|
|
31
36
|
|
|
32
37
|
首次下载前建议使用 OAuth 登录:
|
|
@@ -54,10 +59,20 @@ devart-dl login browser # 打开你的 Chrome 登录,自动保存 Cookie,
|
|
|
54
59
|
- **下载档案**:`--archive` 用 SQLite 记住已下载,跨会话自动跳过。
|
|
55
60
|
- **路径模板 + 元数据 sidecar**:`--directory`/`--filename` 模板与 `--write-info-json`。
|
|
56
61
|
- **代理与降级**:支持 `--proxy`;未登录 OAuth 时自动回退 Cookie。
|
|
62
|
+
- **Resolve 模式**:`devart-dl resolve URL` 输出 `ResolvedDeviation`/`MediaAsset` 的 JSON 契约,不写文件。
|
|
63
|
+
|
|
64
|
+
## 文档
|
|
65
|
+
|
|
66
|
+
| 你想做什么 | 文档 |
|
|
67
|
+
| --- | --- |
|
|
68
|
+
| 下载安装包 | [下载页](https://redtidev1918.github.io/deviantart-downloader/download.md) |
|
|
69
|
+
| 登录与认证 | [登录与认证](docs/LOGIN.md) |
|
|
70
|
+
| 命令、配置与 Python 调用 | [文档站](https://redtidev1918.github.io/deviantart-downloader/) |
|
|
57
71
|
|
|
58
|
-
|
|
72
|
+
## 致谢
|
|
59
73
|
|
|
60
|
-
https://
|
|
74
|
+
- [requests](https://github.com/psf/requests):HTTP 客户端。
|
|
75
|
+
- 登录与归档目录的实现与 [gallery-dl](https://github.com/mikf/gallery-dl) 交叉核对过。
|
|
61
76
|
|
|
62
77
|
## 许可证与免责声明
|
|
63
78
|
|
|
@@ -1,16 +1,18 @@
|
|
|
1
1
|
"""DeviantArt Downloader — a reliable, focused downloader and archival CLI."""
|
|
2
2
|
|
|
3
|
-
__version__ = "4.0
|
|
3
|
+
__version__ = "4.3.0"
|
|
4
4
|
__author__ = "DeviantArt Downloader Team"
|
|
5
5
|
|
|
6
6
|
from .download import Downloader, build_downloader, normalize_quality
|
|
7
7
|
from .errors import DeviantArtError
|
|
8
|
-
from .models import DownloadItem
|
|
8
|
+
from .models import DownloadItem, MediaAsset, ResolvedDeviation
|
|
9
9
|
|
|
10
10
|
__all__ = [
|
|
11
11
|
"Downloader",
|
|
12
12
|
"build_downloader",
|
|
13
13
|
"normalize_quality",
|
|
14
14
|
"DownloadItem",
|
|
15
|
+
"MediaAsset",
|
|
16
|
+
"ResolvedDeviation",
|
|
15
17
|
"DeviantArtError",
|
|
16
18
|
]
|
|
@@ -37,6 +37,7 @@ _HELP = """DeviantArt Downloader
|
|
|
37
37
|
|
|
38
38
|
Usage:
|
|
39
39
|
devart-dl URL [options] # URL-first: artwork/gallery/favourites/tag
|
|
40
|
+
devart-dl resolve URL [options] # resolve to work/media JSON, download nothing
|
|
40
41
|
devart-dl url URL [options]
|
|
41
42
|
devart-dl artist USERNAME [options]
|
|
42
43
|
devart-dl gallery USERNAME [GALLERY_ID] [options]
|
|
@@ -62,7 +63,7 @@ Run `devart-dl login oauth --help` for OAuth details.
|
|
|
62
63
|
"""
|
|
63
64
|
|
|
64
65
|
_KNOWN_COMMANDS = frozenset(
|
|
65
|
-
{"help", "url", "artist", "gallery", "search", "fav", "login", "whoami", "logout"}
|
|
66
|
+
{"help", "url", "artist", "gallery", "search", "fav", "resolve", "login", "whoami", "logout"}
|
|
66
67
|
)
|
|
67
68
|
_REMOVED_COMMANDS = frozenset({"anti-ban", "test"})
|
|
68
69
|
_UUID_RE = re.compile(
|
|
@@ -75,7 +76,7 @@ def _version() -> str:
|
|
|
75
76
|
try:
|
|
76
77
|
return version("devart-dl")
|
|
77
78
|
except PackageNotFoundError:
|
|
78
|
-
return "4.0
|
|
79
|
+
return "4.3.0"
|
|
79
80
|
|
|
80
81
|
|
|
81
82
|
def _looks_like_target(value: str) -> bool:
|
|
@@ -199,6 +200,39 @@ def _run_download(arguments: Sequence[str]) -> int:
|
|
|
199
200
|
)
|
|
200
201
|
|
|
201
202
|
|
|
203
|
+
|
|
204
|
+
def _run_resolve(arguments: Sequence[str]) -> int:
|
|
205
|
+
parser = argparse.ArgumentParser(
|
|
206
|
+
prog="devart-dl resolve",
|
|
207
|
+
description="Resolve a DeviantArt target to work/media JSON (download nothing)",
|
|
208
|
+
)
|
|
209
|
+
parser.add_argument("target", help="DeviantArt URL or artwork id")
|
|
210
|
+
parser.add_argument("--cookies", default=None)
|
|
211
|
+
parser.add_argument("--quality", default="best")
|
|
212
|
+
parser.add_argument("--proxy", default=None)
|
|
213
|
+
parser.add_argument("--timeout", type=float, default=60.0)
|
|
214
|
+
parser.add_argument("--retries", type=int, default=3)
|
|
215
|
+
parser.add_argument("--limit", type=int, default=24)
|
|
216
|
+
args = parser.parse_args(list(arguments))
|
|
217
|
+
try:
|
|
218
|
+
downloader = build_downloader(
|
|
219
|
+
destination=Path("."),
|
|
220
|
+
cookies=_load_cookies(args.cookies),
|
|
221
|
+
quality=args.quality,
|
|
222
|
+
proxy=args.proxy,
|
|
223
|
+
timeout=args.timeout,
|
|
224
|
+
retries=args.retries,
|
|
225
|
+
limit=args.limit,
|
|
226
|
+
)
|
|
227
|
+
works = downloader.resolve(args.target)
|
|
228
|
+
except (ValueError, DeviantArtError) as exc:
|
|
229
|
+
print(f"error: {exc}", file=sys.stderr)
|
|
230
|
+
return 1
|
|
231
|
+
payload = works[0].to_dict() if len(works) == 1 else {"works": [w.to_dict() for w in works]}
|
|
232
|
+
print(json.dumps(payload, ensure_ascii=False, indent=2))
|
|
233
|
+
return 0
|
|
234
|
+
|
|
235
|
+
|
|
202
236
|
def _run_subcommand(command: str, rest: Sequence[str]) -> int:
|
|
203
237
|
positionals: list[str] = []
|
|
204
238
|
options: dict[str, str] = {}
|
|
@@ -406,6 +440,8 @@ def main(argv: Sequence[str] | None = None) -> int:
|
|
|
406
440
|
return 0
|
|
407
441
|
if command in {"url", "artist", "gallery", "search", "fav"}:
|
|
408
442
|
return _run_subcommand(command, rest)
|
|
443
|
+
if command == "resolve":
|
|
444
|
+
return _run_resolve(rest)
|
|
409
445
|
if command == "login":
|
|
410
446
|
if not rest or rest[0] in {"-h", "--help", "help"}:
|
|
411
447
|
print("Usage: devart-dl login oauth|browser|interactive|clear")
|
|
@@ -15,10 +15,16 @@ from .api import DeviantArtAPI
|
|
|
15
15
|
from .archive import DownloadArchive
|
|
16
16
|
from .http import HttpDownloader
|
|
17
17
|
from .manager import DownloadManager, DownloadOutcome
|
|
18
|
+
from .models import ResolvedDeviation, make_media_asset
|
|
18
19
|
from .oauth import OAuthSession
|
|
19
20
|
from .official_api import OfficialApiClient
|
|
20
21
|
from .path import PathFormatter
|
|
21
|
-
from .provider import
|
|
22
|
+
from .provider import (
|
|
23
|
+
CompositeProvider,
|
|
24
|
+
DeviantArtProvider,
|
|
25
|
+
OfficialProvider,
|
|
26
|
+
WebProvider,
|
|
27
|
+
)
|
|
22
28
|
from .targets import DownloadTarget, TargetParser
|
|
23
29
|
|
|
24
30
|
# User-facing quality names (new) and the legacy o/f/p shorthand.
|
|
@@ -50,6 +56,50 @@ class Downloader:
|
|
|
50
56
|
parsed = TargetParser.parse(target) if isinstance(target, str) else target
|
|
51
57
|
return [self.manager.run(item) for item in self.provider.resolve(parsed)]
|
|
52
58
|
|
|
59
|
+
def resolve(self, target: str | DownloadTarget) -> list[ResolvedDeviation]:
|
|
60
|
+
"""Resolve a target to DeviantArt works and media assets (no download)."""
|
|
61
|
+
parsed = TargetParser.parse(target) if isinstance(target, str) else target
|
|
62
|
+
groups: dict[str, dict] = {}
|
|
63
|
+
order: list[str] = []
|
|
64
|
+
for item in self.provider.resolve(parsed):
|
|
65
|
+
extra = item.metadata.get("index") if item.metadata else None
|
|
66
|
+
base = str(item.artwork_id)
|
|
67
|
+
if extra is not None:
|
|
68
|
+
suffix = f"-{extra}"
|
|
69
|
+
if base.endswith(suffix):
|
|
70
|
+
base = base[: -len(suffix)]
|
|
71
|
+
if base not in groups:
|
|
72
|
+
groups[base] = {
|
|
73
|
+
"title": item.title,
|
|
74
|
+
"author": item.author,
|
|
75
|
+
"mature": item.mature,
|
|
76
|
+
"url": item.url,
|
|
77
|
+
"assets": [],
|
|
78
|
+
}
|
|
79
|
+
order.append(base)
|
|
80
|
+
index = int(extra) if extra is not None else len(groups[base]["assets"])
|
|
81
|
+
groups[base]["assets"].append(
|
|
82
|
+
make_media_asset(
|
|
83
|
+
base,
|
|
84
|
+
index,
|
|
85
|
+
item.media_url,
|
|
86
|
+
extension=item.extension,
|
|
87
|
+
mature=item.mature,
|
|
88
|
+
)
|
|
89
|
+
)
|
|
90
|
+
return [
|
|
91
|
+
ResolvedDeviation(
|
|
92
|
+
id=key,
|
|
93
|
+
title=groups[key]["title"],
|
|
94
|
+
author=groups[key]["author"],
|
|
95
|
+
mature=groups[key]["mature"],
|
|
96
|
+
url=groups[key]["url"],
|
|
97
|
+
media=tuple(groups[key]["assets"]),
|
|
98
|
+
)
|
|
99
|
+
for key in order
|
|
100
|
+
]
|
|
101
|
+
|
|
102
|
+
|
|
53
103
|
|
|
54
104
|
def build_downloader(
|
|
55
105
|
*,
|
|
@@ -70,23 +120,20 @@ def build_downloader(
|
|
|
70
120
|
) -> Downloader:
|
|
71
121
|
"""Assemble a wired ``Downloader`` from plain configuration values.
|
|
72
122
|
|
|
73
|
-
|
|
74
|
-
|
|
123
|
+
OAuth and cookies are capabilities, not a global provider mode: the
|
|
124
|
+
official API is preferred where it works and the web session fills the
|
|
125
|
+
capabilities it cannot cover (notably search).
|
|
75
126
|
"""
|
|
76
127
|
normalized_quality = normalize_quality(quality)
|
|
77
128
|
proxies = {"http": proxy, "https": proxy} if proxy else {}
|
|
78
129
|
|
|
79
130
|
oauth_session = OAuthSession.from_store()
|
|
131
|
+
official = web = None
|
|
132
|
+
api = None
|
|
80
133
|
if oauth_session is not None:
|
|
81
134
|
client = OfficialApiClient(oauth_session, timeout=timeout)
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
)
|
|
85
|
-
# Official media URLs are signed CDN links and need no cookies.
|
|
86
|
-
http_session = requests.Session()
|
|
87
|
-
if proxies:
|
|
88
|
-
http_session.proxies.update(proxies)
|
|
89
|
-
else:
|
|
135
|
+
official = OfficialProvider(client, quality=normalized_quality, limit=limit)
|
|
136
|
+
if cookies or oauth_session is None:
|
|
90
137
|
headers = {
|
|
91
138
|
"accept": "application/json, text/plain, */*",
|
|
92
139
|
"user-agent": (
|
|
@@ -99,8 +146,17 @@ def build_downloader(
|
|
|
99
146
|
api = DeviantArtAPI(
|
|
100
147
|
headers, proxies, retry_delay=int(retry_delay), max_retries=retries
|
|
101
148
|
)
|
|
102
|
-
|
|
149
|
+
web = WebProvider(api, quality=normalized_quality, limit=limit)
|
|
150
|
+
|
|
151
|
+
provider: DeviantArtProvider = CompositeProvider(official, web)
|
|
152
|
+
# The web session's cookies are harmless on signed official CDN links and
|
|
153
|
+
# required for web-sourced media (search/gallery fallback), so prefer it.
|
|
154
|
+
if api is not None:
|
|
103
155
|
http_session = api.session
|
|
156
|
+
else:
|
|
157
|
+
http_session = requests.Session()
|
|
158
|
+
if proxies:
|
|
159
|
+
http_session.proxies.update(proxies)
|
|
104
160
|
|
|
105
161
|
http = HttpDownloader(
|
|
106
162
|
session=http_session,
|
|
@@ -113,3 +113,100 @@ class DownloadItem:
|
|
|
113
113
|
published_at: Optional[datetime] = None
|
|
114
114
|
mature: bool = False
|
|
115
115
|
metadata: Mapping[str, Any] = field(default_factory=dict)
|
|
116
|
+
|
|
117
|
+
|
|
118
|
+
_MIME_BY_EXT = {
|
|
119
|
+
"jpg": "image/jpeg",
|
|
120
|
+
"jpeg": "image/jpeg",
|
|
121
|
+
"png": "image/png",
|
|
122
|
+
"gif": "image/gif",
|
|
123
|
+
"webp": "image/webp",
|
|
124
|
+
"bmp": "image/bmp",
|
|
125
|
+
"svg": "image/svg+xml",
|
|
126
|
+
"mp4": "video/mp4",
|
|
127
|
+
"webm": "video/webm",
|
|
128
|
+
"mov": "video/quicktime",
|
|
129
|
+
}
|
|
130
|
+
|
|
131
|
+
_VIDEO_EXTS = frozenset({"mp4", "webm", "mov", "m4v"})
|
|
132
|
+
|
|
133
|
+
_DEFAULT_MIME = {"image": "image/jpeg", "video": "video/mp4"}
|
|
134
|
+
|
|
135
|
+
|
|
136
|
+
@dataclass(frozen=True)
|
|
137
|
+
class MediaAsset:
|
|
138
|
+
"""One resolute media file of a DeviantArt work (``deviantart:<uuid>:p<n>``)."""
|
|
139
|
+
|
|
140
|
+
id: str
|
|
141
|
+
deviation_id: str
|
|
142
|
+
index: int
|
|
143
|
+
kind: str # "image" | "video"
|
|
144
|
+
source_url: str
|
|
145
|
+
extension: str | None = None
|
|
146
|
+
mime_type: str | None = None
|
|
147
|
+
width: int | None = None
|
|
148
|
+
height: int | None = None
|
|
149
|
+
mature: bool = False
|
|
150
|
+
|
|
151
|
+
def to_dict(self) -> dict:
|
|
152
|
+
data: dict = {
|
|
153
|
+
"id": self.id,
|
|
154
|
+
"kind": self.kind,
|
|
155
|
+
"sourceUrl": self.source_url,
|
|
156
|
+
}
|
|
157
|
+
for key, value in (
|
|
158
|
+
("extension", self.extension),
|
|
159
|
+
("mimeType", self.mime_type),
|
|
160
|
+
("width", self.width),
|
|
161
|
+
("height", self.height),
|
|
162
|
+
("mature", self.mature),
|
|
163
|
+
):
|
|
164
|
+
if value is not None:
|
|
165
|
+
data[key] = value
|
|
166
|
+
return data
|
|
167
|
+
|
|
168
|
+
|
|
169
|
+
@dataclass(frozen=True)
|
|
170
|
+
class ResolvedDeviation:
|
|
171
|
+
"""A DeviantArt work plus its media assets, independent of downloading."""
|
|
172
|
+
|
|
173
|
+
id: str
|
|
174
|
+
title: str
|
|
175
|
+
author: str
|
|
176
|
+
mature: bool
|
|
177
|
+
media: tuple[MediaAsset, ...] = ()
|
|
178
|
+
url: str = ""
|
|
179
|
+
|
|
180
|
+
def to_dict(self) -> dict:
|
|
181
|
+
return {
|
|
182
|
+
"work": {
|
|
183
|
+
"id": self.id,
|
|
184
|
+
"title": self.title,
|
|
185
|
+
"author": self.author,
|
|
186
|
+
"url": self.url,
|
|
187
|
+
"mature": self.mature,
|
|
188
|
+
},
|
|
189
|
+
"media": [asset.to_dict() for asset in self.media],
|
|
190
|
+
}
|
|
191
|
+
|
|
192
|
+
|
|
193
|
+
def make_media_asset(
|
|
194
|
+
deviation_id: str,
|
|
195
|
+
index: int,
|
|
196
|
+
source_url: str,
|
|
197
|
+
*,
|
|
198
|
+
extension: str | None = None,
|
|
199
|
+
mature: bool = False,
|
|
200
|
+
) -> MediaAsset:
|
|
201
|
+
ext = (extension or "").lstrip(".").lower() or None
|
|
202
|
+
kind = "video" if ext in _VIDEO_EXTS else "image"
|
|
203
|
+
return MediaAsset(
|
|
204
|
+
id=f"deviantart:{deviation_id}:p{index}",
|
|
205
|
+
deviation_id=deviation_id,
|
|
206
|
+
index=index,
|
|
207
|
+
kind=kind,
|
|
208
|
+
source_url=source_url,
|
|
209
|
+
extension=ext,
|
|
210
|
+
mime_type=_MIME_BY_EXT.get(ext) or _DEFAULT_MIME[kind],
|
|
211
|
+
mature=mature,
|
|
212
|
+
)
|
|
@@ -320,6 +320,9 @@ def additional_media_urls(init_data: dict) -> list:
|
|
|
320
320
|
"""Original-file URLs of a multimedia deviation's extra pages
|
|
321
321
|
(``deviation.extended.additionalMedia``, each entry nests its Wix
|
|
322
322
|
descriptor under ``media``). Prefers the raw ``baseUri`` file + token.
|
|
323
|
+
|
|
324
|
+
Mature pages the server still returns with a ``blur_`` censored image are
|
|
325
|
+
skipped individually; one blocked page does not discard the others.
|
|
323
326
|
"""
|
|
324
327
|
extended = init_data.get("deviation", {}).get("extended") or {}
|
|
325
328
|
entries = extended.get("additionalMedia") or []
|
|
@@ -333,6 +336,8 @@ def additional_media_urls(init_data: dict) -> list:
|
|
|
333
336
|
base = media.get("baseUri")
|
|
334
337
|
if not isinstance(base, str) or not base:
|
|
335
338
|
continue
|
|
339
|
+
if "blur_" in base:
|
|
340
|
+
continue
|
|
336
341
|
raw_token = media.get("token")
|
|
337
342
|
token = raw_token[0] if isinstance(raw_token, list) and raw_token else raw_token
|
|
338
343
|
if token:
|
|
@@ -13,6 +13,7 @@ from typing import Any, Callable, Iterator, Optional, Protocol
|
|
|
13
13
|
|
|
14
14
|
from .api import ActionType, DeviantArtAPI
|
|
15
15
|
from .errors import MediaUnavailableError, ParseError
|
|
16
|
+
from .errors import AuthenticationError, NetworkError
|
|
16
17
|
from .models import Deviation, DownloadItem
|
|
17
18
|
from .official_api import (
|
|
18
19
|
OfficialApiClient,
|
|
@@ -262,6 +263,13 @@ class OfficialProvider:
|
|
|
262
263
|
ext = _ext_from_url(original.filename)
|
|
263
264
|
return original.url, ext
|
|
264
265
|
content = deviation.get("content") or {}
|
|
266
|
+
# `content.src` is a poster for video works; only videos[].src is playable.
|
|
267
|
+
videos = deviation.get("videos") or []
|
|
268
|
+
if isinstance(videos, list):
|
|
269
|
+
for video in videos:
|
|
270
|
+
src = video.get("src") if isinstance(video, dict) else None
|
|
271
|
+
if src:
|
|
272
|
+
return str(src), _ext_from_url(str(src))
|
|
265
273
|
if content.get("src"):
|
|
266
274
|
return str(content["src"]), _ext_from_url(str(content["src"]))
|
|
267
275
|
preview = deviation.get("preview") or {}
|
|
@@ -297,4 +305,55 @@ def _parse_published(value: Any) -> Optional[datetime]:
|
|
|
297
305
|
return datetime.fromtimestamp(timestamp, tz=timezone.utc)
|
|
298
306
|
|
|
299
307
|
|
|
300
|
-
|
|
308
|
+
class CompositeProvider:
|
|
309
|
+
"""Routes each target to the provider that can actually serve it.
|
|
310
|
+
|
|
311
|
+
OAuth and cookies are capabilities, not global provider modes: search
|
|
312
|
+
always uses the web session, artwork/tag use the official API, and
|
|
313
|
+
gallery/favorites prefer the official API with the web session as a
|
|
314
|
+
fallback when the official path is unavailable.
|
|
315
|
+
"""
|
|
316
|
+
|
|
317
|
+
def __init__(
|
|
318
|
+
self,
|
|
319
|
+
official: DeviantArtProvider | None = None,
|
|
320
|
+
web: DeviantArtProvider | None = None,
|
|
321
|
+
) -> None:
|
|
322
|
+
if official is None and web is None:
|
|
323
|
+
raise ValueError("at least one provider is required")
|
|
324
|
+
self.official = official
|
|
325
|
+
self.web = web
|
|
326
|
+
|
|
327
|
+
def resolve(self, target: DownloadTarget) -> Iterator[DownloadItem]:
|
|
328
|
+
kind = target.kind
|
|
329
|
+
if kind == TargetKind.SEARCH:
|
|
330
|
+
if self.web is None:
|
|
331
|
+
raise MediaUnavailableError(
|
|
332
|
+
"search requires a web/cookie session "
|
|
333
|
+
"(`devart-dl login browser`/`interactive` or --cookies)"
|
|
334
|
+
)
|
|
335
|
+
yield from self.web.resolve(target)
|
|
336
|
+
return
|
|
337
|
+
if kind in (TargetKind.ARTWORK, TargetKind.TAG):
|
|
338
|
+
if self.official is None:
|
|
339
|
+
raise MediaUnavailableError(
|
|
340
|
+
"this target requires the official API "
|
|
341
|
+
"(`devart-dl login oauth`)"
|
|
342
|
+
)
|
|
343
|
+
yield from self.official.resolve(target)
|
|
344
|
+
return
|
|
345
|
+
if self.official is not None:
|
|
346
|
+
try:
|
|
347
|
+
yield from self.official.resolve(target)
|
|
348
|
+
except (MediaUnavailableError, NetworkError, AuthenticationError):
|
|
349
|
+
if self.web is not None:
|
|
350
|
+
yield from self.web.resolve(target)
|
|
351
|
+
return
|
|
352
|
+
raise
|
|
353
|
+
elif self.web is not None:
|
|
354
|
+
yield from self.web.resolve(target)
|
|
355
|
+
else:
|
|
356
|
+
raise MediaUnavailableError("no provider can resolve this target")
|
|
357
|
+
|
|
358
|
+
|
|
359
|
+
__all__ = ["CompositeProvider", "DeviantArtProvider", "OfficialProvider", "WebProvider"]
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: devart-dl
|
|
3
|
-
Version: 4.
|
|
3
|
+
Version: 4.3.1
|
|
4
4
|
Summary: 专业的 DeviantArt 作品批量下载工具 - Professional DeviantArt batch downloader with anti-ban features
|
|
5
5
|
Author: DeviantArt Downloader Team
|
|
6
6
|
License-Expression: MIT
|
|
@@ -34,9 +34,11 @@ Dynamic: license-file
|
|
|
34
34
|
|
|
35
35
|
# DeviantArt Downloader
|
|
36
36
|
|
|
37
|
-
|
|
37
|
+
**语言 / Language:** 中文 · [English](README.en.md)
|
|
38
38
|
|
|
39
|
-
|
|
39
|
+
> **一个可靠、聚焦的 DeviantArt 下载 / 归档命令行工具。**
|
|
40
|
+
|
|
41
|
+
[完整文档](https://redtidev1918.github.io/deviantart-downloader/) · [更新日志](CHANGELOG.md)
|
|
40
42
|
|
|
41
43
|
[](https://github.com/redtidev1918/deviantart-downloader/actions/workflows/ci.yml)
|
|
42
44
|
[](https://pypi.org/project/devart-dl/)
|
|
@@ -61,6 +63,9 @@ devart-dl https://www.deviantart.com/username/gallery
|
|
|
61
63
|
# 收藏夹 / 标签
|
|
62
64
|
devart-dl https://www.deviantart.com/username/favourites
|
|
63
65
|
devart-dl https://www.deviantart.com/tag/landscape
|
|
66
|
+
|
|
67
|
+
# 解析为 JSON 描述(不下载文件)
|
|
68
|
+
devart-dl resolve https://www.deviantart.com/username/art/title-123456
|
|
64
69
|
```
|
|
65
70
|
|
|
66
71
|
首次下载前建议使用 OAuth 登录:
|
|
@@ -88,10 +93,20 @@ devart-dl login browser # 打开你的 Chrome 登录,自动保存 Cookie,
|
|
|
88
93
|
- **下载档案**:`--archive` 用 SQLite 记住已下载,跨会话自动跳过。
|
|
89
94
|
- **路径模板 + 元数据 sidecar**:`--directory`/`--filename` 模板与 `--write-info-json`。
|
|
90
95
|
- **代理与降级**:支持 `--proxy`;未登录 OAuth 时自动回退 Cookie。
|
|
96
|
+
- **Resolve 模式**:`devart-dl resolve URL` 输出 `ResolvedDeviation`/`MediaAsset` 的 JSON 契约,不写文件。
|
|
97
|
+
|
|
98
|
+
## 文档
|
|
99
|
+
|
|
100
|
+
| 你想做什么 | 文档 |
|
|
101
|
+
| --- | --- |
|
|
102
|
+
| 下载安装包 | [下载页](https://redtidev1918.github.io/deviantart-downloader/download.md) |
|
|
103
|
+
| 登录与认证 | [登录与认证](docs/LOGIN.md) |
|
|
104
|
+
| 命令、配置与 Python 调用 | [文档站](https://redtidev1918.github.io/deviantart-downloader/) |
|
|
91
105
|
|
|
92
|
-
|
|
106
|
+
## 致谢
|
|
93
107
|
|
|
94
|
-
https://
|
|
108
|
+
- [requests](https://github.com/psf/requests):HTTP 客户端。
|
|
109
|
+
- 登录与归档目录的实现与 [gallery-dl](https://github.com/mikf/gallery-dl) 交叉核对过。
|
|
95
110
|
|
|
96
111
|
## 许可证与免责声明
|
|
97
112
|
|