socialdatax-skills 0.2.26 → 0.2.28
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +7 -3
- package/cli.mjs +38 -3
- package/lib/media/xhs-download.mjs +723 -0
- package/package.json +1 -1
- package/skills/media-detail/SKILL.md +1 -0
package/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
This public package provides the unified skill installer and direct CLI helper for SocialDataX services.
|
|
4
4
|
|
|
5
|
-
The current public tools support 小红书 / Xiaohongshu / XHS / RedNote, 抖音 / Douyin, 快手 / Kuaishou / Kwai, 微博 / Weibo, 视频号 / WeChat Channels, and 微信公众号 / WeChat Official Account article content research and analysis workflows, plus Bilibili / 哔哩哔哩 / B站 local video download and 敏感词检测 / 违禁词检查 text checks. The public skill layer is intentionally named by capability so supported services can evolve without changing the installation model.
|
|
5
|
+
The current public tools support 小红书 / Xiaohongshu / XHS / RedNote, 抖音 / Douyin, 快手 / Kuaishou / Kwai, 微博 / Weibo, 视频号 / WeChat Channels, and 微信公众号 / WeChat Official Account article content research and analysis workflows, plus XHS local media download, Bilibili / 哔哩哔哩 / B站 local video download, and 敏感词检测 / 违禁词检查 text checks. The public skill layer is intentionally named by capability so supported services can evolve without changing the installation model.
|
|
6
6
|
|
|
7
7
|
- direct `npx` JSON commands for agents that can run shell commands
|
|
8
8
|
- AgentSkills-compatible installers split by capability for OpenClaw, Hermes Agent, Codex, Claude Code, and general agent skill directories
|
|
@@ -28,6 +28,9 @@ Common search phrases for this skill package:
|
|
|
28
28
|
- `Xiaohongshu skills`
|
|
29
29
|
- `XHS skills`
|
|
30
30
|
- `RedNote skills`
|
|
31
|
+
- `XHS media download`
|
|
32
|
+
- `小红书图片下载`
|
|
33
|
+
- `小红书视频下载`
|
|
31
34
|
- `Douyin skills`
|
|
32
35
|
- `抖音 skills`
|
|
33
36
|
- `Kuaishou skills`
|
|
@@ -79,7 +82,7 @@ Common search phrases for this skill package:
|
|
|
79
82
|
- Sensitive Words Check MCP server name: `sensitive-check`
|
|
80
83
|
- Sensitive Words Check future MCP registry name: `com.socialdatax/sensitive-check`
|
|
81
84
|
- Unified MCP registry name: none; this package installs skills and calls explicit platform services.
|
|
82
|
-
- Current public capability version: `0.2.
|
|
85
|
+
- Current public capability version: `0.2.28`
|
|
83
86
|
|
|
84
87
|
## Direct CLI
|
|
85
88
|
|
|
@@ -102,6 +105,7 @@ npx -y socialdatax-skills@latest xhs user-posts --user-id "<user_id>" --since-da
|
|
|
102
105
|
npx -y socialdatax-skills@latest xhs transcript --url "<note_url_or_share_text>" --pretty
|
|
103
106
|
npx -y socialdatax-skills@latest xhs transcript --note-id "<note_id>" --pretty
|
|
104
107
|
npx -y socialdatax-skills@latest xhs transcript --job-id "<job_id>" --pretty
|
|
108
|
+
npx -y socialdatax-skills@latest xhs download-media --url "<xhs_media_url>" --output-dir ./downloads --pretty
|
|
105
109
|
npx -y socialdatax-skills@latest douyin hot-search --pretty
|
|
106
110
|
npx -y socialdatax-skills@latest douyin search --keyword "露营" --pretty
|
|
107
111
|
npx -y socialdatax-skills@latest douyin detail --aweme-id "<aweme_id>" --pretty
|
|
@@ -170,7 +174,7 @@ npx -y socialdatax-skills@latest wechat transcript --job-id "<job_id>" --pretty
|
|
|
170
174
|
npx -y socialdatax-skills@latest sensitive-check text --text "<content>" --platform xhs --pretty
|
|
171
175
|
```
|
|
172
176
|
|
|
173
|
-
Most direct CLI commands print a JSON envelope with `platform`, `tool`, `arguments`, and `data`. `sensitive-check` prints `platform`, `tool`, and `data` only; it does not echo the original text in CLI output. `wechat decrypt-media` is a local save command: pass the `video.video_url` returned by WeChat detail and an `--output` file path. It saves the media locally, decrypts when needed, and does not require `SOCIALDATAX_API_KEY`.
|
|
177
|
+
Most direct CLI commands print a JSON envelope with `platform`, `tool`, `arguments`, and `data`. `sensitive-check` prints `platform`, `tool`, and `data` only; it does not echo the original text in CLI output. `wechat decrypt-media` is a local save command: pass the `video.video_url` returned by WeChat detail and an `--output` file path. It saves the media locally, decrypts when needed, and does not require `SOCIALDATAX_API_KEY`. `xhs download-media` is also local: pass one image or video media URL returned by XHS detail and either `--output <file>` or `--output-dir <directory>`. It writes through a `.part` file, resumes partial downloads when the server supports range requests, skips an already existing output file, and infers common image/video extensions in `--output-dir` mode.
|
|
174
178
|
|
|
175
179
|
`bilibili download` calls SocialDataX once to fetch short-lived download links, then downloads video and audio tracks on the local machine and merges them with local `ffmpeg`. Set `SOCIALDATAX_API_KEY`, install `ffmpeg`, and pass either `--output <file>` or `--output-dir <directory>`. The download-links request consumes 10 credits; the local track download and merge do not consume additional SocialDataX credits. Use `--ffmpeg-path <path>` when `ffmpeg` is not on `PATH`, and `--keep-tracks` to retain the separate track files after merge.
|
|
176
180
|
|
package/cli.mjs
CHANGED
|
@@ -17,11 +17,12 @@ import {
|
|
|
17
17
|
downloadBilibiliVideoFromManifest,
|
|
18
18
|
} from "./lib/media/bilibili-download.mjs";
|
|
19
19
|
import { decryptWechatMediaCommand } from "./lib/media/wechat-decrypt.mjs";
|
|
20
|
+
import { downloadXhsMediaFromUrl } from "./lib/media/xhs-download.mjs";
|
|
20
21
|
|
|
21
22
|
export { decryptWechatMediaCommand };
|
|
22
23
|
|
|
23
24
|
const PACKAGE_NAME = "socialdatax-skills";
|
|
24
|
-
const PACKAGE_VERSION = "0.2.
|
|
25
|
+
const PACKAGE_VERSION = "0.2.28";
|
|
25
26
|
const PACKAGE_SPEC = `${PACKAGE_NAME}@latest`;
|
|
26
27
|
const LOG_PREFIX = `[${PACKAGE_NAME}]`;
|
|
27
28
|
const MIN_NODE_VERSION = "20.18.1";
|
|
@@ -163,6 +164,7 @@ const XHS_DIRECT_ACTION_OPTIONS = {
|
|
|
163
164
|
"pretty",
|
|
164
165
|
],
|
|
165
166
|
transcript: ["url", "noteId", "jobId", "pretty"],
|
|
167
|
+
"download-media": ["url", "output", "outputDir", "pretty"],
|
|
166
168
|
};
|
|
167
169
|
const XHS_DIRECT_ACTION_NAMES = Object.keys(XHS_DIRECT_ACTION_OPTIONS).join(", ");
|
|
168
170
|
const XHS_OPTION_DISPLAY_NAMES = {
|
|
@@ -182,6 +184,8 @@ const XHS_OPTION_DISPLAY_NAMES = {
|
|
|
182
184
|
profileUrl: "--profile-url",
|
|
183
185
|
userId: "--user-id",
|
|
184
186
|
jobId: "--job-id",
|
|
187
|
+
output: "--output",
|
|
188
|
+
outputDir: "--output-dir",
|
|
185
189
|
};
|
|
186
190
|
const XHS_SEARCH_SORT_TYPES = [
|
|
187
191
|
"general",
|
|
@@ -1097,6 +1101,26 @@ function validateXhsDirectActionOptions(action, options) {
|
|
|
1097
1101
|
return;
|
|
1098
1102
|
}
|
|
1099
1103
|
|
|
1104
|
+
if (action === "download-media") {
|
|
1105
|
+
validateKnownOptions(options, allowedOptions);
|
|
1106
|
+
validateFlagOption(options, "pretty", "--pretty");
|
|
1107
|
+
for (const key of allowedOptions) {
|
|
1108
|
+
if (!DIRECT_BOOLEAN_OPTIONS.has(key)) {
|
|
1109
|
+
requireOptionValue(options, key, XHS_OPTION_DISPLAY_NAMES[key]);
|
|
1110
|
+
}
|
|
1111
|
+
}
|
|
1112
|
+
if (!options.url) {
|
|
1113
|
+
throw new Error("Missing --url for xhs download-media.");
|
|
1114
|
+
}
|
|
1115
|
+
if (!options.output && !options.outputDir) {
|
|
1116
|
+
throw new Error("Missing --output or --output-dir for xhs download-media.");
|
|
1117
|
+
}
|
|
1118
|
+
if (options.output && options.outputDir) {
|
|
1119
|
+
throw new Error("Use only one of --output or --output-dir for xhs download-media.");
|
|
1120
|
+
}
|
|
1121
|
+
return;
|
|
1122
|
+
}
|
|
1123
|
+
|
|
1100
1124
|
validateKnownOptions(options, allowedDirectOptions(allowedOptions));
|
|
1101
1125
|
validateDirectMetaOptions(options);
|
|
1102
1126
|
validateDirectPaginationOptions("xhs", action, options);
|
|
@@ -1847,6 +1871,9 @@ function printHelp() {
|
|
|
1847
1871
|
console.log(` npx -y ${PACKAGE_SPEC} xhs transcript --url "<xhs_note_url_or_share_text>" --pretty`);
|
|
1848
1872
|
console.log(" Submit or check an XHS video note speech-to-text transcript job.");
|
|
1849
1873
|
console.log("");
|
|
1874
|
+
console.log(` npx -y ${PACKAGE_SPEC} xhs download-media --url "<xhs_media_url>" --output-dir ./downloads --pretty`);
|
|
1875
|
+
console.log(" Save one XHS image or video media URL returned by detail to a local file.");
|
|
1876
|
+
console.log("");
|
|
1850
1877
|
console.log(` npx -y ${PACKAGE_SPEC} douyin hot-search --pretty`);
|
|
1851
1878
|
console.log(" Call the Douyin main hot search list tool directly and print JSON.");
|
|
1852
1879
|
console.log("");
|
|
@@ -2090,13 +2117,14 @@ function printHelp() {
|
|
|
2090
2117
|
console.log(" --keyword <text>");
|
|
2091
2118
|
console.log(" --url <url-or-share-text>");
|
|
2092
2119
|
console.log(" Content link, short link, or share text for URL-based detail/comment/article commands.");
|
|
2120
|
+
console.log(" For xhs download-media, pass one image or video media URL returned by XHS detail.");
|
|
2093
2121
|
console.log(" For Douyin detail/comments, pass a content page link, not video.play_url.");
|
|
2094
2122
|
console.log(" --media-url <video.video_url>");
|
|
2095
2123
|
console.log(" WeChat detail result media URL for local decrypt-media download.");
|
|
2096
2124
|
console.log(" --output <file>");
|
|
2097
|
-
console.log(" Output file path for local decrypt-media or Bilibili download.");
|
|
2125
|
+
console.log(" Output file path for local decrypt-media, XHS download-media, or Bilibili download.");
|
|
2098
2126
|
console.log(" --output-dir <directory>");
|
|
2099
|
-
console.log(" Directory for Bilibili download output when --output is omitted.");
|
|
2127
|
+
console.log(" Directory for XHS download-media or Bilibili download output when --output is omitted.");
|
|
2100
2128
|
console.log(" --ffmpeg-path <path>");
|
|
2101
2129
|
console.log(" ffmpeg executable path for Bilibili download; defaults to ffmpeg.");
|
|
2102
2130
|
console.log(" --keep-tracks");
|
|
@@ -2218,6 +2246,13 @@ async function runXhsDirectCommand(args) {
|
|
|
2218
2246
|
}
|
|
2219
2247
|
validateXhsDirectActionOptions(action, options);
|
|
2220
2248
|
|
|
2249
|
+
if (action === "download-media") {
|
|
2250
|
+
const data = await downloadXhsMediaFromUrl(options.url, options);
|
|
2251
|
+
process.stdout.write(JSON.stringify(data, null, options.pretty ? 2 : 0));
|
|
2252
|
+
process.stdout.write("\n");
|
|
2253
|
+
return;
|
|
2254
|
+
}
|
|
2255
|
+
|
|
2221
2256
|
const operation = attachDirectMetadata(buildXhsOperation(action, options), options);
|
|
2222
2257
|
const data = shouldUsePaginatedDirectOutput(options)
|
|
2223
2258
|
? await callPaginatedDirectOperation(operation, options)
|
|
@@ -0,0 +1,723 @@
|
|
|
1
|
+
import { constants as fsConstants } from "node:fs";
|
|
2
|
+
import {
|
|
3
|
+
copyFile as copyFilePath,
|
|
4
|
+
link as linkPath,
|
|
5
|
+
open as openFile,
|
|
6
|
+
mkdir,
|
|
7
|
+
stat,
|
|
8
|
+
unlink,
|
|
9
|
+
} from "node:fs/promises";
|
|
10
|
+
import { homedir } from "node:os";
|
|
11
|
+
import { basename, dirname, extname, join, resolve } from "node:path";
|
|
12
|
+
|
|
13
|
+
import {
|
|
14
|
+
MEDIA_DOWNLOAD_MAX_BYTES,
|
|
15
|
+
MEDIA_DOWNLOAD_TIMEOUT_MS,
|
|
16
|
+
assertMediaDownloadWithinLimit,
|
|
17
|
+
createMediaDownloadAbortSignal,
|
|
18
|
+
isAbortLikeError,
|
|
19
|
+
parseOptionalIntegerHeader,
|
|
20
|
+
writeAllToFile,
|
|
21
|
+
} from "./common.mjs";
|
|
22
|
+
|
|
23
|
+
const XHS_DOWNLOAD_USER_AGENT =
|
|
24
|
+
"Mozilla/5.0 (compatible; SocialDataX/1.0; +https://socialdatax.com)";
|
|
25
|
+
const XHS_DOWNLOAD_REFERER = "https://www.xiaohongshu.com/";
|
|
26
|
+
const XHS_DOWNLOAD_MAX_ATTEMPTS = 10;
|
|
27
|
+
const XHS_DOWNLOAD_RETRY_DELAY_MS = 250;
|
|
28
|
+
const XHS_RETRYABLE_HTTP_STATUSES = new Set([
|
|
29
|
+
408,
|
|
30
|
+
409,
|
|
31
|
+
425,
|
|
32
|
+
429,
|
|
33
|
+
500,
|
|
34
|
+
502,
|
|
35
|
+
503,
|
|
36
|
+
504,
|
|
37
|
+
]);
|
|
38
|
+
const XHS_EXPIRED_HTTP_STATUSES = new Set([403, 404, 410]);
|
|
39
|
+
const XHS_CONTENT_TYPE_EXTENSIONS = new Map([
|
|
40
|
+
["image/jpeg", "jpg"],
|
|
41
|
+
["image/jpg", "jpg"],
|
|
42
|
+
["image/png", "png"],
|
|
43
|
+
["image/webp", "webp"],
|
|
44
|
+
["image/heif", "heif"],
|
|
45
|
+
["image/heic", "heic"],
|
|
46
|
+
["video/mp4", "mp4"],
|
|
47
|
+
]);
|
|
48
|
+
const XHS_URL_FORMAT_EXTENSIONS = new Map([
|
|
49
|
+
["jpg", "jpg"],
|
|
50
|
+
["jpeg", "jpg"],
|
|
51
|
+
["png", "png"],
|
|
52
|
+
["webp", "webp"],
|
|
53
|
+
["heif", "heif"],
|
|
54
|
+
["heic", "heic"],
|
|
55
|
+
["mp4", "mp4"],
|
|
56
|
+
]);
|
|
57
|
+
export async function downloadXhsMediaFromUrl(
|
|
58
|
+
url,
|
|
59
|
+
options,
|
|
60
|
+
{
|
|
61
|
+
fetchMedia = fetch,
|
|
62
|
+
downloadTimeoutMs = MEDIA_DOWNLOAD_TIMEOUT_MS,
|
|
63
|
+
maxDownloadBytes = MEDIA_DOWNLOAD_MAX_BYTES,
|
|
64
|
+
retryDelayMs = XHS_DOWNLOAD_RETRY_DELAY_MS,
|
|
65
|
+
now = () => new Date(),
|
|
66
|
+
linkFile = linkPath,
|
|
67
|
+
copyFile = copyFilePath,
|
|
68
|
+
} = {}
|
|
69
|
+
) {
|
|
70
|
+
const parsedUrl = parseHttpUrl(url);
|
|
71
|
+
const target = await resolveXhsMediaTarget(parsedUrl, options, now);
|
|
72
|
+
const existingOutput = await stat(target.outputPath).catch(ignoreMissing);
|
|
73
|
+
if (existingOutput?.isDirectory()) {
|
|
74
|
+
throw new Error("--output must be a file path for xhs download-media.");
|
|
75
|
+
}
|
|
76
|
+
if (existingOutput) {
|
|
77
|
+
return buildSkippedExistingResult(parsedUrl.href, target.outputPath, existingOutput.size);
|
|
78
|
+
}
|
|
79
|
+
|
|
80
|
+
return downloadWithPartResume(parsedUrl, target, {
|
|
81
|
+
fetchMedia,
|
|
82
|
+
downloadTimeoutMs,
|
|
83
|
+
maxDownloadBytes,
|
|
84
|
+
retryDelayMs,
|
|
85
|
+
linkFile,
|
|
86
|
+
copyFile,
|
|
87
|
+
});
|
|
88
|
+
}
|
|
89
|
+
|
|
90
|
+
async function resolveXhsMediaTarget(parsedUrl, options = {}, now) {
|
|
91
|
+
if (options.output && options.outputDir) {
|
|
92
|
+
throw new Error("Use only one of --output or --output-dir for xhs download-media.");
|
|
93
|
+
}
|
|
94
|
+
if (options.output) {
|
|
95
|
+
const outputPath = resolve(expandHome(options.output));
|
|
96
|
+
await ensureParentDirectory(outputPath, "--output parent path");
|
|
97
|
+
return {
|
|
98
|
+
outputPath,
|
|
99
|
+
partPath: `${outputPath}.part`,
|
|
100
|
+
fixedOutputPath: true,
|
|
101
|
+
};
|
|
102
|
+
}
|
|
103
|
+
if (!options.outputDir) {
|
|
104
|
+
throw new Error("Missing --output or --output-dir for xhs download-media.");
|
|
105
|
+
}
|
|
106
|
+
|
|
107
|
+
const outputDir = resolve(expandHome(options.outputDir));
|
|
108
|
+
const existingOutputDir = await stat(outputDir).catch(ignoreMissing);
|
|
109
|
+
if (existingOutputDir && !existingOutputDir.isDirectory()) {
|
|
110
|
+
throw new Error("--output-dir must be a directory for xhs download-media.");
|
|
111
|
+
}
|
|
112
|
+
if (!existingOutputDir) {
|
|
113
|
+
await ensureParentDirectory(outputDir, "--output-dir parent path");
|
|
114
|
+
await mkdir(outputDir, { recursive: true });
|
|
115
|
+
}
|
|
116
|
+
|
|
117
|
+
const fileTarget = inferMediaFileTarget(parsedUrl, now);
|
|
118
|
+
const initialExtension =
|
|
119
|
+
fileTarget.extension || fileTarget.urlFormatExtension || "bin";
|
|
120
|
+
const fileName = formatMediaFileName(fileTarget.stem, initialExtension);
|
|
121
|
+
const outputPath = join(outputDir, fileName);
|
|
122
|
+
return {
|
|
123
|
+
outputPath,
|
|
124
|
+
partPath: `${outputPath}.part`,
|
|
125
|
+
outputDir,
|
|
126
|
+
fileStem: fileTarget.stem,
|
|
127
|
+
extension: fileTarget.extension,
|
|
128
|
+
urlFormatExtension: fileTarget.urlFormatExtension,
|
|
129
|
+
fixedOutputPath: false,
|
|
130
|
+
};
|
|
131
|
+
}
|
|
132
|
+
|
|
133
|
+
async function downloadWithPartResume(
|
|
134
|
+
parsedUrl,
|
|
135
|
+
target,
|
|
136
|
+
{
|
|
137
|
+
fetchMedia,
|
|
138
|
+
downloadTimeoutMs,
|
|
139
|
+
maxDownloadBytes,
|
|
140
|
+
retryDelayMs,
|
|
141
|
+
linkFile,
|
|
142
|
+
copyFile,
|
|
143
|
+
}
|
|
144
|
+
) {
|
|
145
|
+
let lastError;
|
|
146
|
+
for (let attempt = 1; attempt <= XHS_DOWNLOAD_MAX_ATTEMPTS; attempt += 1) {
|
|
147
|
+
try {
|
|
148
|
+
const result = await downloadOnce(parsedUrl, target, {
|
|
149
|
+
fetchMedia,
|
|
150
|
+
downloadTimeoutMs,
|
|
151
|
+
maxDownloadBytes,
|
|
152
|
+
linkFile,
|
|
153
|
+
copyFile,
|
|
154
|
+
});
|
|
155
|
+
if (result.skippedExisting) {
|
|
156
|
+
return buildSkippedExistingResult(
|
|
157
|
+
parsedUrl.href,
|
|
158
|
+
result.outputPath,
|
|
159
|
+
result.outputBytes
|
|
160
|
+
);
|
|
161
|
+
}
|
|
162
|
+
return {
|
|
163
|
+
platform: "xhs",
|
|
164
|
+
action: "download-media",
|
|
165
|
+
status: "downloaded",
|
|
166
|
+
url: parsedUrl.href,
|
|
167
|
+
output_path: result.outputPath,
|
|
168
|
+
output_bytes: result.outputBytes,
|
|
169
|
+
resumed: result.resumed,
|
|
170
|
+
content_type: result.contentType,
|
|
171
|
+
};
|
|
172
|
+
} catch (error) {
|
|
173
|
+
if (error instanceof XhsExpiredMediaLinkError) {
|
|
174
|
+
throw error;
|
|
175
|
+
}
|
|
176
|
+
lastError = error;
|
|
177
|
+
if (!shouldRetryDownloadError(error)) {
|
|
178
|
+
throw error;
|
|
179
|
+
}
|
|
180
|
+
if (attempt === XHS_DOWNLOAD_MAX_ATTEMPTS) {
|
|
181
|
+
break;
|
|
182
|
+
}
|
|
183
|
+
await sleep(retryDelayMs);
|
|
184
|
+
}
|
|
185
|
+
}
|
|
186
|
+
|
|
187
|
+
if (isAbortLikeError(lastError)) {
|
|
188
|
+
throw new Error("XHS media download timed out.");
|
|
189
|
+
}
|
|
190
|
+
throw new Error("XHS media download failed before completion.");
|
|
191
|
+
}
|
|
192
|
+
|
|
193
|
+
async function downloadOnce(
|
|
194
|
+
parsedUrl,
|
|
195
|
+
target,
|
|
196
|
+
{
|
|
197
|
+
fetchMedia,
|
|
198
|
+
downloadTimeoutMs,
|
|
199
|
+
maxDownloadBytes,
|
|
200
|
+
linkFile,
|
|
201
|
+
copyFile,
|
|
202
|
+
}
|
|
203
|
+
) {
|
|
204
|
+
await mkdir(dirname(target.outputPath), { recursive: true });
|
|
205
|
+
const partPath = target.partPath;
|
|
206
|
+
const existingPart = await stat(partPath).catch(ignoreMissing);
|
|
207
|
+
let existingPartBytes = existingPart?.isFile() ? existingPart.size : 0;
|
|
208
|
+
const headers = normalizeXhsDownloadHeaders();
|
|
209
|
+
if (existingPartBytes > 0) {
|
|
210
|
+
headers.Range = `bytes=${existingPartBytes}-`;
|
|
211
|
+
delete headers["accept-encoding"];
|
|
212
|
+
}
|
|
213
|
+
|
|
214
|
+
const signal = createMediaDownloadAbortSignal(downloadTimeoutMs);
|
|
215
|
+
const response = await fetchMedia(parsedUrl.href, {
|
|
216
|
+
headers,
|
|
217
|
+
...(signal ? { signal } : {}),
|
|
218
|
+
});
|
|
219
|
+
|
|
220
|
+
if (XHS_EXPIRED_HTTP_STATUSES.has(response.status)) {
|
|
221
|
+
throw new XhsExpiredMediaLinkError();
|
|
222
|
+
}
|
|
223
|
+
if (response.status === 416) {
|
|
224
|
+
return handleRangeNotSatisfiable(response, {
|
|
225
|
+
target,
|
|
226
|
+
partPath,
|
|
227
|
+
existingPartBytes,
|
|
228
|
+
linkFile,
|
|
229
|
+
copyFile,
|
|
230
|
+
});
|
|
231
|
+
}
|
|
232
|
+
if (!response.ok) {
|
|
233
|
+
if (XHS_RETRYABLE_HTTP_STATUSES.has(response.status)) {
|
|
234
|
+
throw new XhsRetryableDownloadError();
|
|
235
|
+
}
|
|
236
|
+
throw new Error(`XHS media download failed with HTTP ${response.status}.`);
|
|
237
|
+
}
|
|
238
|
+
|
|
239
|
+
const contentType = normalizedContentType(headerValue(response.headers, "content-type"));
|
|
240
|
+
const outputPath = resolveFinalOutputPath(target, contentType);
|
|
241
|
+
const existingOutput = await stat(outputPath).catch(ignoreMissing);
|
|
242
|
+
if (existingOutput?.isDirectory()) {
|
|
243
|
+
throw new Error("--output must be a file path for xhs download-media.");
|
|
244
|
+
}
|
|
245
|
+
if (existingOutput) {
|
|
246
|
+
await cancelResponseBody(response);
|
|
247
|
+
return {
|
|
248
|
+
skippedExisting: true,
|
|
249
|
+
outputPath,
|
|
250
|
+
outputBytes: existingOutput.size,
|
|
251
|
+
contentType,
|
|
252
|
+
};
|
|
253
|
+
}
|
|
254
|
+
const contentLength = parseOptionalIntegerHeader(
|
|
255
|
+
headerValue(response.headers, "content-length")
|
|
256
|
+
);
|
|
257
|
+
let append = false;
|
|
258
|
+
let expectedTotalBytes = contentLength;
|
|
259
|
+
let expectedResponseBytes;
|
|
260
|
+
|
|
261
|
+
if (existingPartBytes > 0 && response.status === 206) {
|
|
262
|
+
const contentRange = parseContentRange(
|
|
263
|
+
headerValue(response.headers, "content-range")
|
|
264
|
+
);
|
|
265
|
+
if (
|
|
266
|
+
!isUsableContentRange(contentRange, contentLength) ||
|
|
267
|
+
contentRange.start !== existingPartBytes
|
|
268
|
+
) {
|
|
269
|
+
await cancelResponseBody(response);
|
|
270
|
+
await unlink(partPath).catch(ignoreError);
|
|
271
|
+
throw new XhsRetryableDownloadError();
|
|
272
|
+
}
|
|
273
|
+
append = true;
|
|
274
|
+
expectedResponseBytes = contentRange.end - contentRange.start + 1;
|
|
275
|
+
expectedTotalBytes = contentRange.total;
|
|
276
|
+
} else if (existingPartBytes > 0) {
|
|
277
|
+
existingPartBytes = 0;
|
|
278
|
+
expectedTotalBytes = contentLength;
|
|
279
|
+
} else if (response.status === 206) {
|
|
280
|
+
const contentRange = parseContentRange(
|
|
281
|
+
headerValue(response.headers, "content-range")
|
|
282
|
+
);
|
|
283
|
+
if (
|
|
284
|
+
!isUsableContentRange(contentRange, contentLength) ||
|
|
285
|
+
contentRange.start !== 0
|
|
286
|
+
) {
|
|
287
|
+
await cancelResponseBody(response);
|
|
288
|
+
throw new XhsRetryableDownloadError();
|
|
289
|
+
}
|
|
290
|
+
expectedResponseBytes = contentRange.end - contentRange.start + 1;
|
|
291
|
+
expectedTotalBytes = contentRange.total;
|
|
292
|
+
}
|
|
293
|
+
|
|
294
|
+
assertMediaDownloadWithinLimit(expectedTotalBytes, maxDownloadBytes);
|
|
295
|
+
let outputBytes;
|
|
296
|
+
let bytesWritten;
|
|
297
|
+
try {
|
|
298
|
+
bytesWritten = await writeResponseBodyToPart(response, partPath, {
|
|
299
|
+
append,
|
|
300
|
+
existingBytes: existingPartBytes,
|
|
301
|
+
maxBytes: maxDownloadBytes,
|
|
302
|
+
});
|
|
303
|
+
outputBytes = existingPartBytes + bytesWritten;
|
|
304
|
+
} catch (error) {
|
|
305
|
+
if (isAbortLikeError(error) || isDownloadTooLargeError(error)) {
|
|
306
|
+
throw error;
|
|
307
|
+
}
|
|
308
|
+
throw new XhsRetryableDownloadError();
|
|
309
|
+
}
|
|
310
|
+
|
|
311
|
+
if (
|
|
312
|
+
expectedResponseBytes !== undefined &&
|
|
313
|
+
bytesWritten !== expectedResponseBytes
|
|
314
|
+
) {
|
|
315
|
+
if (bytesWritten > expectedResponseBytes) {
|
|
316
|
+
await unlink(partPath).catch(ignoreError);
|
|
317
|
+
}
|
|
318
|
+
throw new XhsRetryableDownloadError();
|
|
319
|
+
}
|
|
320
|
+
|
|
321
|
+
if (expectedTotalBytes !== undefined && outputBytes !== expectedTotalBytes) {
|
|
322
|
+
if (outputBytes > expectedTotalBytes) {
|
|
323
|
+
await unlink(partPath).catch(ignoreError);
|
|
324
|
+
}
|
|
325
|
+
throw new XhsRetryableDownloadError();
|
|
326
|
+
}
|
|
327
|
+
|
|
328
|
+
await finalizePartFile(partPath, outputPath, { linkFile, copyFile });
|
|
329
|
+
return {
|
|
330
|
+
outputPath,
|
|
331
|
+
outputBytes,
|
|
332
|
+
resumed: append && existingPartBytes > 0,
|
|
333
|
+
contentType,
|
|
334
|
+
};
|
|
335
|
+
}
|
|
336
|
+
|
|
337
|
+
async function handleRangeNotSatisfiable(
|
|
338
|
+
response,
|
|
339
|
+
{
|
|
340
|
+
target,
|
|
341
|
+
partPath,
|
|
342
|
+
existingPartBytes,
|
|
343
|
+
linkFile,
|
|
344
|
+
copyFile,
|
|
345
|
+
}
|
|
346
|
+
) {
|
|
347
|
+
const contentRange = parseContentRange(headerValue(response.headers, "content-range"));
|
|
348
|
+
if (
|
|
349
|
+
existingPartBytes > 0 &&
|
|
350
|
+
contentRange?.total !== undefined &&
|
|
351
|
+
existingPartBytes === contentRange.total
|
|
352
|
+
) {
|
|
353
|
+
const contentType = normalizedContentType(headerValue(response.headers, "content-type"));
|
|
354
|
+
const outputPath = resolveFinalOutputPath(target, contentType);
|
|
355
|
+
const existingOutput = await stat(outputPath).catch(ignoreMissing);
|
|
356
|
+
if (existingOutput?.isDirectory()) {
|
|
357
|
+
throw new Error("--output must be a file path for xhs download-media.");
|
|
358
|
+
}
|
|
359
|
+
if (existingOutput) {
|
|
360
|
+
await cancelResponseBody(response);
|
|
361
|
+
return {
|
|
362
|
+
skippedExisting: true,
|
|
363
|
+
outputPath,
|
|
364
|
+
outputBytes: existingOutput.size,
|
|
365
|
+
contentType,
|
|
366
|
+
};
|
|
367
|
+
}
|
|
368
|
+
await finalizePartFile(partPath, outputPath, { linkFile, copyFile });
|
|
369
|
+
return {
|
|
370
|
+
outputPath,
|
|
371
|
+
outputBytes: existingPartBytes,
|
|
372
|
+
resumed: true,
|
|
373
|
+
contentType,
|
|
374
|
+
};
|
|
375
|
+
}
|
|
376
|
+
await unlink(partPath).catch(ignoreError);
|
|
377
|
+
throw new XhsRetryableDownloadError();
|
|
378
|
+
}
|
|
379
|
+
|
|
380
|
+
async function writeResponseBodyToPart(
|
|
381
|
+
response,
|
|
382
|
+
partPath,
|
|
383
|
+
{
|
|
384
|
+
append,
|
|
385
|
+
existingBytes,
|
|
386
|
+
maxBytes,
|
|
387
|
+
}
|
|
388
|
+
) {
|
|
389
|
+
if (!response.body) {
|
|
390
|
+
throw new XhsRetryableDownloadError();
|
|
391
|
+
}
|
|
392
|
+
|
|
393
|
+
const file = await openFile(partPath, append ? "r+" : "w");
|
|
394
|
+
let bytesWritten = 0;
|
|
395
|
+
try {
|
|
396
|
+
for await (const chunk of response.body) {
|
|
397
|
+
const buffer = Buffer.from(chunk);
|
|
398
|
+
if (buffer.length === 0) {
|
|
399
|
+
continue;
|
|
400
|
+
}
|
|
401
|
+
const nextTotal = existingBytes + bytesWritten + buffer.length;
|
|
402
|
+
assertMediaDownloadWithinLimit(nextTotal, maxBytes);
|
|
403
|
+
await writeAllToFile(file, buffer, existingBytes + bytesWritten);
|
|
404
|
+
bytesWritten += buffer.length;
|
|
405
|
+
}
|
|
406
|
+
} finally {
|
|
407
|
+
await file.close();
|
|
408
|
+
}
|
|
409
|
+
return bytesWritten;
|
|
410
|
+
}
|
|
411
|
+
|
|
412
|
+
async function finalizePartFile(
|
|
413
|
+
partPath,
|
|
414
|
+
outputPath,
|
|
415
|
+
{
|
|
416
|
+
linkFile = linkPath,
|
|
417
|
+
copyFile = copyFilePath,
|
|
418
|
+
} = {}
|
|
419
|
+
) {
|
|
420
|
+
try {
|
|
421
|
+
await ensureParentDirectory(outputPath, "xhs download-media output parent path");
|
|
422
|
+
try {
|
|
423
|
+
await linkFile(partPath, outputPath);
|
|
424
|
+
} catch (error) {
|
|
425
|
+
if (isExistingOutputPathError(error)) {
|
|
426
|
+
throw error;
|
|
427
|
+
}
|
|
428
|
+
if (!shouldFallbackToCopyAfterLinkError(error)) {
|
|
429
|
+
throw error;
|
|
430
|
+
}
|
|
431
|
+
await copyFile(partPath, outputPath, fsConstants.COPYFILE_EXCL);
|
|
432
|
+
}
|
|
433
|
+
} catch (error) {
|
|
434
|
+
if (isExistingOutputPathError(error)) {
|
|
435
|
+
throw await buildExistingOutputPathError(outputPath);
|
|
436
|
+
}
|
|
437
|
+
throw error;
|
|
438
|
+
}
|
|
439
|
+
|
|
440
|
+
await unlink(partPath);
|
|
441
|
+
}
|
|
442
|
+
|
|
443
|
+
async function cancelResponseBody(response) {
|
|
444
|
+
const body = response.body;
|
|
445
|
+
if (!body) {
|
|
446
|
+
return;
|
|
447
|
+
}
|
|
448
|
+
if (typeof body.cancel === "function") {
|
|
449
|
+
await body.cancel().catch(ignoreError);
|
|
450
|
+
return;
|
|
451
|
+
}
|
|
452
|
+
if (typeof body.return === "function") {
|
|
453
|
+
await body.return().catch(ignoreError);
|
|
454
|
+
return;
|
|
455
|
+
}
|
|
456
|
+
if (typeof body.destroy === "function") {
|
|
457
|
+
body.destroy();
|
|
458
|
+
}
|
|
459
|
+
}
|
|
460
|
+
|
|
461
|
+
function parseHttpUrl(value) {
|
|
462
|
+
let parsed;
|
|
463
|
+
try {
|
|
464
|
+
parsed = new URL(String(value || ""));
|
|
465
|
+
} catch {
|
|
466
|
+
throw new Error("XHS media URL must be an HTTP URL.");
|
|
467
|
+
}
|
|
468
|
+
if (!["http:", "https:"].includes(parsed.protocol) || !parsed.hostname) {
|
|
469
|
+
throw new Error("XHS media URL must be an HTTP URL.");
|
|
470
|
+
}
|
|
471
|
+
return parsed;
|
|
472
|
+
}
|
|
473
|
+
|
|
474
|
+
function normalizeXhsDownloadHeaders() {
|
|
475
|
+
return {
|
|
476
|
+
"User-Agent": XHS_DOWNLOAD_USER_AGENT,
|
|
477
|
+
Referer: XHS_DOWNLOAD_REFERER,
|
|
478
|
+
"accept-encoding": "identity",
|
|
479
|
+
};
|
|
480
|
+
}
|
|
481
|
+
|
|
482
|
+
function resolveFinalOutputPath(target, contentType) {
|
|
483
|
+
if (target.fixedOutputPath) {
|
|
484
|
+
return target.outputPath;
|
|
485
|
+
}
|
|
486
|
+
const extension =
|
|
487
|
+
target.extension ||
|
|
488
|
+
mediaExtensionFromContentType(contentType) ||
|
|
489
|
+
target.urlFormatExtension ||
|
|
490
|
+
"bin";
|
|
491
|
+
return join(target.outputDir, formatMediaFileName(target.fileStem, extension));
|
|
492
|
+
}
|
|
493
|
+
|
|
494
|
+
function inferMediaFileTarget(parsedUrl, now) {
|
|
495
|
+
const fallbackStem = `xhs-media-${formatTimestamp(now())}`;
|
|
496
|
+
const urlFormatExtension = inferUrlFormatExtension(parsedUrl);
|
|
497
|
+
const rawBasename = safeDecodeURIComponent(basename(parsedUrl.pathname || ""));
|
|
498
|
+
const sanitized = sanitizeFileName(rawBasename);
|
|
499
|
+
if (sanitized && sanitized !== "." && sanitized !== "..") {
|
|
500
|
+
const currentExt = extname(sanitized);
|
|
501
|
+
const extension = sanitizeFileExtension(currentExt.slice(1));
|
|
502
|
+
const stem = extension
|
|
503
|
+
? sanitizeFileName(sanitized.slice(0, -currentExt.length)) || fallbackStem
|
|
504
|
+
: sanitized;
|
|
505
|
+
return {
|
|
506
|
+
stem,
|
|
507
|
+
extension,
|
|
508
|
+
urlFormatExtension,
|
|
509
|
+
};
|
|
510
|
+
}
|
|
511
|
+
return {
|
|
512
|
+
stem: fallbackStem,
|
|
513
|
+
extension: "",
|
|
514
|
+
urlFormatExtension,
|
|
515
|
+
};
|
|
516
|
+
}
|
|
517
|
+
|
|
518
|
+
async function ensureParentDirectory(outputPath, displayName) {
|
|
519
|
+
const parent = dirname(outputPath);
|
|
520
|
+
const existingParent = await stat(parent).catch(ignoreMissing);
|
|
521
|
+
if (existingParent && !existingParent.isDirectory()) {
|
|
522
|
+
throw new Error(`${displayName} must be a directory for xhs download-media.`);
|
|
523
|
+
}
|
|
524
|
+
if (!existingParent) {
|
|
525
|
+
await mkdir(parent, { recursive: true });
|
|
526
|
+
}
|
|
527
|
+
}
|
|
528
|
+
|
|
529
|
+
function buildSkippedExistingResult(url, outputPath, outputBytes) {
|
|
530
|
+
return {
|
|
531
|
+
platform: "xhs",
|
|
532
|
+
action: "download-media",
|
|
533
|
+
status: "skipped_existing",
|
|
534
|
+
url,
|
|
535
|
+
output_path: outputPath,
|
|
536
|
+
output_bytes: outputBytes,
|
|
537
|
+
};
|
|
538
|
+
}
|
|
539
|
+
|
|
540
|
+
function parseContentRange(value) {
|
|
541
|
+
if (!value) {
|
|
542
|
+
return undefined;
|
|
543
|
+
}
|
|
544
|
+
const trimmed = String(value).trim();
|
|
545
|
+
const unsatisfied = /^bytes\s+\*\/(\d+)$/i.exec(trimmed);
|
|
546
|
+
if (unsatisfied) {
|
|
547
|
+
return {
|
|
548
|
+
start: undefined,
|
|
549
|
+
end: undefined,
|
|
550
|
+
total: Number.parseInt(unsatisfied[1], 10),
|
|
551
|
+
};
|
|
552
|
+
}
|
|
553
|
+
const matched = /^bytes\s+(\d+)-(\d+)\/(\d+|\*)$/i.exec(trimmed);
|
|
554
|
+
if (!matched) {
|
|
555
|
+
return undefined;
|
|
556
|
+
}
|
|
557
|
+
return {
|
|
558
|
+
start: Number.parseInt(matched[1], 10),
|
|
559
|
+
end: Number.parseInt(matched[2], 10),
|
|
560
|
+
total: matched[3] === "*" ? undefined : Number.parseInt(matched[3], 10),
|
|
561
|
+
};
|
|
562
|
+
}
|
|
563
|
+
|
|
564
|
+
function isUsableContentRange(contentRange, contentLength) {
|
|
565
|
+
if (
|
|
566
|
+
contentRange?.start === undefined ||
|
|
567
|
+
contentRange.end === undefined ||
|
|
568
|
+
contentRange.total === undefined ||
|
|
569
|
+
!Number.isSafeInteger(contentRange.start) ||
|
|
570
|
+
!Number.isSafeInteger(contentRange.end) ||
|
|
571
|
+
!Number.isSafeInteger(contentRange.total) ||
|
|
572
|
+
contentRange.end < contentRange.start ||
|
|
573
|
+
contentRange.end >= contentRange.total
|
|
574
|
+
) {
|
|
575
|
+
return false;
|
|
576
|
+
}
|
|
577
|
+
const rangeBytes = contentRange.end - contentRange.start + 1;
|
|
578
|
+
return contentLength === undefined || contentLength === rangeBytes;
|
|
579
|
+
}
|
|
580
|
+
|
|
581
|
+
function shouldRetryDownloadError(error) {
|
|
582
|
+
return (
|
|
583
|
+
error instanceof XhsRetryableDownloadError ||
|
|
584
|
+
isAbortLikeError(error) ||
|
|
585
|
+
isNetworkLikeError(error)
|
|
586
|
+
);
|
|
587
|
+
}
|
|
588
|
+
|
|
589
|
+
function isNetworkLikeError(error) {
|
|
590
|
+
return (
|
|
591
|
+
error?.name === "TypeError" ||
|
|
592
|
+
["ECONNRESET", "ECONNREFUSED", "EPIPE", "ETIMEDOUT"].includes(error?.code)
|
|
593
|
+
);
|
|
594
|
+
}
|
|
595
|
+
|
|
596
|
+
function isDownloadTooLargeError(error) {
|
|
597
|
+
return error?.message === "Media download is too large for local media processing.";
|
|
598
|
+
}
|
|
599
|
+
|
|
600
|
+
function normalizedContentType(value) {
|
|
601
|
+
if (!value) {
|
|
602
|
+
return null;
|
|
603
|
+
}
|
|
604
|
+
return String(value).split(";")[0].trim().toLowerCase() || null;
|
|
605
|
+
}
|
|
606
|
+
|
|
607
|
+
function headerValue(headers, name) {
|
|
608
|
+
if (!headers || typeof headers.get !== "function") {
|
|
609
|
+
return null;
|
|
610
|
+
}
|
|
611
|
+
return headers.get(name);
|
|
612
|
+
}
|
|
613
|
+
|
|
614
|
+
function sanitizeFileName(value) {
|
|
615
|
+
const normalized = String(value || "")
|
|
616
|
+
.replace(/[\\/:*?"<>|\u0000-\u001f]/g, "_")
|
|
617
|
+
.replace(/\s+/g, " ")
|
|
618
|
+
.trim();
|
|
619
|
+
return normalized.slice(0, 120);
|
|
620
|
+
}
|
|
621
|
+
|
|
622
|
+
function sanitizeFileExtension(value) {
|
|
623
|
+
const normalized = String(value || "").toLowerCase().replace(/[^a-z0-9]/g, "");
|
|
624
|
+
if (!normalized || normalized.length > 8) {
|
|
625
|
+
return "";
|
|
626
|
+
}
|
|
627
|
+
return normalized;
|
|
628
|
+
}
|
|
629
|
+
|
|
630
|
+
function formatMediaFileName(stem, extension) {
|
|
631
|
+
return `${stem}.${sanitizeFileExtension(extension) || "bin"}`;
|
|
632
|
+
}
|
|
633
|
+
|
|
634
|
+
function mediaExtensionFromContentType(contentType) {
|
|
635
|
+
return XHS_CONTENT_TYPE_EXTENSIONS.get(contentType) || "";
|
|
636
|
+
}
|
|
637
|
+
|
|
638
|
+
function inferUrlFormatExtension(parsedUrl) {
|
|
639
|
+
const matched = /(?:^|[/?&])format\/([a-z0-9]+)(?:[/?&#]|$)/i.exec(
|
|
640
|
+
`${parsedUrl.pathname}${parsedUrl.search}`
|
|
641
|
+
);
|
|
642
|
+
if (!matched) {
|
|
643
|
+
return "";
|
|
644
|
+
}
|
|
645
|
+
return XHS_URL_FORMAT_EXTENSIONS.get(sanitizeFileExtension(matched[1])) || "";
|
|
646
|
+
}
|
|
647
|
+
|
|
648
|
+
function isExistingOutputPathError(error) {
|
|
649
|
+
return error?.code === "EEXIST" || error?.code === "EISDIR";
|
|
650
|
+
}
|
|
651
|
+
|
|
652
|
+
function shouldFallbackToCopyAfterLinkError(error) {
|
|
653
|
+
return ["EPERM", "ENOTSUP", "ENOSYS", "EOPNOTSUPP", "EXDEV"].includes(
|
|
654
|
+
error?.code
|
|
655
|
+
);
|
|
656
|
+
}
|
|
657
|
+
|
|
658
|
+
async function buildExistingOutputPathError(outputPath) {
|
|
659
|
+
const existingOutput = await stat(outputPath).catch(ignoreMissing);
|
|
660
|
+
if (existingOutput?.isDirectory()) {
|
|
661
|
+
return new Error("--output must be a file path for xhs download-media.");
|
|
662
|
+
}
|
|
663
|
+
return new Error("XHS media download output file already exists.");
|
|
664
|
+
}
|
|
665
|
+
|
|
666
|
+
function formatTimestamp(value) {
|
|
667
|
+
const date = value instanceof Date ? value : new Date(value);
|
|
668
|
+
const pad = (part) => String(part).padStart(2, "0");
|
|
669
|
+
return [
|
|
670
|
+
date.getFullYear(),
|
|
671
|
+
pad(date.getMonth() + 1),
|
|
672
|
+
pad(date.getDate()),
|
|
673
|
+
pad(date.getHours()),
|
|
674
|
+
pad(date.getMinutes()),
|
|
675
|
+
pad(date.getSeconds()),
|
|
676
|
+
].join("");
|
|
677
|
+
}
|
|
678
|
+
|
|
679
|
+
function safeDecodeURIComponent(value) {
|
|
680
|
+
try {
|
|
681
|
+
return decodeURIComponent(value);
|
|
682
|
+
} catch {
|
|
683
|
+
return value;
|
|
684
|
+
}
|
|
685
|
+
}
|
|
686
|
+
|
|
687
|
+
function expandHome(path) {
|
|
688
|
+
if (!path || path === "~") {
|
|
689
|
+
return homedir();
|
|
690
|
+
}
|
|
691
|
+
if (path.startsWith("~/")) {
|
|
692
|
+
return join(homedir(), path.slice(2));
|
|
693
|
+
}
|
|
694
|
+
return path;
|
|
695
|
+
}
|
|
696
|
+
|
|
697
|
+
function ignoreMissing(error) {
|
|
698
|
+
if (error?.code === "ENOENT" || error?.code === "ENOTDIR") {
|
|
699
|
+
return undefined;
|
|
700
|
+
}
|
|
701
|
+
throw error;
|
|
702
|
+
}
|
|
703
|
+
|
|
704
|
+
function ignoreError() {}
|
|
705
|
+
|
|
706
|
+
function sleep(ms) {
|
|
707
|
+
if (!Number.isFinite(ms) || ms <= 0) {
|
|
708
|
+
return Promise.resolve();
|
|
709
|
+
}
|
|
710
|
+
return new Promise((resolvePromise) => {
|
|
711
|
+
setTimeout(resolvePromise, ms);
|
|
712
|
+
});
|
|
713
|
+
}
|
|
714
|
+
|
|
715
|
+
class XhsRetryableDownloadError extends Error {}
|
|
716
|
+
|
|
717
|
+
class XhsExpiredMediaLinkError extends Error {
|
|
718
|
+
constructor() {
|
|
719
|
+
super(
|
|
720
|
+
"XHS media link is unavailable or expired. Get a fresh media URL from the XHS detail result and retry."
|
|
721
|
+
);
|
|
722
|
+
}
|
|
723
|
+
}
|
package/package.json
CHANGED
|
@@ -148,6 +148,7 @@ If MCP tools are already available in the current agent, use one of these tools:
|
|
|
148
148
|
Return factual fields such as title or description, content, author, publish time, interaction counts, images, and media summary when available.
|
|
149
149
|
For XHS detail results, in every use of a returned `note_url`, such as final answers, display, references, storage, output, or forwarding, preserve it exactly as the full URL, including `xsec_token` query parameters. Do not modify, truncate, redact, mask, normalize, rebuild, or synthesize the URL from `note_id`; if `note_url` is null, show the `note_id` or say that no directly openable full link is available.
|
|
150
150
|
For XHS `note_id`, copy the complete 24-character lowercase hexadecimal ID exactly; do not pass or display only a prefix.
|
|
151
|
+
When the user wants to save XHS images or videos after detail, pass each returned `image_items[].image_url` or `video.video_url` to `npx -y socialdatax-skills@latest xhs download-media --url "<media_url>" --output-dir <directory> --pretty`; this local save command does not require `SOCIALDATAX_API_KEY`.
|
|
151
152
|
For Douyin detail, include `content_type` when available.
|
|
152
153
|
For Douyin detail, use `images` for image/text posts; `video` is the platform player resource and may be audio for image/text posts; `music` is the bound music or original-sound asset.
|
|
153
154
|
Detail access is read-only and does not provide account actions.
|