wechat-article-parser 0.0.1__tar.gz

wechat_article_parser-0.0.1/.gitignore

@@ -0,0 +1,27 @@
+# Python
+__pycache__/
+*.py[cod]
+*.egg-info/
+*.egg
+dist/
+build/
+
+# Virtual environments
+.venv/
+venv/
+
+# IDE
+.idea/
+.vscode/
+*.swp
+*.swo
+
+# pytest
+.pytest_cache/
+
+# Claude Code
+.claude/
+
+# OS
+.DS_Store
+Thumbs.db

wechat_article_parser-0.0.1/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Gang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

wechat_article_parser-0.0.1/PKG-INFO

@@ -0,0 +1,183 @@
+Metadata-Version: 2.4
+Name: wechat-article-parser
+Version: 0.0.1
+Summary: WeChat MP article parser - extract metadata and content from WeChat public account articles
+License-Expression: MIT
+License-File: LICENSE
+Requires-Python: >=3.10
+Requires-Dist: beautifulsoup4>=4.12
+Requires-Dist: httpx>=0.27
+Requires-Dist: markdownify>=0.13
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# wechat-article-parser
+
+微信公众号文章解析器。传入一个公众号文章链接，自动抓取页面并提取公众号信息、文章元数据及正文的 Markdown 内容。
+
+支持的文章类型：
+
+- 标准富文本文章
+- 转载式文章
+- 纯文本文章
+- 视频分享类文章
+- 小红书风格图片轮播文章
+- 全屏布局短文本文章
+
+## 安装
+
+```bash
+pip install wechat-article-parser
+```
+
+**本地开发安装：**
+
+```bash
+git clone https://github.com/huanjuedadehen/wechat-article-parser.git
+cd wechat-article-parser
+pip install -e ".[dev]"
+```
+
+依赖项（会自动安装）：
+
+- Python >= 3.10
+- httpx
+- beautifulsoup4
+- markdownify
+
+## 使用方式
+
+### 同步调用
+
+```python
+from wechat_article_parser import parse
+
+result = parse("https://mp.weixin.qq.com/s/xxxxx")
+print(result.article_title)
+print(result.article_markdown)
+```
+
+### 异步调用
+
+```python
+from wechat_article_parser import parse_async
+
+result = await parse_async("https://mp.weixin.qq.com/s/xxxxx")
+print(result.article_title)
+print(result.article_markdown)
+```
+
+### 可选参数
+
+两个方法都支持以下可选参数：
+
+- `timeout`：请求超时时间，单位秒，默认 15
+- `user_agent`：自定义 User-Agent，不传则使用内置默认值
+
+```python
+result = parse(
+    "https://mp.weixin.qq.com/s/xxxxx",
+    timeout=30,
+    user_agent="MyBot/1.0",
+)
+```
+
+## 返回结构
+
+`parse` 和 `parse_async` 均返回 `ArticleResult` 数据类，包含以下字段：
+
+| 字段 | 类型 | 说明 |
+|---|---|---|
+| `mp_id_b64` | `str` | 公众号 ID（Base64 编码原始值） |
+| `mp_id` | `int` | 公众号 ID（解码后的整数） |
+| `mp_name` | `str` | 公众号名称 |
+| `mp_alias` | `str` | 公众号别名 |
+| `mp_image` | `str` | 公众号头像链接 |
+| `mp_description` | `str` | 公众号简介 |
+| `article_id` | `str` | 文章 ID |
+| `article_msg_id` | `int` | 文章所在的群发消息 ID |
+| `article_idx` | `int` | 群发图文中的位置（从 1 开始） |
+| `article_sn` | `str` | 文章签名（防伪校验） |
+| `article_title` | `str` | 文章标题 |
+| `article_cover_image` | `str` | 文章封面图链接 |
+| `article_description` | `str` | 文章摘要 |
+| `article_markdown` | `str` | 文章正文的 Markdown 内容 |
+| `article_publish_time` | `int` | 发布时间（Unix 时间戳） |
+| `images` | `list[str]` | 文章中提取的所有图片链接 |
+| `is_valid` | `bool` | 关键字段是否全部解析成功（属性） |
+
+`is_valid` 为 `True` 的条件：`mp_id`、`mp_name`、`article_id`、`article_msg_id`、`article_idx`、`article_sn`、`article_title`、`article_markdown`、`article_publish_time` 均不为空/零。
+
+## 异常处理
+
+### WeChatVerifyError
+
+当请求频率过高或 IP 被限流时，微信会返回验证码页面而非文章内容。此时会抛出 `WeChatVerifyError`：
+
+```python
+from wechat_article_parser import parse, WeChatVerifyError
+
+try:
+    result = parse("https://mp.weixin.qq.com/s/xxxxx")
+except WeChatVerifyError:
+    print("触发了微信人机验证，请稍后重试或更换 IP")
+```
+
+### httpx.HTTPStatusError
+
+当 HTTP 请求返回非 2xx 状态码时，会抛出 `httpx.HTTPStatusError`：
+
+```python
+import httpx
+from wechat_article_parser import parse
+
+try:
+    result = parse("https://mp.weixin.qq.com/s/xxxxx")
+except httpx.HTTPStatusError as e:
+    print(f"请求失败: {e.response.status_code}")
+```
+
+### 解析不完整
+
+部分文章类型可能无法提取所有字段（如某些文章没有封面图或摘要）。这种情况不会抛出异常，但 `result.is_valid` 会返回 `False`。建议在业务逻辑中检查：
+
+```python
+result = parse("https://mp.weixin.qq.com/s/xxxxx")
+if not result.is_valid:
+    print("部分关键字段未能解析")
+```
+
+## 测试
+
+安装开发依赖：
+
+```bash
+pip install -e ".[dev]"
+```
+
+### 运行全部测试
+
+```bash
+pytest tests/test_parser.py -v -s
+```
+
+### 只运行同步 / 异步测试
+
+```bash
+pytest tests/test_parser.py -v -s -k "sync"
+pytest tests/test_parser.py -v -s -k "async"
+```
+
+### 测试单个链接 - 查看所有字段
+
+```bash
+pytest tests/test_parser.py::test_fetch_all -s --url "https://mp.weixin.qq.com/s/xxxxx"
+```
+
+### 测试单个链接 - 只看 Markdown 内容
+
+```bash
+pytest tests/test_parser.py::test_fetch_markdown -s --url "https://mp.weixin.qq.com/s/xxxxx"
+```

wechat_article_parser-0.0.1/README.md

@@ -0,0 +1,168 @@
+# wechat-article-parser
+
+微信公众号文章解析器。传入一个公众号文章链接，自动抓取页面并提取公众号信息、文章元数据及正文的 Markdown 内容。
+
+支持的文章类型：
+
+- 标准富文本文章
+- 转载式文章
+- 纯文本文章
+- 视频分享类文章
+- 小红书风格图片轮播文章
+- 全屏布局短文本文章
+
+## 安装
+
+```bash
+pip install wechat-article-parser
+```
+
+**本地开发安装：**
+
+```bash
+git clone https://github.com/huanjuedadehen/wechat-article-parser.git
+cd wechat-article-parser
+pip install -e ".[dev]"
+```
+
+依赖项（会自动安装）：
+
+- Python >= 3.10
+- httpx
+- beautifulsoup4
+- markdownify
+
+## 使用方式
+
+### 同步调用
+
+```python
+from wechat_article_parser import parse
+
+result = parse("https://mp.weixin.qq.com/s/xxxxx")
+print(result.article_title)
+print(result.article_markdown)
+```
+
+### 异步调用
+
+```python
+from wechat_article_parser import parse_async
+
+result = await parse_async("https://mp.weixin.qq.com/s/xxxxx")
+print(result.article_title)
+print(result.article_markdown)
+```
+
+### 可选参数
+
+两个方法都支持以下可选参数：
+
+- `timeout`：请求超时时间，单位秒，默认 15
+- `user_agent`：自定义 User-Agent，不传则使用内置默认值
+
+```python
+result = parse(
+    "https://mp.weixin.qq.com/s/xxxxx",
+    timeout=30,
+    user_agent="MyBot/1.0",
+)
+```
+
+## 返回结构
+
+`parse` 和 `parse_async` 均返回 `ArticleResult` 数据类，包含以下字段：
+
+| 字段 | 类型 | 说明 |
+|---|---|---|
+| `mp_id_b64` | `str` | 公众号 ID（Base64 编码原始值） |
+| `mp_id` | `int` | 公众号 ID（解码后的整数） |
+| `mp_name` | `str` | 公众号名称 |
+| `mp_alias` | `str` | 公众号别名 |
+| `mp_image` | `str` | 公众号头像链接 |
+| `mp_description` | `str` | 公众号简介 |
+| `article_id` | `str` | 文章 ID |
+| `article_msg_id` | `int` | 文章所在的群发消息 ID |
+| `article_idx` | `int` | 群发图文中的位置（从 1 开始） |
+| `article_sn` | `str` | 文章签名（防伪校验） |
+| `article_title` | `str` | 文章标题 |
+| `article_cover_image` | `str` | 文章封面图链接 |
+| `article_description` | `str` | 文章摘要 |
+| `article_markdown` | `str` | 文章正文的 Markdown 内容 |
+| `article_publish_time` | `int` | 发布时间（Unix 时间戳） |
+| `images` | `list[str]` | 文章中提取的所有图片链接 |
+| `is_valid` | `bool` | 关键字段是否全部解析成功（属性） |
+
+`is_valid` 为 `True` 的条件：`mp_id`、`mp_name`、`article_id`、`article_msg_id`、`article_idx`、`article_sn`、`article_title`、`article_markdown`、`article_publish_time` 均不为空/零。
+
+## 异常处理
+
+### WeChatVerifyError
+
+当请求频率过高或 IP 被限流时，微信会返回验证码页面而非文章内容。此时会抛出 `WeChatVerifyError`：
+
+```python
+from wechat_article_parser import parse, WeChatVerifyError
+
+try:
+    result = parse("https://mp.weixin.qq.com/s/xxxxx")
+except WeChatVerifyError:
+    print("触发了微信人机验证，请稍后重试或更换 IP")
+```
+
+### httpx.HTTPStatusError
+
+当 HTTP 请求返回非 2xx 状态码时，会抛出 `httpx.HTTPStatusError`：
+
+```python
+import httpx
+from wechat_article_parser import parse
+
+try:
+    result = parse("https://mp.weixin.qq.com/s/xxxxx")
+except httpx.HTTPStatusError as e:
+    print(f"请求失败: {e.response.status_code}")
+```
+
+### 解析不完整
+
+部分文章类型可能无法提取所有字段（如某些文章没有封面图或摘要）。这种情况不会抛出异常，但 `result.is_valid` 会返回 `False`。建议在业务逻辑中检查：
+
+```python
+result = parse("https://mp.weixin.qq.com/s/xxxxx")
+if not result.is_valid:
+    print("部分关键字段未能解析")
+```
+
+## 测试
+
+安装开发依赖：
+
+```bash
+pip install -e ".[dev]"
+```
+
+### 运行全部测试
+
+```bash
+pytest tests/test_parser.py -v -s
+```
+
+### 只运行同步 / 异步测试
+
+```bash
+pytest tests/test_parser.py -v -s -k "sync"
+pytest tests/test_parser.py -v -s -k "async"
+```
+
+### 测试单个链接 - 查看所有字段
+
+```bash
+pytest tests/test_parser.py::test_fetch_all -s --url "https://mp.weixin.qq.com/s/xxxxx"
+```
+
+### 测试单个链接 - 只看 Markdown 内容
+
+```bash
+pytest tests/test_parser.py::test_fetch_markdown -s --url "https://mp.weixin.qq.com/s/xxxxx"
+```

wechat_article_parser-0.0.1/pyproject.toml

@@ -0,0 +1,28 @@
+[build-system]
+requires = ["hatchling"]
+build-backend = "hatchling.build"
+
+[project]
+name = "wechat-article-parser"
+version = "0.0.1"
+description = "WeChat MP article parser - extract metadata and content from WeChat public account articles"
+readme = "README.md"
+requires-python = ">=3.10"
+license = "MIT"
+dependencies = [
+    "httpx>=0.27",
+    "beautifulsoup4>=4.12",
+    "markdownify>=0.13",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8.0",
+    "pytest-asyncio>=0.24",
+]
+
+[tool.hatch.build.targets.wheel]
+packages = ["src/wechat_article_parser"]
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"

wechat_article_parser-0.0.1/src/wechat_article_parser/__init__.py

@@ -0,0 +1,4 @@
+from .models import ArticleResult, WeChatVerifyError
+from .parser import parse, parse_async
+
+__all__ = ["parse", "parse_async", "ArticleResult", "WeChatVerifyError"]

wechat_article_parser-0.0.1/src/wechat_article_parser/models.py

@@ -0,0 +1,49 @@
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+
+
+class WeChatVerifyError(Exception):
+    """微信返回了验证码/人机验证页面，而非文章内容时抛出此异常。"""
+
+
+@dataclass
+class ArticleResult:
+    """微信公众号文章的解析结果。"""
+
+    # 公众号信息
+    mp_id_b64: str = ""
+    mp_id: int = 0
+    mp_name: str = ""
+    mp_alias: str = ""
+    mp_image: str = ""
+    mp_description: str = ""
+
+    # 文章信息
+    article_id: str = ""
+    article_msg_id: int = 0
+    article_idx: int = 0
+    article_sn: str = ""
+    article_title: str = ""
+    article_cover_image: str = ""
+    article_description: str = ""
+    article_markdown: str = ""
+    article_publish_time: int = 0
+
+    # 文章中提取的图片列表
+    images: list[str] = field(default_factory=list)
+
+    @property
+    def is_valid(self) -> bool:
+        """检查关键字段是否解析成功。"""
+        return bool(
+            self.mp_id
+            and self.mp_name
+            and self.article_id
+            and self.article_msg_id
+            and self.article_idx
+            and self.article_sn
+            and self.article_title
+            and self.article_markdown
+            and self.article_publish_time
+        )

wechat_article_parser-0.0.1/src/wechat_article_parser/parser.py

@@ -0,0 +1,472 @@
+"""微信公众号文章解析器核心模块。"""
+
+from __future__ import annotations
+
+import base64
+import html as html_module
+import re
+from urllib.parse import unquote
+
+import httpx
+from bs4 import BeautifulSoup, Tag
+from markdownify import MarkdownConverter
+
+from .models import ArticleResult, WeChatVerifyError
+
+_USER_AGENT = (
+    "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) "
+    "AppleWebKit/537.36 (KHTML, like Gecko) "
+    "Chrome/139.0.0.0 Safari/537.36"
+)
+_TIMEOUT = 15
+
+
+# ---------------------------------------------------------------------------
+# Markdown 转换器
+# ---------------------------------------------------------------------------
+
+class _MarkdownConverter(MarkdownConverter):
+    def convert_span(self, el, text, parent_tags):
+        return text.strip()
+
+    def convert_p(self, el, text, parent_tags):
+        text = " ".join(text.split())
+        return super().convert_p(el, text, parent_tags)
+
+
+def _to_markdown(html: str) -> str:
+    return _MarkdownConverter().convert(html)
+
+
+# ---------------------------------------------------------------------------
+# 文本解码辅助函数
+# ---------------------------------------------------------------------------
+
+def _strip_html_tags(text: str) -> str:
+    """移除文本中的 HTML 标签，只保留纯文本内容。"""
+    return BeautifulSoup(text, "html.parser").get_text()
+
+
+def _decode_hex_escapes(text: str) -> str:
+    return re.sub(r"\\x([0-9a-fA-F]{2})", lambda m: chr(int(m.group(1), 16)), text)
+
+
+def _decode_text(text: str, *, preserve_newlines: bool = False) -> str:
+    if not text:
+        return ""
+    text = _decode_hex_escapes(text)
+    text = html_module.unescape(text)
+    text = text.replace("&", "&")
+    if preserve_newlines:
+        text = text.replace("\r", "").replace("\n", "<br>")
+    else:
+        text = text.replace("\r", "").replace("\n", "")
+    return re.sub(r"\s+", " ", text)
+
+
+# ---------------------------------------------------------------------------
+# 图片链接标准化
+# ---------------------------------------------------------------------------
+
+def _normalize_image_url(url: str) -> str:
+    """将微信图片链接标准化为 640px 宽度版本。"""
+    parts = url.split("/")
+    if len(parts) >= 5:
+        return f"{'/'.join(parts[:5])}/640"
+    return url
+
+
+def _extract_picture_cdn_urls(script_text: str) -> list[str]:
+    """从 picture_page_info_list 中只提取正文图片的 cdn_url，排除 watermark_info 和 share_cover 中的。"""
+    urls: list[str] = []
+    seen: set[str] = set()
+    for m in re.finditer(r"(watermark_info|share_cover)?\s*(?::\s*\{[^}]*?)?\bcdn_url:\s*'([^']*)'", script_text):
+        prefix = m.group(1)
+        url = m.group(2)
+        if prefix or not url:
+            continue
+        normalized = _normalize_image_url(url)
+        if normalized not in seen:
+            seen.add(normalized)
+            urls.append(normalized)
+    return urls
+
+
+# ---------------------------------------------------------------------------
+# 内部提取函数
+# ---------------------------------------------------------------------------
+
+def _extract_article_id(url: str) -> str:
+    parts = url.split("/")
+    if len(parts) == 5 and len(parts[4]) == 22:
+        return parts[4]
+    return ""
+
+
+def _extract_meta(soup: BeautifulSoup, result: ArticleResult) -> None:
+    for prop, attr in [
+        ("og:title", "article_title"),
+        ("og:image", "article_cover_image"),
+        ("og:description", "article_description"),
+    ]:
+        tag = soup.find("meta", attrs={"property": prop})
+        if tag and tag.get("content"):
+            value = tag["content"]
+            if attr != "article_cover_image":
+                value = _decode_text(value)
+                value = _strip_html_tags(value)
+            if attr == "article_description" and len(value) > 2048:
+                value = value[:2048]
+            setattr(result, attr, value)
+
+
+def _extract_rich_text_meta(script_text: str, result: ArticleResult) -> None:
+    """从富文本文章的 script 标签中提取元数据。"""
+    if "var hd_head_img" in script_text:
+        m = re.search(r'var hd_head_img = "([^"]+)"', script_text)
+        if m:
+            result.mp_image = m.group(1)
+
+        m = re.search(r"var nickname = htmlDecode\((.*)\);", script_text)
+        if m:
+            result.mp_name = m.group(1).strip('"').strip("'")
+
+        m = re.search(r'var profile_signature = "([^"]+)"', script_text)
+        if m:
+            result.mp_description = m.group(1)
+
+        m = re.search(r"alias: '([^']+)'", script_text)
+        if m:
+            result.mp_alias = m.group(1)
+
+    if "var oriCreateTime" in script_text:
+        m = re.search(r"var oriCreateTime = '(\d+)'", script_text)
+        if m:
+            result.article_publish_time = int(m.group(1))
+
+    if "window.__allowLoadResFromMp" in script_text:
+        variables: dict[str, str] = {}
+        for m in re.finditer(r"var\s+(\w+)\s*=\s*(.*?);", script_text):
+            literals = re.findall(r'"(.*?)"', m.group(2))
+            variables[m.group(1)] = next((s for s in literals if s.strip()), "")
+
+        biz = variables.get("biz", "")
+        if biz:
+            result.mp_id_b64 = biz
+            try:
+                result.mp_id = int(base64.b64decode(biz).decode())
+            except Exception:
+                pass
+        result.article_msg_id = int(variables["mid"]) if variables.get("mid") else 0
+        result.article_idx = int(variables["idx"]) if variables.get("idx") else 0
+        result.article_sn = variables.get("sn", "")
+
+
+def _extract_swiper_meta(script_text: str, result: ArticleResult) -> None:
+    """从图片轮播 / 纯文本 / 视频分享页面的 script 标签中提取元数据。"""
+    if "window.__initCgiDataConfig =" in script_text:
+        m = re.search(r"d\.hd_head_img.*?:\s*'([^']+)'", script_text)
+        if m:
+            result.mp_image = m.group(1)
+
+        m = re.search(r"d\.nick_name.*?:\s*'([^']+)'", script_text)
+        if m:
+            result.mp_name = m.group(1).strip('"').strip("'")
+
+        m = re.search(r"d\.biz.*?:\s*'([^']+)'", script_text)
+        if m:
+            biz = m.group(1)
+            result.mp_id_b64 = biz
+            try:
+                result.mp_id = int(base64.b64decode(biz).decode())
+            except Exception:
+                pass
+
+        m = re.search(r"d\.mid.*?:\s*'([^']+)'", script_text)
+        if m:
+            result.article_msg_id = int(m.group(1))
+
+        m = re.search(r"d\.idx.*?:\s*'([^']+)'", script_text)
+        if m:
+            result.article_idx = int(m.group(1))
+
+        m = re.search(r"d\.sn.*?:\s*'([^']+)'", script_text)
+        if m:
+            result.article_sn = m.group(1)
+
+        m = re.search(r"d\.create_time.*?:\s*'([^']+)'", script_text)
+        if m:
+            result.article_publish_time = int(m.group(1))
+
+        if not result.article_id:
+            m = re.search(r"d\.msg_link.*?:\s*'([^']+)'", script_text)
+            if m:
+                parts = m.group(1).split("/")
+                if len(parts) == 5 and len(parts[4]) == 22:
+                    result.article_id = parts[4]
+
+    if "window.alias =" in script_text:
+        m = re.search(r'window.alias = "([^"]+)"', script_text)
+        if m:
+            result.mp_alias = m.group(1)
+
+
+# ---------------------------------------------------------------------------
+# 内容提取
+# ---------------------------------------------------------------------------
+
+def _extract_rich_media_content(content_tag: Tag, result: ArticleResult) -> None:
+    """将 rich_media_content 区块转换为 Markdown。"""
+    soup = BeautifulSoup(content_tag.prettify(), "html.parser")
+    seen: set[str] = set()
+
+    # 处理 <img> 标签
+    to_remove = []
+    for img in soup.find_all("img"):
+        src = img.get("src") or img.get("data-src")
+        if src and src.startswith("http"):
+            normalized = _normalize_image_url(src)
+            img["src"] = normalized
+            if normalized not in seen:
+                seen.add(normalized)
+                result.images.append(normalized)
+        else:
+            to_remove.append(img)
+    for tag in to_remove:
+        tag.decompose()
+
+    # 处理含有 background-image 的 <svg> 标签
+    # 这些通常是装饰性元素（分隔线、底纹等），转为 <img> 用于 Markdown 渲染，但不计入 images 列表
+    to_remove = []
+    for svg in soup.find_all("svg"):
+        style = svg.get("style", "")
+        if "background-image" in style:
+            m = re.search(r'url\("([^"]+)"\)', style)
+            if m:
+                normalized = _normalize_image_url(m.group(1))
+                new_img = soup.new_tag("img", src=normalized)
+                svg.replace_with(new_img)
+                continue
+        to_remove.append(svg)
+    for tag in to_remove:
+        tag.decompose()
+
+    result.article_markdown = _to_markdown(soup.prettify())
+
+
+def _extract_repost_content(content_tag: Tag, result: ArticleResult) -> None:
+    """从转载式文章中提取内容。"""
+    soup = BeautifulSoup(content_tag.prettify(), "html.parser")
+    notice = soup.find("p", id="js_share_notice")
+    if not notice:
+        return
+
+    m = re.search(r'innerHTML = "([^"]+)"', str(notice))
+    if not m:
+        return
+
+    text = _decode_text(m.group(1), preserve_newlines=True)
+    html_content = f"<p>{text}</p>"
+
+    share_link = soup.find("span", id="js_share_source")
+    if share_link:
+        href = share_link.get("data-url")
+        if href:
+            html_content += f'<p><a href="{href}">查看原文</a></p>'
+
+    result.article_markdown = _to_markdown(html_content)
+
+
+def _extract_plain_text_content(soup: BeautifulSoup, result: ArticleResult) -> None:
+    """从纯文本文章中提取内容。"""
+    for script in soup.find_all("script", attrs={"type": "text/javascript"}):
+        if "var TextContentNoEncode =" in script.text:
+            m = re.search(
+                r"var ContentNoEncode = window\.a_value_which_never_exists \|\| '([^']+)';",
+                script.text,
+            )
+            if m:
+                text = _decode_text(m.group(1), preserve_newlines=True)
+                text = unquote(text)
+                result.article_markdown = _to_markdown(f"<p>{text}</p>")
+
+
+def _extract_swiper_content(soup: BeautifulSoup, result: ArticleResult) -> None:
+    """从小红书风格的图片轮播文章中提取内容。"""
+    for script in soup.find_all("script", attrs={"type": "text/javascript"}):
+        if "window.picture_page_info_list =" not in script.text:
+            continue
+
+        result.images = _extract_picture_cdn_urls(script.text)
+
+        html_parts = [f'<img src="{img}" /><br>' for img in result.images]
+
+        m = re.search(r'window.desc = "([^"]+)"', script.text)
+        if m:
+            text = _decode_text(m.group(1), preserve_newlines=True)
+            html_parts.append(f"<p>{text}</p>")
+
+        if html_parts:
+            result.article_markdown = _to_markdown("".join(html_parts))
+
+
+def _extract_fullscreen_content(soup: BeautifulSoup, result: ArticleResult) -> None:
+    """从全屏布局文章中提取内容（appmsg_type 10002）。
+
+    此类文章的文本存储在 text_page_info.content 中，通过 JsDecode() 编码；
+    图片存储在 picture_page_info_list 的 cdn_url 字段中。
+    """
+    for script in soup.find_all("script", attrs={"type": "text/javascript"}):
+        if "picture_page_info_list" not in script.text:
+            continue
+
+        html_parts: list[str] = []
+
+        # 从 picture_page_info_list 中提取图片
+        result.images = _extract_picture_cdn_urls(script.text)
+        for img in result.images:
+            html_parts.append(f'<img src="{img}" /><br>')
+
+        # 从 text_page_info.content_noencode 或 content 中提取文本
+        for field in ("content_noencode", "content"):
+            m = re.search(
+                rf"{field}:\s*JsDecode\('(.*?)'\)",
+                script.text,
+                re.DOTALL,
+            )
+            if m:
+                text = _decode_text(m.group(1), preserve_newlines=True)
+                text = unquote(text)
+                html_parts.append(f"<p>{text}</p>")
+                break
+
+        if html_parts:
+            result.article_markdown = _to_markdown("".join(html_parts))
+        return
+
+
+# ---------------------------------------------------------------------------
+# 主解析流程
+# ---------------------------------------------------------------------------
+
+def _parse_html(url: str, html: str) -> ArticleResult:
+    """将原始 HTML 解析为 ArticleResult。"""
+    # 检测微信验证码/人机验证页面
+    if "secitptpage/template/verify.js" in html or "register_code" in html[:3000]:
+        raise WeChatVerifyError(
+            f"WeChat returned a verification page for {url}. "
+            "This usually means the IP has been rate-limited. "
+            "Try again later or use a different IP."
+        )
+
+    result = ArticleResult()
+    soup = BeautifulSoup(html, "html.parser")
+
+    result.article_id = _extract_article_id(url)
+    _extract_meta(soup, result)
+
+    scripts = soup.find_all("script", attrs={"type": "text/javascript"})
+
+    # 尝试解析标准富文本文章
+    content = soup.find("div", class_="rich_media_content")
+    if content:
+        for s in scripts:
+            _extract_rich_text_meta(s.text, result)
+        _extract_rich_media_content(content, result)
+        return result
+
+    # 尝试解析转载式文章
+    content = soup.find("div", class_="original_page")
+    if content:
+        for s in scripts:
+            _extract_rich_text_meta(s.text, result)
+        _extract_repost_content(content, result)
+        return result
+
+    # 尝试解析纯文本文章
+    content = soup.find("p", id="js_text_desc")
+    if content:
+        for s in scripts:
+            _extract_swiper_meta(s.text, result)
+        _extract_plain_text_content(soup, result)
+        if result.article_title and len(result.article_title) > 50:
+            short = result.article_title.split("。")[0]
+            result.article_title = short if len(short) <= 50 else result.article_title[:30]
+        return result
+
+    # 尝试解析视频分享类文章
+    content = soup.find("div", id="js_common_share_desc_wrap")
+    if content:
+        for s in scripts:
+            _extract_swiper_meta(s.text, result)
+        _extract_plain_text_content(soup, result)
+        return result
+
+    # 尝试解析小红书风格图片轮播文章
+    content = soup.find("div", class_="share_media_swiper_content")
+    if content:
+        for s in scripts:
+            _extract_swiper_meta(s.text, result)
+        _extract_swiper_content(soup, result)
+        return result
+
+    # 尝试解析全屏布局文章（如短文本帖子，appmsg_type 10002）
+    content = soup.find("div", id="js_fullscreen_layout_padding")
+    if content:
+        for s in scripts:
+            _extract_swiper_meta(s.text, result)
+        _extract_fullscreen_content(soup, result)
+        if result.article_title and len(result.article_title) > 50:
+            short = result.article_title.split("。")[0]
+            result.article_title = short if len(short) <= 50 else result.article_title[:30]
+        return result
+
+    # 兜底：尽可能提取元数据
+    for s in scripts:
+        _extract_rich_text_meta(s.text, result)
+        _extract_swiper_meta(s.text, result)
+
+    return result
+
+
+# ---------------------------------------------------------------------------
+# 公开接口
+# ---------------------------------------------------------------------------
+
+def parse(url: str, *, timeout: int = _TIMEOUT, user_agent: str | None = None) -> ArticleResult:
+    """抓取并解析微信公众号文章（同步方式）。
+
+    Args:
+        url: 微信文章链接。
+        timeout: 请求超时时间（秒）。
+        user_agent: 自定义 User-Agent，不传则使用内置默认值。
+
+    Returns:
+        包含解析数据的 ArticleResult。
+    """
+    ua = user_agent or _USER_AGENT
+    response = httpx.get(url, headers={"User-Agent": ua}, timeout=timeout, follow_redirects=True)
+    response.raise_for_status()
+    return _parse_html(url, response.text)
+
+
+async def parse_async(url: str, *, timeout: int = _TIMEOUT, user_agent: str | None = None) -> ArticleResult:
+    """抓取并解析微信公众号文章（异步方式）。
+
+    Args:
+        url: 微信文章链接。
+        timeout: 请求超时时间（秒）。
+        user_agent: 自定义 User-Agent，不传则使用内置默认值。
+
+    Returns:
+        包含解析数据的 ArticleResult。
+    """
+    ua = user_agent or _USER_AGENT
+    async with httpx.AsyncClient(
+        headers={"User-Agent": ua},
+        timeout=timeout,
+        follow_redirects=True,
+    ) as client:
+        response = await client.get(url)
+        response.raise_for_status()
+        return _parse_html(url, response.text)

wechat_article_parser-0.0.1/tests/conftest.py

@@ -0,0 +1,13 @@
+import pytest
+
+
+def pytest_addoption(parser):
+    parser.addoption("--url", default=None, help="微信公众号文章链接")
+
+
+@pytest.fixture
+def url(request):
+    value = request.config.getoption("--url")
+    if not value:
+        pytest.skip("需要通过 --url 参数传入链接")
+    return value

wechat_article_parser-0.0.1/tests/test_parser.py

@@ -0,0 +1,124 @@
+"""使用真实微信公众号文章链接测试 wechat_article_parser。"""
+
+import pytest
+
+from wechat_article_parser import ArticleResult, WeChatVerifyError, parse, parse_async
+
+TEST_URLS = [
+    "https://mp.weixin.qq.com/s/OkKlPnSbLOP9J3heC0CaPw",
+    "https://mp.weixin.qq.com/s/z7zNi_DayzevcTe0EUTv5g",
+    "https://mp.weixin.qq.com/s/9NjneDOJiteXBM9Cupb9Ig",
+    "https://mp.weixin.qq.com/s/0Wz3JeMbtWBL5iWJgYPS_Q",
+    "https://mp.weixin.qq.com/s/DmZXjgIzq5gBo3H-YtcjVw",
+    "https://mp.weixin.qq.com/s/80bysSCadvy9VbaovXBv2g",
+]
+
+
+@pytest.mark.parametrize("url", TEST_URLS)
+def test_parse_sync(url: str) -> None:
+    try:
+        result = parse(url)
+    except WeChatVerifyError:
+        pytest.skip("WeChat returned verification page (IP rate-limited)")
+        return
+    _assert_result(result, url)
+
+
+@pytest.mark.parametrize("url", TEST_URLS)
+@pytest.mark.asyncio
+async def test_parse_async(url: str) -> None:
+    try:
+        result = await parse_async(url)
+    except WeChatVerifyError:
+        pytest.skip("WeChat returned verification page (IP rate-limited)")
+        return
+    _assert_result(result, url)
+
+
+def _assert_result(result: ArticleResult, url: str) -> None:
+    print(f"\n{'='*60}")
+    print(f"URL: {url}")
+    print(f"公众号ID(B64):{result.mp_id_b64}")
+    print(f"公众号ID:     {result.mp_id}")
+    print(f"公众号名称:   {result.mp_name}")
+    print(f"公众号别名:   {result.mp_alias}")
+    print(
+        f"公众号图片:   {result.mp_image[:80]}..."
+        if result.mp_image
+        else "公众号图片:   (empty)"
+    )
+    print(
+        f"公众号简介:   {result.mp_description[:80]}"
+        if result.mp_description
+        else "公众号简介:   (empty)"
+    )
+    print(f"文章ID:       {result.article_id}")
+    print(f"群发消息ID:   {result.article_msg_id}")
+    print(f"文章idx:      {result.article_idx}")
+    print(f"文章签名:     {result.article_sn}")
+    print(f"文章标题:     {result.article_title}")
+    print(
+        f"封面图:       {result.article_cover_image[:80]}..."
+        if result.article_cover_image
+        else "封面图:       (empty)"
+    )
+    print(
+        f"文章摘要:     {result.article_description[:80]}"
+        if result.article_description
+        else "文章摘要:     (empty)"
+    )
+    print(f"发布时间:     {result.article_publish_time}")
+    print(f"Markdown长度: {len(result.article_markdown)}")
+    print(f"Markdown预览: {result.article_markdown[:200]}")
+    print(f"有效性:       {result.is_valid}")
+    print(f"{'='*60}")
+
+    assert result.mp_id > 0, "mp_id should be positive"
+    assert result.mp_name, "mp_name should not be empty"
+    assert result.article_id, "article_id should not be empty"
+    assert result.article_msg_id > 0, "article_msg_id should be positive"
+    assert result.article_idx > 0, "article_idx should be positive"
+    assert result.article_sn, "article_sn should not be empty"
+    assert result.article_title, "article_title should not be empty"
+    assert result.article_markdown, "article_markdown should not be empty"
+    assert result.article_publish_time > 0, "article_publish_time should be positive"
+    assert result.is_valid
+
+
+def test_fetch_all(url: str) -> None:
+    """抓取指定 URL 并打印所有采集到的参数。
+
+    用法: pytest tests/test_parser.py::test_fetch_all -s --url <URL>
+    """
+    result = parse(url)
+    print(f"\n{'='*60}")
+    print(f"URL:          {url}")
+    print(f"公众号ID(B64):{result.mp_id_b64}")
+    print(f"公众号ID:     {result.mp_id}")
+    print(f"公众号名称:   {result.mp_name}")
+    print(f"公众号别名:   {result.mp_alias}")
+    print(f"公众号图片:   {result.mp_image}")
+    print(f"公众号简介:   {result.mp_description}")
+    print(f"文章ID:       {result.article_id}")
+    print(f"群发消息ID:   {result.article_msg_id}")
+    print(f"文章idx:      {result.article_idx}")
+    print(f"文章签名:     {result.article_sn}")
+    print(f"文章标题:     {result.article_title}")
+    print(f"封面图:       {result.article_cover_image}")
+    print(f"文章摘要:     {result.article_description}")
+    print(f"发布时间:     {result.article_publish_time}")
+    print(f"图片列表({len(result.images)}张):")
+    for i, img in enumerate(result.images, 1):
+        print(f"  {i}. {img}")
+    print(f"Markdown内容:\n{result.article_markdown}")
+    print(f"有效性:       {result.is_valid}")
+    print(f"{'='*60}")
+
+
+def test_fetch_markdown(url: str) -> None:
+    """抓取指定 URL 并只打印 Markdown 内容。
+
+    用法: pytest tests/test_parser.py::test_fetch_markdown -s --url <URL>
+    """
+    result = parse(url)
+    print(f"\n{result.article_markdown}")