pyPaperFlow 0.6.2__tar.gz → 0.6.3__tar.gz

{pypaperflow-0.6.2 → pypaperflow-0.6.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: pyPaperFlow
-Version: 0.6.2
+Version: 0.6.3
 Summary: Automated paper fetching and analysis platform.
 Project-URL: Homepage, https://github.com/MaybeBio/pyPaperFlow
 Project-URL: Issues, https://github.com/MaybeBio/pyPaperFlow/issues
@@ -1453,6 +1453,8 @@ In theory, all DOI‑driven literature workflows can be standardised following t
 
 > Modules dedicated to the aforementioned preprint platforms are still under development and refinement. Preprint‑related subcommands are provided for testing purposes only. For detailed test cases, refer to [Cases](./docs/Cases.md)
 
+> **arXiv search backend note.** The `arxiv-search` / `arxiv-fetch` `native` backend talks to `export.arxiv.org` over `requests` (urllib3), not `httpx`: arXiv's Fastly CDN rejects `httpx`'s TLS fingerprint with HTTP 406 on boolean queries (fielded `OR` / quoted phrases), while `requests` and `curl` route through Google's edge and return 200. See [Cases](./docs/Cases.md) for details.
+
 #### Preprint full-text fetch (Python API)
 
 For preprints without an open-access PDF, each fetcher exposes a `fetch_full_text()` method that returns clean section-headed text without any PDF parsing:

{pypaperflow-0.6.2 → pypaperflow-0.6.3}/README.md

@@ -1424,6 +1424,8 @@ In theory, all DOI‑driven literature workflows can be standardised following t
 
 > Modules dedicated to the aforementioned preprint platforms are still under development and refinement. Preprint‑related subcommands are provided for testing purposes only. For detailed test cases, refer to [Cases](./docs/Cases.md)
 
+> **arXiv search backend note.** The `arxiv-search` / `arxiv-fetch` `native` backend talks to `export.arxiv.org` over `requests` (urllib3), not `httpx`: arXiv's Fastly CDN rejects `httpx`'s TLS fingerprint with HTTP 406 on boolean queries (fielded `OR` / quoted phrases), while `requests` and `curl` route through Google's edge and return 200. See [Cases](./docs/Cases.md) for details.
+
 #### Preprint full-text fetch (Python API)
 
 For preprints without an open-access PDF, each fetcher exposes a `fetch_full_text()` method that returns clean section-headed text without any PDF parsing:

{pypaperflow-0.6.2 → pypaperflow-0.6.3}/README_zh.md

@@ -1606,10 +1606,12 @@ paperflow arxiv-fetch "deep learning for biology" --max-results 10 --download-pd
 常用参数：
 
 - `--start-date` / `--end-date`：按 `YYYY-MM-DD` 格式限制日期范围。
-- `--backend`：可选 `native`（内置的 httpx 方案）或 `paperscraper`（安装了第三方包时可用, ⚠️ 暂时未测试paperscraper）。
+- `--backend`：可选 `native`（内置的 requests 方案）或 `paperscraper`（安装了第三方包时可用, ⚠️ 暂时未测试paperscraper）。
 - `--output-dir`：把 ID 列表或抓取结果保存到其他目录。
 - `--no-download-pdf`：只保存元数据，不下载 PDF。
 
+> ⚠️ **为什么 `native` 用 `requests` 而非 `httpx`**：arXiv 的 `export.arxiv.org` API 在 Fastly CDN 后面，会对 `httpx` 的 TLS 指纹在**布尔查询**（任何带字段的 `OR` 或引号短语——也就是多词检索时 query builder 生成的形式）上返回 **HTTP 406**。`requests`（urllib3）和 `curl` 会经 Google 边缘节点返回 200。单个裸词恰好 `httpx` 也能通过，但真实布尔查询需要非 `httpx` 客户端。
+
 日期过滤示例：
 
 ```bash

pypaperflow-0.6.3/src/pyPaperFlow/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.6.3"

{pypaperflow-0.6.2 → pypaperflow-0.6.3}/src/pyPaperFlow/preprint/arxiv_fetcher.py

@@ -9,6 +9,7 @@ from pathlib import Path
 from typing import Any, Dict, Iterable, List, Optional
 
 import httpx
+import requests
 from bs4 import BeautifulSoup
 
 from .source_models import SourcePaper
@@ -372,9 +373,12 @@ class ArxivFetcher:
         last_error: Optional[Exception] = None
 
         for attempt in range(self.max_retries):
-            response: Optional[httpx.Response] = None
+            response = None
             try:
-                response = self._get_http_client().get(
+                # arXiv's Fastly CDN returns 406 for httpx's TLS fingerprint on
+                # boolean queries (OR / quoted phrases). requests (urllib3) and
+                # curl pass through to Google's edge, so use requests here.
+                response = requests.get(
                     ARXIV_API_URL,
                     params=params,
                     headers=self.headers,
@@ -395,7 +399,7 @@ class ArxivFetcher:
                         sleep_before_retry(response, attempt)
                         continue
                     break
-            except (httpx.HTTPStatusError, httpx.TimeoutException, httpx.TransportError) as exc:
+            except requests.exceptions.RequestException as exc:
                 last_error = exc
                 if attempt + 1 < self.max_retries:
                     sleep_before_retry(response, attempt)

pypaperflow-0.6.2/src/pyPaperFlow/__init__.py

@@ -1 +0,0 @@
-__version__ = "0.6.2"