dhole-mcp 14.0__tar.gz

dhole_mcp-14.0/.gitattributes

@@ -0,0 +1,21 @@
+# Default to LF in the repo; git converts to the platform native on checkout.
+* text=auto eol=lf
+
+# These must stay CRLF on Windows (pip/console launchers parse them).
+*.bat text eol=crlf
+*.cmd text eol=crlf
+
+# Binary assets — never normalize.
+*.png binary
+*.jpg binary
+*.jpeg binary
+*.gif binary
+*.webp binary
+*.ico binary
+*.svg text
+*.pdf binary
+
+# Vendored / generated code shouldn't skew the language bar.
+NOTICE.ddgs.txt linguist-vendored
+src/dhole_mcp/search_metasearch.py linguist-vendored=true
+docs/** linguist-generated=true

dhole_mcp-14.0/.github/workflows/lint.yml

@@ -0,0 +1,29 @@
+name: Lint
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+
+concurrency:
+  group: lint-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  ruff:
+    runs-on: ubuntu-latest
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python
+        uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+
+      - name: Install ruff
+        run: python -m pip install --upgrade pip "ruff>=0.6"
+
+      - name: Ruff check
+        # 规则集与豁免项在 pyproject.toml 的 [tool.ruff] 下配置
+        run: ruff check src tests

dhole_mcp-14.0/.github/workflows/test.yml

@@ -0,0 +1,41 @@
+name: Tests
+
+on:
+  push:
+    branches: [master]
+  pull_request:
+
+concurrency:
+  group: tests-${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  test:
+    runs-on: ${{ matrix.os }}
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest]
+        python-version: ["3.11", "3.12", "3.13", "3.14"]
+        include:
+          # Windows 只跑 3.11：历史上 3.12/3.13 的 Windows job 会挂起
+          # （见 git log "ci: drop Windows 3.12/3.13"），覆盖率由 Ubuntu 保证。
+          - os: windows-latest
+            python-version: "3.11"
+
+    steps:
+      - uses: actions/checkout@v4
+
+      - name: Set up Python ${{ matrix.python-version }}
+        uses: actions/setup-python@v5
+        with:
+          python-version: ${{ matrix.python-version }}
+
+      - name: Install
+        run: |
+          python -m pip install --upgrade pip
+          python -m pip install -e ".[dev]"
+
+      - name: Run tests
+        # e2e 测试会真实拉起 dhole.exe 子进程，默认跳过（见 pyproject 的 addopts）
+        run: python -m pytest -q

dhole_mcp-14.0/.gitignore

@@ -0,0 +1,76 @@
+# Python
+__pycache__/
+*.py[cod]
+*.so
+*.egg-info/
+dist/
+build/
+*.egg
+
+# Virtual environment
+.venv/
+venv/
+env/
+
+# IDE
+.vscode/
+.idea/
+*.swp
+*.swo
+
+# OS
+.DS_Store
+Thumbs.db
+
+# Environment/Secrets
+.env
+.env.*
+*.pem
+*.key
+credentials.json
+
+# Project caches (runtime)
+.dhole_mcp_cache/
+.dhole_cache/
+*.log
+
+# Test artifacts
+.pytest_cache/
+htmlcov/
+.coverage
+coverage.xml
+
+# --- personal / agent scratch (never commit) ---
+# These are local-only planning, status, and AI-agent working files. They are
+# not for the public audience and stay out of the repo.
+STATUS.md
+PROJECT.md
+PLAN.md
+TODO.md
+ROADMAP-personal.md
+research/
+notes/
+scratch/
+.pi/
+.claude/
+.cursor/
+keys.md
+dhole-mcp-bug-report.md
+
+# Local-only temp helper scripts (one-off fixes, release-note drafts).
+/.fix_*.py
+/.v7*.py
+/.serl*.py
+/.engine_surgery.py
+/.drop_*.py
+/.soften_*.py
+/.release_notes*.md
+V7-SEARCH-PLAN.md
+V10_PLAN.md
+docs/dhole-demo.mp4
+pi-extension/.npmrc
+node_modules/
+package-lock.json
+
+# Lint cache (ruff writes .ruff_cache/.gitignore, this is belt-and-braces)
+.ruff_cache/

dhole_mcp-14.0/CHANGELOG.md

@@ -0,0 +1,139 @@
+# 更新日志
+
+本文件在 11.1.8 版重新建立。更早的发布**未**在此回溯补记——历史请查看
+`git log` 与 GitHub releases 页面。
+
+格式遵循 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。
+
+> **版本说明。** 自 13.14 起本仓库为个人衍生作品，不再跟随上游项目的发布，
+> 版本号是自己的，与上游版本不可比。`src/dhole_mcp/__init__.py` 中的
+> `__version__` 是版本的唯一权威来源。
+
+## [14.0] - 2026-09-20
+
+### 重大变更（破坏性）
+- **项目更名：`hound-mcp` → `dhole-mcp`。** 原名与 PyPI 上的其他项目撞名，
+  自更新也因此长期关闭。改名覆盖：
+  - Python 包目录 `src/hound_mcp/` → `src/dhole_mcp/`，CLI 命令 `hound` → `dhole`
+  - 环境变量前缀 `HOUND_*` → `DHOLE_*`（全部读取点）
+  - 数据目录 `~/.hound` → `~/.dhole`，缓存 `~/.hound_mcp_cache` →
+    `~/.dhole_mcp_cache`（旧缓存不迁移，TTL 到期自然重建）
+  - 日志名 `hound-mcp.*` → `dhole-mcp.*`，仓库地址同步更名
+    （`github.com/ouli-1242/dhole-mcp`；GitHub 仓库本身需在 Settings 里改同名）
+- **自更新模块注释按新语义改写。** `_DIST_NAME` 指向本仓库自己的分发名；
+  `dhole-mcp` 发布到 PyPI 后用 `DHOLE_UPDATE_PACKAGE` 打开自更新。
+
+### 兼容措施（迁移期零破坏）
+- 旧环境变量 `HOUND_*` 在包导入时自动迁移为 `DHOLE_*`（同名 `DHOLE_*`
+  优先），既有 client 配置里的 env 无需改动。
+- 保留 `hound` CLI 命令别名与 `hound_mcp` 兼容模块（导入时发
+  DeprecationWarning），`python -m hound_mcp` 等旧启动方式继续可用；
+  确认全部 client 配置已迁移后可删除这两处兼容层。
+
+## [13.16] - 2026-09-20
+
+### 修复
+- **`links.py` 的 `_norm_host` 会损毁 w 开头的域名。** 它用 `lstrip("www.")` 剥离
+  `www.` 前缀，但 `lstrip` 是按字符集剥离而非剥字符串：`wikipedia.org` 变成
+  `ikipedia.org`、`web.example.com` 变成 `eb.example.com`、`www.wikipedia.org`
+  变成 `ikipedia.org`。这类域名的所有比较与 `.{domain}` 后缀匹配（外链分类、
+  primary source 识别）因此全部失效。现改为 `startswith("www.")` + 切片，
+  userinfo/端口交给 `urlparse().hostname` 处理。
+- **`fetcher.py` 的 `ElementWrapper.text_content()` 只返回元素自身首段文本**
+  （`.text`），嵌套子元素的文字被静默丢弃。现改为拼接 `itertext()`，与 lxml
+  原生 `.text_content()` 语义一致。（既有的 3 个测试恰好都用叶子元素，两种
+  实现都能通过——现已补嵌套内容回归测试钉住正确行为。）
+- **`smart_search(fetch_content=true)` 静默吞掉单页抓取错误。** 某条 top 结果
+  抓取失败时直接从 `fetched_pages` 中消失。现在失败也占位，带
+  `content_ok=false` 与脱敏后的 `error` 字段，Agent 能看到缺口及其原因。
+
+### 变更
+- **清除"刻舟求剑"式测试（-13 +1）。** 删除把当前实现快照进断言的变更
+  探测器：4 个签名/属性存在性测试（`inspect.signature` 查参数、
+  `hasattr` 查字段——只在改名时报假警，抓不到行为 bug）、2 个搜索引擎池
+  数量/成员冻结（`DEFAULT_ENGINES` 是数据不是逻辑，增删引擎属正常演进）、
+  6 个隐身层冻结断言（内存调优参数、单个 stealth flag 字符串、指纹档案
+  数量、语言/设备内存枚举）。保留并改造了真契约：关系型不变量（自有参数
+  永不含有害自动化参数）、档案自洽（MacIntel⇒Apple GPU、GPU 厂商多样性、
+  plugins 非空）、跨引擎共识正确性（DDG/Yahoo 必须同属 Bing 索引家族）。
+  test_cli 的 `hasattr` 结构检查替换为真行为测试：子进程中用 import hook
+  屏蔽全部重依赖后导入 `dhole_mcp.cli` 必须成功——这才是"坏安装下自愈
+  入口仍可用"的测法。
+- **`get()`/`bulk_get()` 的 `auth` / `proxy_auth` 现在真正生效**（闭合 13.x
+  记录的已知缺口）：`auth` 转成 Basic `Authorization` 头（绝不覆盖调用方
+  显式设置的头）；`proxy_auth` 经 URL 编码后嵌入代理地址交给 HTTP 层；显式
+  `proxy_auth` 优先于代理串里已嵌入的凭据。dict 代理
+  （`{server, username, password}`）现在也能到达 HTTP 层——此前所有
+  `proxy if isinstance(proxy, str) else None` 写法都把它静默忽略，
+  `smart_fetch` 带 dict 代理时只有浏览器层走代理、HTTP 层直连。
+- **依赖刷新，约束两端均验证通过：** mcp 2.0 → 2.2、pydantic 2.13.5、
+  trafilatura 2.2、httpx 0.27 → 0.28（dhole 早已使用新的 `proxy=` API）、
+  anyio/starlette/uvicorn/lxml/beautifulsoup4/cssselect/h2/markdownify 升至
+  最新，primp 1.3 → **2.0**（主版本）。primp 2.0 移除了响应的 `.reason`
+  属性——`fetcher.py` 已有 `hasattr` 守护；完整测试套件 + 真实冒烟（构造
+  参数、`headers_update`、`follow_redirects=False` 逐跳重定向解析、cookies）
+  在 1.3.1 与 2.0.1 上均通过。patchright/playwright 保持 1.61 以匹配本地
+  已安装的浏览器。
+- **工具可发现性重写（为什么模型此前很少主动使用 dhole）。** 模型选工具
+  取决于描述的第一句话，而 dhole 的描述原本是功能清单。8 个工具全部改为：
+  先说任务触发场景 + 显式声明"用本工具而非内置 WebFetch / web search" +
+  理由（反爬绕过、JS 渲染、PDF/OCR、无密钥引擎），功能/信号细节后置。
+  connect-time `instructions` 同样重写：祈使句路由表，"prefer dhole over
+  built-ins" 放首行。顺带修正两处过时描述：默认引擎池补上 `bing`（代码
+  默认早已包含它），删除已失效的 "'bing' maps to yahoo" 映射说明。
+- README 环境变量表现在列出代码实际读取的全部变量
+  （`DHOLE_SEARCH_DEADLINE`、`DHOLE_BRIGHTDATA_*`、`DHOLE_SSRF_DNS_RECHECK`、
+  `DHOLE_UPDATE_*`）。
+
+### 新增
+- `include_media` 现在也能抓取懒加载图片：当 `src` 为空或 `data:` 占位图时，
+  从同一个 `<img>` 标签读取 `data-src`（`src` 是真实 URL 时仍优先）。
+
+## [13.14] - 2026-09-10
+
+### 修复
+- **日志凭据泄漏。** `fetcher.py` 重试时把原始异常文本写进 `logger.warning`，
+  含 `user:pass@` 的代理 URL 会落进日志。现统一走 `security.redact_api_key()`
+  （代码库其他地方已在用）。
+- **`RuntimeWarning: coroutine 'ProxyPool.health_check' was never awaited`。**
+  `_kick_health_check()` 在调用 `asyncio.create_task()` 之前就求值了
+  `pool.health_check()`，没有运行中的事件循环时协程被创建后即丢弃。现在
+  先检查事件循环。
+- **过期的类型标注。** `server.py` 两个辅助函数仍标注已不存在的
+  `_ScraplingResponse`；`ocr.py` 和 `server.py` 引用 `PdfResult` /
+  `CrawlResponseModel` 但未导入。标注是惰性的所以没有崩溃，但类型是错的。
+- 清理死赋值（`browser.py`、`pdf_extractor.py`、`search_engines.py`、
+  `server.py`），重命名歧义变量 `l`。
+
+### 变更
+- **版本单一来源。** `pyproject.toml` 不再硬编码 `99.0.0`，改经
+  `[tool.hatch.version]` 读取 `src/dhole_mcp/__init__.py` 的 `__version__`。
+  此前 `pip show` 报 99.0.0、CLI 报 11.1.8、package.json 写 11.1.6，三处
+  不一致。
+- 补声明"被导入但未列出"的依赖：`beautifulsoup4`（`search_engines.py`
+  导入——缺失时该代码路径静默降级为空结果），以及仅经
+  `httpx[http2,socks]` 传递解析的 `h2` 和 `httpcore`。
+- **自更新不再指向上游包。** 本仓库是个人衍生作品，但 `dhole -u` 原先执行
+  `pip install dhole-mcp==<latest>`——那是*上游*发行版，会把上游代码装进来
+  覆盖本 fork。自更新现默认关闭：`dhole -v` 显示 "self-update off"，
+  `dhole -u` 拒绝执行（含显式版本号）并指向
+  `git pull && python -m pip install -e .`。发布自己的发行版后可用
+  `DHOLE_UPDATE_PACKAGE=<发行版名>` 加可选 `DHOLE_UPDATE_INDEX_URL`
+  重新启用。生成的修复脚本与 Windows 助手跟随同一发行版名。
+- 自 13.14 起版本号是本 fork 自己的，与上游版本不可比。
+- 移除代码注释与测试分组标题里的 `(upstream v12.0.0)` 式溯源标记。
+- 删除 `package.json` 里悬空的 `pi.extensions` 条目：它指向的
+  `pi-extension/extensions/dhole.ts` 不在本仓库中。
+
+### 新增
+- `.github/workflows/test.yml` 与 `.github/workflows/lint.yml`——仓库此前
+  没有 CI，测试套件和 lint 无强制。
+- `CONTRIBUTING.md` 与本 changelog。
+- `[tool.ruff]` 配置及 `dev` extras 中的 `ruff`。
+
+### 已知缺口
+- 抓取工具的 `auth` / `proxy_auth` 只校验不生效：`HTTPSession` / `http_get`
+  不接受它们。当时保留原行为；见 `src/dhole_mcp/server.py` 中 `bulk_get`
+  的注释。（已于 13.15 修复。）
+- `LICENSE` 与 `NOTICE.ddgs.txt` 保留原始版权声明。停止追踪上游*版本*并
+  不改变这一点：除非重写代码，MIT 要求保留这些声明。

dhole_mcp-14.0/CONTRIBUTING.md

@@ -0,0 +1,80 @@
+# Contributing
+
+Thanks for looking at Dhole. This is a small, opinionated MCP server; the notes
+below are what you need to get a green local run.
+
+## Setup
+
+```bash
+git clone https://github.com/ouli-1242/dhole-mcp.git
+cd dhole-mcp
+python -m pip install -e ".[dev]"
+```
+
+Optional extras (browser engine, PDF, OCR, local file parsing) are in `[all]`:
+
+```bash
+python -m pip install -e ".[dev,all]"
+playwright install chromium          # required for the anti-bot browser path
+```
+
+Without `[all]`, dhole degrades to HTTP-only mode instead of failing — that
+degradation is a supported configuration and is covered by tests.
+
+## Tests
+
+```bash
+pytest                    # default run; e2e tests are deselected
+pytest -m e2e             # spawns a real dhole.exe subprocess
+pytest tests/test_search.py -q
+```
+
+Two things worth knowing before you debug a "test doesn't see my change":
+
+1. **`pythonpath = ["src"]` in `pyproject.toml` is load-bearing.** The dev venv
+   often has a *built* wheel installed rather than an editable install. Without
+   that setting, pytest silently imports the stale installed copy and your edits
+   to `src/` go untested. This has bitten the project more than once.
+2. `addopts = ["-m", "not e2e"]` deselects the e2e tests by default. They are
+   slower and need a built executable.
+
+## Lint
+
+```bash
+ruff check src tests      # config lives in pyproject.toml [tool.ruff]
+```
+
+CI runs `pytest` (Ubuntu 3.11–3.14, Windows 3.11) and `ruff check` on every push
+to `master` and every pull request.
+
+## Layout
+
+```
+src/dhole_mcp/
+  server.py              MCP tool schemas + dispatch + orchestration
+  fetcher.py             primp-based HTTP session and Response
+  browser.py             patchright stealth browser
+  search*.py             keyless metasearch backends, proxy pool, reranker
+  crawl.py, sitemap.py   same-domain crawling
+  pdf_extractor.py, ocr.py, parse.py, structured.py, metadata.py
+  extractor.py, trafilatura_extractor.py    HTML -> markdown paths
+  security.py            URL validation, SSRF checks, credential redaction
+  envelope.py, errors.py, cache.py          cross-cutting helpers
+tests/                   unit + integration tests (no live network by default)
+```
+
+## Conventions
+
+- Commit messages follow `type(scope): summary` — e.g. `fix(search): ...`,
+  `feat(server): ...`, `docs: ...`, `chore: ...`.
+- Never commit credentials. API keys are read from environment variables only;
+  see the note in `src/dhole_mcp/search_metasearch.py`. Error text and logs must
+  go through `security.redact_api_key()` before being surfaced.
+- Every behaviour change ships with a test that fails without it.
+- Keep tool definitions token-lean — the tool descriptions are part of the
+  prompt budget for every agent session.
+
+## Reporting bugs
+
+Include the `dhole -v` output, the exact tool call, and whether the browser
+extras were installed. `DHOLE_DEBUG=1` (or `-v`) increases log verbosity.

dhole_mcp-14.0/LICENSE

@@ -0,0 +1,22 @@
+MIT License
+
+Copyright (c) 2026 Bishesh Bhandari
+Modifications Copyright (c) 2026 ouli-1242
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

dhole_mcp-14.0/NOTICE.ddgs.txt

@@ -0,0 +1,38 @@
+NOTICE — vendored ddgs metasearch code
+
+The file `src/dhole_mcp/search_metasearch.py` in this project is derived
+from ddgs (https://github.com/deedy5/ddgs), a metasearch library aggregating
+diverse web search services. ddgs is distributed under the MIT License
+(see below). It has been vendored, stripped to text search only, and adapted
+(async-native parallel aggregation with early-return-on-quorum, dhole-specific
+tuning) for use inside dhole-mcp.
+
+Per the MIT terms, the original copyright notice + permission notice are
+reproduced here and the ddgs LICENSE is preserved alongside this notice.
+
+ddgs dependencies used transitively at runtime (primp, httpx[http2]/h2,
+fake-useragent, lxml) remain separate upstream packages under their own
+licenses (primp/httpx/fake-useragent/lxml are MIT/BSD-compatible).
+
+------------------------------------------------------------------------------
+MIT License
+
+Copyright (c) 2022 deedy5
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

dhole_mcp-14.0/PKG-INFO

@@ -0,0 +1,153 @@
+Metadata-Version: 2.5
+Name: dhole-mcp
+Version: 14.0
+Summary: Web research for AI agents. Fetch any page with anti-bot bypass plus web search. $0 forever. No scrapling dependency: dhole uses primp + patchright directly. Graceful degradation when browser deps are unavailable (HTTP-only mode on Termux/Android). Token-optimized tool definitions, polished cross-platform CLI, brick-proof self-update with dhole doctor + rollback.
+Project-URL: Repository, https://github.com/ouli-1242/dhole-mcp
+Project-URL: Issues, https://github.com/ouli-1242/dhole-mcp/issues
+Author-email: ouli-1242 <ouli1242@gmail.com>
+License-File: LICENSE
+License-File: NOTICE.ddgs.txt
+Keywords: ai-agent,anti-bot,cloudflare-bypass,content-extraction,mcp,mcp-server,web-scraping
+Classifier: Development Status :: 4 - Beta
+Classifier: Environment :: Console
+Classifier: Framework :: AsyncIO
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: Internet :: WWW/HTTP :: Dynamic Content
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.11
+Requires-Dist: aiosqlite>=0.20.0
+Requires-Dist: beautifulsoup4>=4.12
+Requires-Dist: cssselect>=1.2
+Requires-Dist: fake-useragent>=2.0
+Requires-Dist: h2>=4.0
+Requires-Dist: httpcore<2.0,>=1.0
+Requires-Dist: httpx[http2,socks]>=0.27
+Requires-Dist: lxml>=5.0
+Requires-Dist: markdownify>=1.2.0
+Requires-Dist: mcp<3,>=2.0
+Requires-Dist: primp>=1.0
+Requires-Dist: pydantic>=2.0
+Requires-Dist: trafilatura>=2.0.0
+Provides-Extra: all
+Requires-Dist: browserforge>=1.2.4; extra == 'all'
+Requires-Dist: onnxruntime>=1.16; extra == 'all'
+Requires-Dist: openpyxl>=3.1; extra == 'all'
+Requires-Dist: patchright>=1.50; extra == 'all'
+Requires-Dist: pdfplumber>=0.11.0; extra == 'all'
+Requires-Dist: playwright>=1.50; extra == 'all'
+Requires-Dist: pypdfium2>=4.30; extra == 'all'
+Requires-Dist: python-docx>=1.0; extra == 'all'
+Requires-Dist: rapidocr>=3.0; extra == 'all'
+Requires-Dist: requests>=2.31; extra == 'all'
+Requires-Dist: tokenizers>=0.20; extra == 'all'
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.24; extra == 'dev'
+Requires-Dist: pytest-mock>=3.14; extra == 'dev'
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: ruff>=0.6; extra == 'dev'
+Description-Content-Type: text/markdown
+
+<div align="center">
+
+# Dhole
+
+**让 AI 代理访问互联网：抓取 · 爬取 · 搜索，内置反爬，$0 无密钥。**
+
+[MIT](LICENSE) · Python 3.11+
+
+</div>
+
+Dhole 是一个 [MCP](https://modelcontextprotocol.io) 服务器，为 AI 代理提供网页抓取、整站爬取和无密钥网页搜索。HTTP 被拦截时自动升级到反检测浏览器，可读取 PDF（含扫描件 OCR），全部本地运行、无需任何 API key。
+
+## 功能特性
+
+- **智能抓取**：HTTP 优先（~1 秒），被拦截或遇到 JS 空壳时自动升级 Patchright 隐身浏览器并求解 Cloudflare 验证；全部失败时回退 Wayback Machine 快照
+- **网页搜索**：多引擎并行（bing / duckduckgo / brave / yahoo / yandex 等），本地 ONNX 神经重排序，跨引擎共识排名；`fetch_content=true` 直接抓回 top3 全文
+- **整站爬取**：同域最佳优先遍历，内容自适应提取，sitemap 模式，页数 / 深度 / token 预算控制
+- **PDF + OCR**：结构化 Markdown 输出，扫描件与 CID 损坏自动 OCR
+- **结构化提取**：CSS 选择器 / JSON-LD / 自动模式 schema，支持多 URL 批量并行
+- **Agent 友好**：每次调用返回 `next_action` 建议与来源分类；超长内容自动分页；SQLite 缓存加速重复抓取
+- **优雅降级**：浏览器依赖缺失时自动切换纯 HTTP 模式（Termux / 精简容器可用）
+
+## 安装
+
+未发布到 PyPI，从源码安装。需要 Python 3.11+。
+
+```bash
+git clone https://github.com/ouli-1242/dhole-mcp.git
+cd dhole-mcp
+pip install .[all]              # 完整版；精简版（纯 HTTP + 搜索）用 pip install .
+playwright install chromium     # 反检测浏览器引擎（~150MB，完整版需要）
+```
+
+卸载：
+
+```bash
+pip uninstall dhole-mcp
+rm -rf ~/.dhole                 # 运行时缓存（Windows: rd /s /q %USERPROFILE%\.dhole）
+```
+
+## 使用
+
+在 MCP 客户端（Claude Code / Cursor / OpenCode 等）配置中添加：
+
+```json
+{ "mcpServers": { "dhole": { "command": "dhole" } } }
+```
+
+无需参数、无需密钥、无需环境变量。CLI 自带诊断命令：
+
+```bash
+dhole -v    # 查看版本 + 更新状态
+dhole -u    # 自更新（本 fork 默认关闭）
+```
+
+## 工具
+
+| 工具 | 功能 |
+|------|------|
+| `smart_fetch` | 抓取任意 URL：自动反爬升级、PDF/OCR、批量、聚焦提取、页面交互、结构化提取 |
+| `smart_search` | 无密钥网页搜索：多引擎并行、神经重排序、可同时抓回全文 |
+| `smart_crawl` | 同域最佳优先爬取，支持 sitemap 模式与关键词过滤 |
+| `screenshot` | 页面截图（多模态代理专用） |
+| `parse` | 本地文件解析（.html/.docx/.xlsx/.csv → Markdown） |
+| `feed_fetch` | 批量抓取 RSS/Atom feed 最新条目 |
+| `resolve_url` | 解析 URL 最终地址（跟随重定向，不下载页面体） |
+| `cache_clear` | 清除抓取缓存 |
+
+## 配置
+
+所有环境变量均可选，默认零配置可用：
+
+| 变量 | 用途 |
+|------|------|
+| `DHOLE_SEARCH_PROXY` | 搜索引擎代理，逗号分隔可轮换（也自动读取 `HTTPS_PROXY` 等） |
+| `DHOLE_BROWSER_IDLE_TIMEOUT` | 浏览器空闲关闭秒数（默认 300，`0` 永不关闭） |
+| `DHOLE_SEARCH_DEADLINE` | 单次搜索整体截止秒数（默认 16） |
+| `DHOLE_BRIGHTDATA_API_KEY` | 启用 Bright Data SERP 付费后端（可选） |
+| `DHOLE_SSRF_DNS_RECHECK` | 设 `1` 开启 DNS 解析内网复查（默认关闭） |
+| `DHOLE_UPDATE_PACKAGE` | 自更新目标发行名（发布自己的发行版后设置以启用） |
+
+## 已知限制
+
+- 无法绕过 DataDome / Akamai / 交互式 Turnstile；需要登录的网站不在设计范围内
+- duckduckgo / brave / yahoo 需要 VPN 可达（bing / yandex 国内直连）
+- 搜索引擎限速时熔断器自动冷却 60 秒，重度使用建议配置代理
+- YouTube 仅能获取少量文本
+
+## 贡献
+
+欢迎 issue 与 PR，规范见 [CONTRIBUTING.md](CONTRIBUTING.md)。
+
+## 致谢
+
+本项目是 [dondai1234/master-fetch](https://github.com/dondai1234/master-fetch) 的二创（衍生作品），上游以 MIT 协议发布，原始版权文本已完整保留于 [LICENSE](LICENSE)。`src/dhole_mcp/search_metasearch.py` 派生自 [ddgs](https://github.com/deedy5/ddgs)，声明见 [NOTICE.ddgs.txt](NOTICE.ddgs.txt)。
+
+## 许可证
+
+[MIT](LICENSE)