zleap-parser 0.1.0__tar.gz

zleap_parser-0.1.0/.gitignore

@@ -0,0 +1,63 @@
+# ===== Python =====
+__pycache__/
+*.py[cod]
+*$py.class
+*.so
+*.egg-info/
+.eggs/
+build/
+dist/
+develop-eggs/
+downloads/
+eggs/
+parts/
+sdist/
+var/
+wheels/
+.installed.cfg
+*.egg
+MANIFEST
+
+# ===== 虚拟环境 =====
+.venv/
+venv/
+env/
+ENV/
+
+# ===== 测试 / 静态检查 =====
+.pytest_cache/
+.mypy_cache/
+.ruff_cache/
+.tox/
+nox/
+.coverage
+.coverage.*
+htmlcov/
+coverage.xml
+*.cover
+
+# ===== 缓存与产物 =====
+*.octx
+.octx-workspace/
+tasks/
+
+# ===== 环境变量 / 密钥 =====
+.env
+.env.*
+!.env.example
+
+# ===== 本地配置文件（含密钥，不入库）=====
+config.local.toml
+*.local.toml
+
+# ===== 企业微信会话内容存档 C SDK（随包分发，需入库）=====
+!src/zleap_parser/connector/sources/wxwork_sdk/
+!src/zleap_parser/connector/sources/wxwork_sdk/*.so
+
+# ===== IDE / 编辑器 / 系统 =====
+.idea/
+.vscode/
+*.swp
+*.swo
+.DS_Store
+Thumbs.db

zleap_parser-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 zleap
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

zleap_parser-0.1.0/LICENSES/Apache-2.0.txt

@@ -0,0 +1,200 @@
+
+                                 Apache License
+                           Version 2.0, January 2004
+                        http://www.apache.org/licenses/
+
+   TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
+
+   1. Definitions.
+
+      "License" shall mean the terms and conditions for use, reproduction,
+      and distribution as defined by Sections 1 through 9 of this document.
+
+      "Licensor" shall mean the copyright owner or entity authorized by
+      the copyright owner that is granting the License.
+
+      "Legal Entity" shall mean the union of the acting entity and all
+      other entities that control, are controlled by, or are under common
+      control with that entity. For the purposes of this definition,
+      "control" means (i) the power, direct or indirect, to cause the
+      direction or management of such entity, whether by contract or
+      otherwise, or (ii) ownership of fifty percent (50%) or more of the
+      outstanding shares, or (iii) beneficial ownership of such entity.
+
+      "You" (or "Your") shall mean an individual or Legal Entity
+      exercising permissions granted by this License.
+
+      "Source" form shall mean the preferred form for making modifications,
+      including but not limited to software source code, documentation
+      source, and configuration files.
+
+      "Object" form shall mean any form resulting from mechanical
+      transformation or translation of a Source form, including but
+      not limited to compiled object code, generated documentation,
+      and conversions to other media types.
+
+      "Work" shall mean the work of authorship, whether in Source or
+      Object form, made available under the License, as indicated by a
+      copyright notice that is included in or attached to the work
+      (an example is provided in the Appendix below).
+
+      "Derivative Works" shall mean any work, whether in Source or Object
+      form, that is based on (or derived from) the Work and for which the
+      editorial revisions, annotations, elaborations, or other modifications
+      represent, as a whole, an original work of authorship. For the purposes
+      of this License, Derivative Works shall not include works that remain
+      separable from, or merely link (or bind by name) to the interfaces of,
+      the Work and Derivative Works thereof.
+
+      "Contribution" shall mean any work of authorship, including
+      the original version of the Work and any modifications or additions
+      to that Work or Derivative Works thereof, that is intentionally
+      submitted to Licensor for inclusion in the Work by the copyright owner
+      or by an individual or Legal Entity authorized to submit on behalf of
+      the copyright owner. For the purposes of this definition, "submitted"
+      means any form of electronic, verbal, or written communication sent
+      to the Licensor or its representatives, including but not limited to
+      communication on electronic mailing lists, source code control systems,
+      and issue tracking systems that are managed by, or on behalf of, the
+      Licensor for the purpose of discussing and improving the Work, but
+      excluding communication that is conspicuously marked or otherwise
+      designated in writing by the copyright owner as "Not a Contribution."
+
+      "Contributor" shall mean Licensor and any individual or Legal Entity
+      on behalf of whom a Contribution has been received by Licensor and
+      subsequently incorporated within the Work.
+
+   2. Grant of Copyright License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      copyright license to reproduce, prepare Derivative Works of,
+      publicly display, publicly perform, sublicense, and distribute the
+      Work and such Derivative Works in Source or Object form.
+
+   3. Grant of Patent License. Subject to the terms and conditions of
+      this License, each Contributor hereby grants to You a perpetual,
+      worldwide, non-exclusive, no-charge, royalty-free, irrevocable
+      (except as stated in this section) patent license to make, have made,
+      use, offer to sell, sell, import, and otherwise transfer the Work,
+      where such license applies only to those patent claims licensable
+      by such Contributor that are necessarily infringed by their
+      Contribution(s) alone or by combination of their Contribution(s)
+      with the Work to which such Contribution(s) was submitted. If You
+      institute patent litigation against any entity (including a
+      cross-claim or counterclaim in a lawsuit) alleging that the Work
+      or a Contribution incorporated within the Work constitutes direct
+      or contributory patent infringement, then any patent licenses
+      granted to You under this License for that Work shall terminate
+      as of the date such litigation is filed.
+
+   4. Redistribution. You may reproduce and distribute copies of the
+      Work or Derivative Works thereof in any medium, with or without
+      modifications, and in Source or Object form, provided that You
+      meet the following conditions:
+
+      (a) You must give any other recipients of the Work or
+          Derivative Works a copy of this License; and
+
+      (b) You must cause any modified files to carry prominent notices
+          stating that You changed the files; and
+
+      (c) You must retain, in the Source form of any Derivative Works
+          that You distribute, all copyright, patent, trademark, and
+          attribution notices from the Source form of the Work,
+          excluding those notices that do not pertain to any part of
+          the Derivative Works; and
+
+      (d) If the Work includes a "NOTICE" text file as part of its
+          distribution, then any Derivative Works that You distribute must
+          include a readable copy of the attribution notices contained
+          within such NOTICE file, excluding those notices that do not
+          pertain to any part of the Derivative Works, in at least one
+          of the following places: within a NOTICE text file distributed
+          as part of the Derivative Works; within the Source form or
+          documentation, if provided along with the Derivative Works; or,
+          within a display generated by the Derivative Works, if and
+          wherever such third-party notices normally appear. The contents
+          of the NOTICE file are for informational purposes only and
+          do not modify the License. You may add Your own attribution
+          notices within Derivative Works that You distribute, alongside
+          or as an addendum to the NOTICE text from the Work, provided
+          that such additional attribution notices cannot be construed
+          as modifying the License.
+
+      You may add Your own copyright statement to Your modifications and
+      may provide additional or different license terms and conditions
+      for use, reproduction, or distribution of Your modifications, or
+      for any such Derivative Works as a whole, provided Your use,
+      reproduction, and distribution of the Work otherwise complies with
+      the conditions stated in this License.
+
+   5. Submission of Contributions. Unless You explicitly state otherwise,
+      any Contribution intentionally submitted for inclusion in the Work
+      by You to the Licensor shall be under the terms and conditions of
+      this License, without any additional terms or conditions.
+      Notwithstanding the above, nothing herein shall supersede or modify
+      the terms of any separate license agreement you may have executed
+      with Licensor regarding such Contributions.
+
+   6. Trademarks. This License does not grant permission to use the trade
+      names, trademarks, service marks, or product names of the Licensor,
+      except as required for reasonable and customary use in describing the
+      origin of the Work and reproducing the content of the NOTICE file.
+
+   7. Disclaimer of Warranty. Unless required by applicable law or
+      agreed to in writing, Licensor provides the Work (and each
+      Contributor provides its Contributions) on an "AS IS" BASIS,
+      WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
+      implied, including, without limitation, any warranties or conditions
+      of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
+      PARTICULAR PURPOSE. You are solely responsible for determining the
+      appropriateness of using or redistributing the Work and assume any
+      risks associated with Your exercise of permissions under this License.
+
+   8. Limitation of Liability. In no event and under no legal theory,
+      whether in tort (including negligence), contract, or otherwise,
+      unless required by applicable law (such as deliberate and grossly
+      negligent acts) or agreed to in writing, shall any Contributor be
+      liable to You for damages, including any direct, indirect, special,
+      incidental, or consequential damages of any character arising as a
+      result of this License or out of the use or inability to use the
+      Work (including but not limited to damages for loss of goodwill,
+      work stoppage, computer failure or malfunction, or any and all
+      other commercial damages or losses), even if such Contributor
+      has been advised of the possibility of such damages.
+
+   9. Accepting Warranty or Additional Liability. While redistributing
+      the Work or Derivative Works thereof, You may choose to offer,
+      and charge a fee for, acceptance of support, warranty, indemnity,
+      or other liability obligations and/or rights consistent with this
+      License. However, in accepting such obligations, You may act only
+      on Your own behalf and on Your sole responsibility, not on behalf
+      of any other Contributor, and only if You agree to indemnify,
+      defend, and hold each Contributor harmless for any liability
+      incurred by, or claims asserted against, such Contributor by reason
+      of your accepting any such warranty or additional liability.
+
+   END OF TERMS AND CONDITIONS
+
+   APPENDIX: How to apply the Apache License to your work.
+
+      To apply the Apache License to your work, attach the following
+      boilerplate notice, with the fields enclosed by brackets "[]"
+      replaced with your own identifying information. (Don't include
+      the brackets!)  The text should be enclosed in the appropriate
+      comment syntax for the file format. We also recommend that a
+      file or class name and description of purpose be included on the
+      same "printed page" as the copyright notice for easier
+      identification within third-party archives.
+   Copyright 2024 Alibaba Cloud
+   Copyright 2025 OpenDatalab Shanghai AILab
+   Licensed under the Apache License, Version 2.0 (the "License");
+   you may not use this file except in compliance with the License.
+   You may obtain a copy of the License at
+       http://www.apache.org/licenses/LICENSE-2.0
+   Unless required by applicable law or agreed to in writing, software
+   distributed under the License is distributed on an "AS IS" BASIS,
+   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+   See the License for the specific language governing permissions and
+   limitations under the License.
+

zleap_parser-0.1.0/PKG-INFO

@@ -0,0 +1,131 @@
+Metadata-Version: 2.5
+Name: zleap_parser
+Version: 0.1.0
+Summary: zleap 文档解析 SDK：多格式文件/URL 解析为 markdown，经 zleap-sag 提取、octx 打包为 *.octx 归档
+Author: zleap
+License-Expression: MIT
+License-File: LICENSE
+License-File: LICENSES/Apache-2.0.txt
+Keywords: document,markdown,octx,parser,sag
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Text Processing :: Markup
+Classifier: Typing :: Typed
+Requires-Python: >=3.11
+Requires-Dist: beautifulsoup4>=4.14.3
+Requires-Dist: cloakbrowser<0.4,>=0.3.26
+Requires-Dist: octx>=0.1.3
+Requires-Dist: pycryptodome>=3.19
+Requires-Dist: pydantic<3,>=2
+Requires-Dist: uuid6>=2024.7.10
+Requires-Dist: zleap-sag>=0.7.1
+Provides-Extra: dev
+Requires-Dist: build>=1.2; extra == 'dev'
+Requires-Dist: mypy>=1.10; extra == 'dev'
+Requires-Dist: pytest-mock>=3.14; extra == 'dev'
+Requires-Dist: pytest>=8.2; extra == 'dev'
+Requires-Dist: ruff>=0.5; extra == 'dev'
+Requires-Dist: twine>=5.0; extra == 'dev'
+Provides-Extra: docx
+Requires-Dist: docling>=2.0; extra == 'docx'
+Requires-Dist: markitdown[docx]>=0.1.7; extra == 'docx'
+Provides-Extra: pdf
+Requires-Dist: docling>=2.0; extra == 'pdf'
+Requires-Dist: markitdown[pdf]>=0.1.7; extra == 'pdf'
+Requires-Dist: pypdfium2>=4.30; extra == 'pdf'
+Requires-Dist: rapidocr>=3.3; extra == 'pdf'
+Provides-Extra: redis
+Requires-Dist: redis>=5.0; extra == 'redis'
+Description-Content-Type: text/markdown
+
+# zleap_parser
+
+`zleap_parser` 是 zleap 公司开源的**文档解析 SDK**：把本地文件或远程 HTML URL 解析为 markdown，经 `sag` 提取结构化产物后由 `octx` 打包为可传播的 `.octx` Package，供下游项目以 pip 方式集成使用。
+
+> **当前状态：M1 基础框架已实施（开发中）。** 源码位于 `src/`（src 布局）：`Parser`（`parse` / `markdown_to_octx` / 自定义适配器注册）、同步浏览器 URL 采集器、六类内置适配器（plain / image / pdf / docx / html / xml，插件化接入）、`zleap-sag` 提取封装与 `octx` 打包封装、旧文件下载器 SSRF 防护与本地二级缓存。计划文档见 `plan/`。
+
+## 阅读顺序
+
+1. **计划总览**（[`plan/PLAN.md`](./plan/PLAN.md)）：项目定位、里程碑划分（M1 解析器 / M2 连接器 / M3 公共服务）与门禁
+2. **SDK 导入手册**（[`docs/sdk/install.md`](./docs/sdk/install.md)）：环境要求、安装、快速上手
+3. **配置手册**（[`docs/sdk/configuration.md`](./docs/sdk/configuration.md)）：模块级一次配置，M1/M2 复用
+4. **文档解析 API 手册**（[`docs/sdk/parse_api.md`](./docs/sdk/parse_api.md)）：`Parser` 用法与异常表
+5. **连接器 API 手册**（[`docs/sdk/connector_api.md`](./docs/sdk/connector_api.md)）：数据源同步
+6. **公共服务手册**（[`docs/service/`](./docs/service/)）：部署与 HTTP API
+
+## 快速上手
+
+需要 Python 3.11 或更高版本：
+
+```bash
+# 安装 Python 依赖
+pip install -r requirements.txt   # 上游依赖（octx / zleap-sag / uuid6 / pycryptodome + 开发工具链）
+pip install -e .                  # 或 python -m pip install zleap_parser
+
+# URL 采集首次运行前安装并检查 CloakBrowser 浏览器二进制
+python -m cloakbrowser install
+python -m cloakbrowser info
+```
+
+模块级配置一次（`llm` 与 `embedding` 必填），`Parser` 与 `Connector` 自动继承复用：
+
+```python
+import zleap_parser
+from zleap_parser import Parser, LLMConfig, EmbeddingConfig
+
+zleap_parser.config(
+    llm=LLMConfig(base_url="https://api.xxx.com/v1", api_key="sk-...", model="qwen3.6-flash"),
+    embedding=EmbeddingConfig(model="Qwen/Qwen3-Embedding-0.6B", dimensions=1024),
+)
+
+parser = Parser()
+
+# 本地文件走 magic bytes、格式适配器和文件内容缓存链路
+result = parser.parse(file="/path/to/document.pdf")
+
+# URL 只支持 HTML，走同步 CloakBrowser 正文采集链路
+url_result = parser.parse(url="https://example.com/article")
+
+# 两种入口都返回 {result: *.octx 归档文件路径, usage: LLM token 用量}
+print(result["result"], result["usage"])     # usage: prompt_tokens / completion_tokens / total_tokens
+print(url_result["result"], url_result["usage"])
+```
+
+本地冒烟测试（读取根目录 `.env` 初始化配置）：
+
+```bash
+cp .env.example .env      # 填入 LLM / Embedding 配置
+python src/main.py --list                 # 打印生效配置
+python src/main.py tests/fixtures/sample.txt   # 解析本地文件
+python src/main.py                        # 默认解析 tests/fixtures/ 下全部样例
+```
+
+## 核心能力
+
+| 能力 | 说明 | 手册 |
+| --- | --- | --- |
+| 文档解析（M1） | HTML URL / 本地文件 → markdown → `*.octx`；URL 正文规则与本地格式适配器分流 | [`parse_api.md`](./docs/sdk/parse_api.md) |
+| 数据同步（M2） | 数据源（网页搜索等）文档清单 → 逐文档解析 → `*.octx` | [`connector_api.md`](./docs/sdk/connector_api.md) |
+| 公共服务（M3） | 文档解析、连接器同步 HTTP API | [`service/`](./docs/service/) |
+
+## 要点
+
+- 产出物统一为 `*.octx` 归档文件：可读 markdown + 稳定身份、版本、完整性信息及可选的 chunks、events、entities、vectors。
+- 文件类型以 **magic bytes** 判定，不依赖扩展名。
+- **配置**：模块级 `zleap_parser.config(...)` 一次配置，`Parser`/`Connector` 自动继承；实例级 `*.config()` 与全局合并，便于高度自定义。
+- **缓存**：本地文件解析使用 Redis 或本地二级缓存；URL 输入每次重新执行浏览器采集，不使用持久缓存。
+- **URL 采集**：只校验 HTTP(S) URL 格式，不执行 DNS/IP 类型判定；每次调用独立创建并关闭 Browser、Context、Page，不使用进程级信号量，多个线程可并发调用。正文提取、正文图片筛选和 Markdown 转换沿用参考 webcrawler 的规则语义，正文质量不足严格抛 `ConversionError`。
+- **URL 附件**：成功渲染的 `page.html` 与成功下载的正文图片随 `markdown_to_octx(source_files=[...])` 以同一 `document_id` 写入 OCTX；远程 PDF/图片/普通文件不接受。
+- **pdf 兜底链路**：用户自定义适配器 → Docling → markitdown（其他兜底可追加）。
+- **docx 兜底链路**：用户自定义适配器 → Docling → markitdown；DOCX 内嵌图片随原文件写入 OCTX 附件。
+- **异常**：所有 API 失败均抛 `ZleapParserError` 及子类（`ConfigError` / `AdapterNotFoundError` / `ConversionError` / `DownloadError` / `TimeoutError` / `ExtractError`），不裸奔底层异常；URL 采集编排层保留已有 SDK 异常，并将其他内部异常包装为 `ConversionError`。
+
+## 边界
+
+- 不定义搜索 API、召回算法或 Agent 协议；解析与归档之外的能力由下游自行实现。
+- 核心解析框架与格式实现解耦：新增格式即新增适配器，不改核心代码。
+- 上游参考实现：`sag`（markdown 结构化提取）与 `octx`（`*.octx` 打包）位于本地仓库 `../SAG` 与 `../open-context`。
+- URL 正文规则直接复制 MinerU HTML webcrawler 的正文抽取器（本地命名为 `article_extractor.py`）；许可与归属见 [`THIRD_PARTY_NOTICES.md`](./THIRD_PARTY_NOTICES.md) 及 `LICENSES/`。

zleap_parser-0.1.0/README.md

@@ -0,0 +1,88 @@
+# zleap_parser
+
+`zleap_parser` 是 zleap 公司开源的**文档解析 SDK**：把本地文件或远程 HTML URL 解析为 markdown，经 `sag` 提取结构化产物后由 `octx` 打包为可传播的 `.octx` Package，供下游项目以 pip 方式集成使用。
+
+> **当前状态：M1 基础框架已实施（开发中）。** 源码位于 `src/`（src 布局）：`Parser`（`parse` / `markdown_to_octx` / 自定义适配器注册）、同步浏览器 URL 采集器、六类内置适配器（plain / image / pdf / docx / html / xml，插件化接入）、`zleap-sag` 提取封装与 `octx` 打包封装、旧文件下载器 SSRF 防护与本地二级缓存。计划文档见 `plan/`。
+
+## 阅读顺序
+
+1. **计划总览**（[`plan/PLAN.md`](./plan/PLAN.md)）：项目定位、里程碑划分（M1 解析器 / M2 连接器 / M3 公共服务）与门禁
+2. **SDK 导入手册**（[`docs/sdk/install.md`](./docs/sdk/install.md)）：环境要求、安装、快速上手
+3. **配置手册**（[`docs/sdk/configuration.md`](./docs/sdk/configuration.md)）：模块级一次配置，M1/M2 复用
+4. **文档解析 API 手册**（[`docs/sdk/parse_api.md`](./docs/sdk/parse_api.md)）：`Parser` 用法与异常表
+5. **连接器 API 手册**（[`docs/sdk/connector_api.md`](./docs/sdk/connector_api.md)）：数据源同步
+6. **公共服务手册**（[`docs/service/`](./docs/service/)）：部署与 HTTP API
+
+## 快速上手
+
+需要 Python 3.11 或更高版本：
+
+```bash
+# 安装 Python 依赖
+pip install -r requirements.txt   # 上游依赖（octx / zleap-sag / uuid6 / pycryptodome + 开发工具链）
+pip install -e .                  # 或 python -m pip install zleap_parser
+
+# URL 采集首次运行前安装并检查 CloakBrowser 浏览器二进制
+python -m cloakbrowser install
+python -m cloakbrowser info
+```
+
+模块级配置一次（`llm` 与 `embedding` 必填），`Parser` 与 `Connector` 自动继承复用：
+
+```python
+import zleap_parser
+from zleap_parser import Parser, LLMConfig, EmbeddingConfig
+
+zleap_parser.config(
+    llm=LLMConfig(base_url="https://api.xxx.com/v1", api_key="sk-...", model="qwen3.6-flash"),
+    embedding=EmbeddingConfig(model="Qwen/Qwen3-Embedding-0.6B", dimensions=1024),
+)
+
+parser = Parser()
+
+# 本地文件走 magic bytes、格式适配器和文件内容缓存链路
+result = parser.parse(file="/path/to/document.pdf")
+
+# URL 只支持 HTML，走同步 CloakBrowser 正文采集链路
+url_result = parser.parse(url="https://example.com/article")
+
+# 两种入口都返回 {result: *.octx 归档文件路径, usage: LLM token 用量}
+print(result["result"], result["usage"])     # usage: prompt_tokens / completion_tokens / total_tokens
+print(url_result["result"], url_result["usage"])
+```
+
+本地冒烟测试（读取根目录 `.env` 初始化配置）：
+
+```bash
+cp .env.example .env      # 填入 LLM / Embedding 配置
+python src/main.py --list                 # 打印生效配置
+python src/main.py tests/fixtures/sample.txt   # 解析本地文件
+python src/main.py                        # 默认解析 tests/fixtures/ 下全部样例
+```
+
+## 核心能力
+
+| 能力 | 说明 | 手册 |
+| --- | --- | --- |
+| 文档解析（M1） | HTML URL / 本地文件 → markdown → `*.octx`；URL 正文规则与本地格式适配器分流 | [`parse_api.md`](./docs/sdk/parse_api.md) |
+| 数据同步（M2） | 数据源（网页搜索等）文档清单 → 逐文档解析 → `*.octx` | [`connector_api.md`](./docs/sdk/connector_api.md) |
+| 公共服务（M3） | 文档解析、连接器同步 HTTP API | [`service/`](./docs/service/) |
+
+## 要点
+
+- 产出物统一为 `*.octx` 归档文件：可读 markdown + 稳定身份、版本、完整性信息及可选的 chunks、events、entities、vectors。
+- 文件类型以 **magic bytes** 判定，不依赖扩展名。
+- **配置**：模块级 `zleap_parser.config(...)` 一次配置，`Parser`/`Connector` 自动继承；实例级 `*.config()` 与全局合并，便于高度自定义。
+- **缓存**：本地文件解析使用 Redis 或本地二级缓存；URL 输入每次重新执行浏览器采集，不使用持久缓存。
+- **URL 采集**：只校验 HTTP(S) URL 格式，不执行 DNS/IP 类型判定；每次调用独立创建并关闭 Browser、Context、Page，不使用进程级信号量，多个线程可并发调用。正文提取、正文图片筛选和 Markdown 转换沿用参考 webcrawler 的规则语义，正文质量不足严格抛 `ConversionError`。
+- **URL 附件**：成功渲染的 `page.html` 与成功下载的正文图片随 `markdown_to_octx(source_files=[...])` 以同一 `document_id` 写入 OCTX；远程 PDF/图片/普通文件不接受。
+- **pdf 兜底链路**：用户自定义适配器 → Docling → markitdown（其他兜底可追加）。
+- **docx 兜底链路**：用户自定义适配器 → Docling → markitdown；DOCX 内嵌图片随原文件写入 OCTX 附件。
+- **异常**：所有 API 失败均抛 `ZleapParserError` 及子类（`ConfigError` / `AdapterNotFoundError` / `ConversionError` / `DownloadError` / `TimeoutError` / `ExtractError`），不裸奔底层异常；URL 采集编排层保留已有 SDK 异常，并将其他内部异常包装为 `ConversionError`。
+
+## 边界
+
+- 不定义搜索 API、召回算法或 Agent 协议；解析与归档之外的能力由下游自行实现。
+- 核心解析框架与格式实现解耦：新增格式即新增适配器，不改核心代码。
+- 上游参考实现：`sag`（markdown 结构化提取）与 `octx`（`*.octx` 打包）位于本地仓库 `../SAG` 与 `../open-context`。
+- URL 正文规则直接复制 MinerU HTML webcrawler 的正文抽取器（本地命名为 `article_extractor.py`）；许可与归属见 [`THIRD_PARTY_NOTICES.md`](./THIRD_PARTY_NOTICES.md) 及 `LICENSES/`。

zleap_parser-0.1.0/THIRD_PARTY_NOTICES.md

@@ -0,0 +1,27 @@
+# Third-Party Notices
+
+## MinerU HTML webcrawler rules
+
+URL 正文提取、正文图片筛选和 Markdown 渲染的部分实现位于：
+
+- `src/zleap_parser/url_collector/webcrawler/article_extractor.py`（参考原 `htmlx.py` 逐字节复制后改名）
+- `src/zleap_parser/url_collector/webcrawler/models.py`
+- `src/zleap_parser/url_collector/webcrawler/http_client.py`（同步入口保留参考异步签名，依赖延迟导入）
+- `src/zleap_parser/url_collector/webcrawler/proxy.py`
+- `src/zleap_parser/url_collector/webcrawler/resource_cache.py`
+- `src/zleap_parser/url_collector/markdown.py`
+- `src/zleap_parser/url_collector/resource_cache.py`
+
+`article_extractor.py`（原 `htmlx.py`）副本 SHA-256：
+`3b54647de752f05d28c65bc2229b3a85f6d09583237188977c4f61d6608ceeaf`
+
+这些实现派生自用户提供的 `mineru-html-feat-crawl-0714` 中
+`dripper/crawl/vendors/webcrawler` 代码，原项目采用 Apache License 2.0。
+Apache License 2.0 全文随本项目发布于 `LICENSES/Apache-2.0.txt`。
+
+原项目 NOTICE：
+
+> This project contains code and model weights derived from Qwen3.  
+> Original Qwen3 Copyright 2024 Alibaba Cloud, licensed under Apache License 2.0.  
+> Modifications and additional training Copyright 2025 OpenDatalab Shanghai AILab,
+> licensed under Apache License 2.0.

zleap_parser-0.1.0/docs/README.md

@@ -0,0 +1,28 @@
+# zleap_parser 用户手册
+
+> 本文档集面向下游用户与部署运维方。**M1（解析器 SDK）与 M2（连接器）已实施**，SDK 手册随实现同步修订；**M3（公共服务）待规划**，对应手册仍基于计划文档（`plan/`）编写，实现后随代码修订。
+
+## 手册索引
+
+### SDK 手册（M1 + M2，面向 SDK 使用者）
+
+| 手册 | 内容 |
+| --- | --- |
+| [SDK 导入手册](./sdk/install.md) | 环境要求、安装、导入、快速上手 |
+| [配置手册](./sdk/configuration.md) | 模块级/实例级配置、配置项字段、合并规则、缓存策略 |
+| [文档解析 API 手册](./sdk/parse_api.md) | `Parser`：URL/文件 → `*.octx`、异常表 |
+| [连接器数据同步 API 手册](./sdk/connector_api.md) | `Connector`：数据源同步 → `*.octx` 路径列表（单次可加载多文件）、可用性检查 |
+
+### 公共服务手册（M3，面向部署运维与调用方）
+
+| 手册 | 内容 |
+| --- | --- |
+| [公共服务部署手册](./service/deployment.md) | 服务构建部署、config.toml 全字段与环境变量、探针、运维 |
+| [公共服务 API 手册](./service/api.md) | HTTP API：鉴权、端点、任务状态机、错误码 |
+
+## 概念速览
+
+- **格式适配器**（`zleap_parser.adapters`）：把文件格式转 markdown 的插件（M1），类 `BaseAdapter`/`AdapterRegistry`。
+- **数据源**（`zleap_parser.connector.sources`）：获取文档清单的插件（M2），类 `DataSourceAdapter`/`SourceRegistry`。
+- **`*.octx`**：解析产物归档文件，SDK 与公共服务所有 API 的最终产出物。
+- **兜底链路**：pdf 转换（用户自定义适配器 → Docling → markitdown）、缓存（Redis → 本地二级）等优先路径失败时自动降级。