sdpy-kit 0.1.0__tar.gz

sdpy_kit-0.1.0/.gitignore

@@ -0,0 +1,95 @@
+# 忽略匹配下列规则的Git 提交 V2.1.0
+### gradle ###
+.gradle
+/build/
+!gradle/wrapper/gradle-wrapper.jar
+
+### STS ###
+.settings/
+.apt_generated
+.classpath
+.factorypath
+.project
+.settings
+.springBeans
+bin/
+
+### IntelliJ IDEA ###
+.idea
+*.iws
+*.iml
+*.ipr
+*.lock
+rebel.xml
+
+### NetBeans ###
+nbproject/private/
+build/
+nbbuild/
+nbdist/
+.nb-gradle/
+
+### maven ###
+target/
+*.war
+*.ear
+*.zip
+*.tar
+*.tar.gz
+
+### logs ####
+/logs/
+*.log
+
+### temp ignore ###
+*.cache
+*.diff
+*.patch
+*.tmp
+*.java~
+*.properties~
+*.xml~
+
+### system ignore ###
+.DS_Store
+Thumbs.db
+Servers
+.metadata
+# 根目录 data/ 运行时数据（zvec 向量库、SQLite 测试库等，不入库）
+data/
+# AI 文生图 e2e 测试产物（生成的海报图片，不入库）
+tests/image/
+# AMiner 检索测试产物（md 报告，不入库）
+tests/aminer/
+# SQLite 测试库（运行时生成，不入库）
+metadata_log.db
+upload
+gen_code
+
+### node ###
+node_modules
+__pycache__/
+.vscode/
+.venv/
+.trae/
+.codebuddy/
+.qoder/
+
+### python dependencies ###
+# 项目统一使用 uv + pyproject.toml 管理依赖，禁止 requirements.txt
+requirements.txt
+requirements-dev.txt
+requirements*.txt
+
+### secrets ###
+# 环境变量文件含 API 密钥等敏感信息，禁止入库
+.env
+.env.*
+docs/current-task/
+research/
+
+### sdpy-kit 发布工程 ###
+# kit/ 为发布脚本从主工程同步的副本（构建时生成，不入库）
+publish/sdpy-kit/kit/
+publish/sdpy-kit/dist/
+publish/sdpy-kit/*.egg-info/

sdpy_kit-0.1.0/PKG-INFO

@@ -0,0 +1,128 @@
+Metadata-Version: 2.5
+Name: sdpy-kit
+Version: 0.1.0
+Summary: SDPY Kit——统一数据库访问层（PG/MySQL/SQLite）与 AI/MCP/Redis/文件/文档/向量工具集
+Author-email: Clark <changhongyuan@126.com>
+License-Expression: MIT
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3 :: Only
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Python: >=3.13
+Requires-Dist: fastmcp>=4.0.3
+Requires-Dist: firecrawl-anydoc>=0.2.4
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: markdown-it-py>=3.0
+Requires-Dist: pimd[pdfa]>=2.2.5
+Requires-Dist: psycopg2-binary>=2.9.12
+Requires-Dist: pydantic-settings>=2.0
+Requires-Dist: pydantic>=2.0
+Requires-Dist: pymysql>=1.2.0
+Requires-Dist: python-dotenv>=1.2.3
+Requires-Dist: redis>=5.0.0
+Requires-Dist: requests>=2.34.0
+Requires-Dist: sqlalchemy~=2.0.51
+Requires-Dist: sqliteai-vector>=1.0.0
+Requires-Dist: sqlmodel>=0.0.39
+Requires-Dist: zvec>=0.6.0
+Description-Content-Type: text/markdown
+
+# sdpy-kit
+
+SDPY 统一工具集（Kit 体系），供 Python 项目开箱即用：统一数据库访问层（PostgreSQL / MySQL / SQLite，三个 Kit API 完全一致）+ AI 多模型服务池（chat / embedding / rerank / image，多厂商候选链与故障转移）+ MCP 客户端 + Redis / 文件 / 文档 / 向量库 / MinerU 工具。
+
+> 要求 Python 3.13+
+
+## 安装
+
+```bash
+pip install sdpy-kit
+# 或
+uv add sdpy-kit
+```
+
+## 配置（.env）
+
+kit 通过 pydantic-settings 读取项目根目录 `.env`（默认值见下，均可省略）：
+
+```dotenv
+# 日志
+LOG_LEVEL=INFO
+LOG_TO_CONSOLE=true
+LOG_TO_FILE=false
+
+# 数据库（三选一，切换只改 DEFAULT_DB_TYPE）
+DEFAULT_DB_TYPE=sqlite          # postgresql / mysql / sqlite
+DB_HOST=localhost
+DB_PORT=5432
+DB_DATABASE=postgres
+DB_USER=admin
+DB_PASSWORD=
+MYSQL_PORT=3306
+MYSQL_DATABASE=mydb
+MYSQL_USER=root
+SQLITE_DB_PATH=./data/app.db
+
+# Redis
+REDIS_HOST=localhost
+REDIS_PORT=6379
+```
+
+## AI / MCP 功能的档案文件
+
+AI 与 MCP 功能需要在项目根目录提供两个 JSON 档案文件：
+
+- `config/ai_models.json` —— chat / embedding / rerank / image 四类型多厂商模型档案
+- `config/mcp_servers.json` —— MCP 服务器档案
+
+文件缺失时，首次使用相应功能会报错，**报错信息中附带可直接复制的最小模板**；
+其中 `api_key` / `token_env` 字段填写环境变量名，真实密钥配置在 `.env`。
+
+## 快速开始
+
+### 日志与配置
+
+```python
+from kit.config import get_settings, get_default_logger
+
+settings = get_settings()
+logger = get_default_logger(__name__)
+logger.info("hello %s", settings.APP_NAME)
+```
+
+### 数据库（SQLiteKit / PgKit / MySqlKit API 完全一致，切换只改 import）
+
+```python
+from sqlmodel import SQLModel, Field
+from kit.db.sqlite_kit import SQLiteKit
+
+class Item(SQLModel, table=True):
+    id: int | None = Field(default=None, primary_key=True)
+    name: str
+
+SQLiteKit.connect()
+SQLiteKit.create_tables([Item])
+item = SQLiteKit.create(Item, {"name": "demo"})
+```
+
+约定：`SQLiteKit/PgKit/MySqlKit` 方法自动提交；`CRUDBase`（配合 `session_scope`）不自动提交；事务用 `kit.transactional`。
+
+## 模块总览
+
+| 模块 | 内容 |
+|------|------|
+| `kit.db` | 统一数据库访问层：PgKit / MySqlKit / SQLiteKit、CRUDBase、transactional |
+| `kit.ai` | AI 服务池：chat / embedding / rerank / image，多厂商候选链 + 故障转移 |
+| `kit.mcp_kit` | 通用 MCP 客户端（streamable-http / sse，档案驱动多服务器） |
+| `kit.redis_kit` | Redis 工具 |
+| `kit.file_kit` | 文件工具 |
+| `kit.doc_kit` | 文档转换（基于 firecrawl-anydoc / pimd） |
+| `kit.zvec_kit` | zvec 本地向量库 |
+| `kit.mineru_kit` | MinerU 文档精准解析 API |
+| `kit.common_kit` | 通用工具 |
+
+## License
+
+MIT

sdpy_kit-0.1.0/README.md

@@ -0,0 +1,97 @@
+# sdpy-kit
+
+SDPY 统一工具集（Kit 体系），供 Python 项目开箱即用：统一数据库访问层（PostgreSQL / MySQL / SQLite，三个 Kit API 完全一致）+ AI 多模型服务池（chat / embedding / rerank / image，多厂商候选链与故障转移）+ MCP 客户端 + Redis / 文件 / 文档 / 向量库 / MinerU 工具。
+
+> 要求 Python 3.13+
+
+## 安装
+
+```bash
+pip install sdpy-kit
+# 或
+uv add sdpy-kit
+```
+
+## 配置（.env）
+
+kit 通过 pydantic-settings 读取项目根目录 `.env`（默认值见下，均可省略）：
+
+```dotenv
+# 日志
+LOG_LEVEL=INFO
+LOG_TO_CONSOLE=true
+LOG_TO_FILE=false
+
+# 数据库（三选一，切换只改 DEFAULT_DB_TYPE）
+DEFAULT_DB_TYPE=sqlite          # postgresql / mysql / sqlite
+DB_HOST=localhost
+DB_PORT=5432
+DB_DATABASE=postgres
+DB_USER=admin
+DB_PASSWORD=
+MYSQL_PORT=3306
+MYSQL_DATABASE=mydb
+MYSQL_USER=root
+SQLITE_DB_PATH=./data/app.db
+
+# Redis
+REDIS_HOST=localhost
+REDIS_PORT=6379
+```
+
+## AI / MCP 功能的档案文件
+
+AI 与 MCP 功能需要在项目根目录提供两个 JSON 档案文件：
+
+- `config/ai_models.json` —— chat / embedding / rerank / image 四类型多厂商模型档案
+- `config/mcp_servers.json` —— MCP 服务器档案
+
+文件缺失时，首次使用相应功能会报错，**报错信息中附带可直接复制的最小模板**；
+其中 `api_key` / `token_env` 字段填写环境变量名，真实密钥配置在 `.env`。
+
+## 快速开始
+
+### 日志与配置
+
+```python
+from kit.config import get_settings, get_default_logger
+
+settings = get_settings()
+logger = get_default_logger(__name__)
+logger.info("hello %s", settings.APP_NAME)
+```
+
+### 数据库（SQLiteKit / PgKit / MySqlKit API 完全一致，切换只改 import）
+
+```python
+from sqlmodel import SQLModel, Field
+from kit.db.sqlite_kit import SQLiteKit
+
+class Item(SQLModel, table=True):
+    id: int | None = Field(default=None, primary_key=True)
+    name: str
+
+SQLiteKit.connect()
+SQLiteKit.create_tables([Item])
+item = SQLiteKit.create(Item, {"name": "demo"})
+```
+
+约定：`SQLiteKit/PgKit/MySqlKit` 方法自动提交；`CRUDBase`（配合 `session_scope`）不自动提交；事务用 `kit.transactional`。
+
+## 模块总览
+
+| 模块 | 内容 |
+|------|------|
+| `kit.db` | 统一数据库访问层：PgKit / MySqlKit / SQLiteKit、CRUDBase、transactional |
+| `kit.ai` | AI 服务池：chat / embedding / rerank / image，多厂商候选链 + 故障转移 |
+| `kit.mcp_kit` | 通用 MCP 客户端（streamable-http / sse，档案驱动多服务器） |
+| `kit.redis_kit` | Redis 工具 |
+| `kit.file_kit` | 文件工具 |
+| `kit.doc_kit` | 文档转换（基于 firecrawl-anydoc / pimd） |
+| `kit.zvec_kit` | zvec 本地向量库 |
+| `kit.mineru_kit` | MinerU 文档精准解析 API |
+| `kit.common_kit` | 通用工具 |
+
+## License
+
+MIT

sdpy_kit-0.1.0/kit/__init__.py

@@ -0,0 +1,68 @@
+"""
+Copyright (c) 2021-2026 Clark Chang. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+Project: sdpy
+Author: Clark Chang
+"""
+
+# date: 2026-08-23
+"""Kit 模块——统一数据库访问层 + 工具集
+
+数据库访问通过 kit.db 子包提供（基于 SQLModel）。
+"""
+
+__version__ = "0.1.0"
+
+from kit.db import (
+    MYSQL,
+    POSTGRESQL,
+    SQLITE,
+    CRUDBase,
+    DBError,
+    NotFoundError,
+    close_all_engines,
+    close_engine,
+    create_tables,
+    get_engine,
+    get_session,
+    list_tables,
+    session_scope,
+    table_exists,
+    transactional,
+)
+
+__all__ = [
+    # 引擎管理
+    "get_engine",
+    "create_tables",
+    "list_tables",
+    "table_exists",
+    "close_all_engines",
+    "close_engine",
+    # 常量
+    "POSTGRESQL",
+    "MYSQL",
+    "SQLITE",
+    # 会话
+    "session_scope",
+    "get_session",
+    # 异常
+    "DBError",
+    "NotFoundError",
+    # CRUD
+    "CRUDBase",
+    # 事务
+    "transactional",
+]

sdpy_kit-0.1.0/kit/ai/__init__.py

@@ -0,0 +1,47 @@
+"""
+Copyright (c) 2021-2026 Clark Chang. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+Project: sdpy
+Author: Clark Chang
+"""
+
+# date: 2026-08-21
+"""kit/ai - AI 模型服务模块（纯异步，Python 3.13+）
+
+提供开箱即用的各类模型基础服务：
+- ChatService: 对话模型
+- EmbeddingService: 文本嵌入向量
+- RerankService: 重排序
+- ImageService: 文生图
+"""
+
+from kit.ai.chat_service import ChatRequest, ChatService
+from kit.ai.embedding_service import EmbeddingRequest, EmbeddingService
+from kit.ai.image_service import ImageRequest, ImageService
+from kit.ai.model_builder import ModelBuilder
+from kit.ai.rerank_service import RerankHit, RerankRequest, RerankService
+
+__all__ = [
+    "ChatRequest",
+    "ChatService",
+    "EmbeddingRequest",
+    "EmbeddingService",
+    "ImageRequest",
+    "ImageService",
+    "ModelBuilder",
+    "RerankHit",
+    "RerankRequest",
+    "RerankService",
+]

sdpy_kit-0.1.0/kit/ai/base_service.py

@@ -0,0 +1,94 @@
+"""
+Copyright (c) 2021-2026 Clark Chang. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+Project: sdpy
+Author: Clark Chang
+"""
+
+from abc import ABC, abstractmethod
+from typing import Self, override
+
+from kit.ai.exceptions import AIResponseError
+from kit.ai.model_client import ModelClient
+from kit.ai.profiles import ModelProfile
+from kit.ai.types import JSONObject, JSONValue
+
+__all__ = ["BaseAIService", "safe_get"]
+
+
+def safe_get(data: JSONValue, *path: str | int) -> JSONValue:
+    """按层级路径安全取值，支持 dict 键（str）与 list 索引（int）。
+
+    Raises:
+        AIResponseError: 路径任一环节缺失或类型不符。
+    """
+    cur: JSONValue = data
+    for key in path:
+        if isinstance(key, int):
+            if not isinstance(cur, list) or key < 0 or key >= len(cur):
+                raise AIResponseError(f"响应缺少索引 {path}: {str(data)[:200]}")
+            cur = cur[key]
+        else:
+            if not isinstance(cur, dict) or key not in cur:
+                raise AIResponseError(
+                    f"响应缺少字段 {'.'.join(map(str, path))}: {str(data)[:200]}"
+                )
+            cur = cur[key]
+    return cur
+
+
+class BaseAIService[RequestT, ResponseT](ABC):
+    """AI 服务基类（模板方法 + 泛型）。
+
+    子类实现同步的 ``_build_payload``（请求模型→JSON 请求体）与
+    ``_parse_response``（JSON 响应体→业务结果），公共的「构造请求、
+    发送、解析」由 ``_call`` 统一编排。请求类型 ``RequestT`` 与结果
+    类型 ``ResponseT`` 由泛型绑定，保证覆写签名一致、类型自洽。
+    """
+
+    def __init__(self, profile: ModelProfile) -> None:
+        self._profile: ModelProfile = profile
+        self._client: ModelClient = ModelClient(profile)
+
+    @property
+    def profile(self) -> ModelProfile:
+        return self._profile
+
+    @abstractmethod
+    def _build_payload(self, request: RequestT) -> JSONObject:
+        """将请求模型构造为 JSON 请求体。"""
+
+    @abstractmethod
+    def _parse_response(self, data: JSONObject) -> ResponseT:
+        """将 JSON 响应体解析为业务结果。"""
+
+    async def _call(self, request: RequestT) -> ResponseT:
+        """统一编排：构造请求体 → 发送 → 解析响应。"""
+        payload = self._build_payload(request)
+        data = await self._client.post_json(payload)
+        return self._parse_response(data)
+
+    async def close(self) -> None:
+        await self._client.close()
+
+    async def __aenter__(self) -> Self:
+        return self
+
+    async def __aexit__(self, *exc_info: object) -> None:
+        await self.close()
+
+    @override
+    def __str__(self) -> str:
+        return f"{type(self).__name__}({self._profile.name})"

sdpy_kit-0.1.0/kit/ai/chat_service.py

@@ -0,0 +1,110 @@
+"""
+Copyright (c) 2021-2026 Clark Chang. All Rights Reserved.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+
+Project: sdpy
+Author: Clark Chang
+"""
+
+from collections.abc import AsyncIterator
+from dataclasses import dataclass
+from typing import cast, override
+
+from kit.ai.base_service import BaseAIService, safe_get
+from kit.ai.exceptions import AIResponseError
+from kit.ai.types import JSONObject
+from kit.config.log_config import get_default_logger
+
+__all__ = ["ChatRequest", "ChatService"]
+
+logger = get_default_logger(__name__)
+
+Message = dict[str, str]
+
+
+@dataclass(frozen=True)
+class ChatRequest:
+    """对话请求模型。"""
+
+    messages: list[Message]
+    temperature: float | None = None
+    max_tokens: int | None = None
+    stream: bool = False
+
+
+class ChatService(BaseAIService[ChatRequest, str]):
+    """对话类模型服务（全异步）。"""
+
+    @override
+    def _build_payload(self, request: ChatRequest) -> JSONObject:
+        payload: dict[str, object] = {
+            "model": self._profile.model,
+            "messages": request.messages,
+            "stream": request.stream,
+        }
+        if request.temperature is not None:
+            payload["temperature"] = request.temperature
+        if request.max_tokens is not None:
+            payload["max_tokens"] = request.max_tokens
+        return cast(JSONObject, payload)
+
+    @override
+    def _parse_response(self, data: JSONObject) -> str:
+        content = safe_get(data, "choices", 0, "message", "content")
+        if not isinstance(content, str):
+            raise AIResponseError(f"content 类型异常: {type(content).__name__}")
+        return content
+
+    async def ask(
+        self,
+        messages: list[Message],
+        *,
+        temperature: float | None = None,
+        max_tokens: int | None = None,
+    ) -> str:
+        request = ChatRequest(messages, temperature=temperature, max_tokens=max_tokens)
+        return await self._call(request)
+
+    async def complete(
+        self,
+        prompt: str,
+        *,
+        temperature: float | None = None,
+        max_tokens: int | None = None,
+    ) -> str:
+        messages: list[Message] = [{"role": "user", "content": prompt}]
+        return await self.ask(messages, temperature=temperature, max_tokens=max_tokens)
+
+    async def stream(
+        self,
+        messages: list[Message],
+        *,
+        temperature: float | None = None,
+        max_tokens: int | None = None,
+    ) -> AsyncIterator[str]:
+        request = ChatRequest(
+            messages, temperature=temperature, max_tokens=max_tokens, stream=True
+        )
+        payload = self._build_payload(request)
+        async for parsed in self._client.stream_json(payload):
+            if parsed is None:
+                continue
+            try:
+                delta = safe_get(parsed, "choices", 0, "delta", "content")
+            except AIResponseError:
+                # 流式结束帧等无 delta 的正常帧，跳过
+                logger.debug("流式帧缺少 delta.content，跳过")
+                continue
+            if isinstance(delta, str) and delta:
+                yield delta