xiaoshiai-hub 0.1.0__tar.gz → 0.1.2__tar.gz

xiaoshiai_hub-0.1.2/MANIFEST.in

@@ -0,0 +1,31 @@
+# Include documentation files
+include README.md
+include README_EN.md
+include LICENSE
+
+# Include requirements files
+include requirements.txt
+include requirements-dev.txt
+include requirements-test.txt
+
+# Include configuration files
+include pyproject.toml
+include setup.py
+
+# Include package files
+recursive-include xiaoshiai_hub *.py
+
+# Include examples (optional, for reference)
+recursive-include examples *.py *.md
+
+# Exclude test files from distribution
+recursive-exclude tests *
+
+# Exclude development files
+exclude .gitignore
+
+# Exclude build artifacts
+global-exclude __pycache__
+global-exclude *.py[cod]
+global-exclude *.so
+global-exclude .DS_Store

xiaoshiai_hub-0.1.2/PKG-INFO

@@ -0,0 +1,560 @@
+Metadata-Version: 2.4
+Name: xiaoshiai-hub
+Version: 0.1.2
+Summary: Python SDK for XiaoShi AI Hub - Upload, download, and manage AI models and datasets with encryption support
+Home-page: https://github.com/poxiaoyun/moha-sdk
+Author: XiaoShi AI
+Author-email: XiaoShi AI <support@xiaoshiai.cn>
+Maintainer: XiaoShi AI Team
+Maintainer-email: XiaoShi AI Team <support@xiaoshiai.cn>
+License: Apache-2.0
+Project-URL: Homepage, https://github.com/poxiaoyun/moha-sdk
+Project-URL: Repository, https://github.com/poxiaoyun/moha-sdk
+Project-URL: Issues, https://github.com/poxiaoyun/moha-sdk/issues
+Keywords: xiaoshi,ai-hub,machine-learning,deep-learning,model-hub,dataset,encryption,sdk,api-client
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: Science/Research
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.7
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3 :: Only
+Requires-Python: >=3.7
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests>=2.20.0
+Requires-Dist: tqdm>=4.62.0
+Provides-Extra: upload
+Requires-Dist: gitpython>=3.1.0; extra == "upload"
+Requires-Dist: cryptography>=3.4.0; extra == "upload"
+Provides-Extra: encryption
+Requires-Dist: cryptography>=3.4.0; extra == "encryption"
+Requires-Dist: gmssl>=3.2.1; extra == "encryption"
+Provides-Extra: dev
+Requires-Dist: pytest>=6.0; extra == "dev"
+Requires-Dist: pytest-cov>=2.0; extra == "dev"
+Requires-Dist: pytest-mock>=3.6.0; extra == "dev"
+Requires-Dist: black>=22.0; extra == "dev"
+Requires-Dist: flake8>=4.0; extra == "dev"
+Requires-Dist: mypy>=0.950; extra == "dev"
+Requires-Dist: isort>=5.10.0; extra == "dev"
+Requires-Dist: build>=0.8.0; extra == "dev"
+Requires-Dist: twine>=4.0.0; extra == "dev"
+Requires-Dist: gitpython>=3.1.0; extra == "dev"
+Requires-Dist: cryptography>=3.4.0; extra == "dev"
+Requires-Dist: gmssl>=3.2.1; extra == "dev"
+Provides-Extra: all
+Requires-Dist: gitpython>=3.1.0; extra == "all"
+Requires-Dist: cryptography>=3.4.0; extra == "all"
+Requires-Dist: gmssl>=3.2.1; extra == "all"
+Dynamic: author
+Dynamic: home-page
+Dynamic: license-file
+Dynamic: maintainer
+Dynamic: requires-python
+
+# XiaoShi AI Hub Python SDK
+
+[![PyPI version](https://badge.fury.io/py/xiaoshiai-hub.svg)](https://badge.fury.io/py/xiaoshiai-hub)
+[![Python Support](https://img.shields.io/pypi/pyversions/xiaoshiai-hub.svg)](https://pypi.org/project/xiaoshiai-hub/)
+[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
+
+XiaoShi AI Hub Python SDK 是一个功能强大的 Python 库，用于与 XiaoShi AI Hub 平台进行交互。它提供了简单易用的 API，支持模型和数据集的上传、下载、加密等功能。
+
+[English](README_EN.md) | 简体中文
+
+## ✨ 特性
+
+- 🚀 **简单易用** - 类似 Hugging Face Hub 的 API 设计，上手即用
+- 📥 **下载功能** - 支持下载单个文件或整个仓库
+- 📤 **上传功能** - 支持上传文件和文件夹到仓库
+- 🔐 **加密支持** - 内置多种加密算法（AES、SM4、RSA、SM2）
+- 🎯 **模式匹配** - 支持使用 allow/ignore 模式过滤文件
+- 📊 **进度显示** - 下载和上传时显示进度条
+- 🔑 **多种认证** - 支持用户名/密码和 Token 认证
+- 🌐 **环境变量配置** - 灵活的 Hub URL 配置
+- 💾 **缓存支持** - 高效的文件缓存机制
+- 🔍 **类型提示** - 完整的类型注解，IDE 友好
+
+## 📦 安装
+
+### 基础安装（仅下载功能）
+
+```bash
+pip install xiaoshiai-hub
+```
+
+### 完整安装（包含上传功能）
+
+```bash
+pip install xiaoshiai-hub[upload]
+```
+
+### 开发安装
+
+```bash
+pip install xiaoshiai-hub[dev]
+```
+
+### 完整安装（所有功能）
+
+```bash
+pip install xiaoshiai-hub[all]
+```
+
+## 🚀 快速开始
+
+### 下载单个文件
+
+```python
+from xiaoshiai_hub import moha_hub_download
+
+# 下载单个文件
+file_path = moha_hub_download(
+    repo_id="demo/demo",
+    filename="config.yaml",
+    repo_type="models",  # 或 "datasets"
+    username="your-username",
+    password="your-password",
+)
+print(f"文件已下载到: {file_path}")
+```
+
+### 下载整个仓库
+
+```python
+from xiaoshiai_hub import snapshot_download
+
+# 下载整个仓库
+repo_path = snapshot_download(
+    repo_id="demo/demo",
+    repo_type="models",
+    username="your-username",
+    password="your-password",
+)
+print(f"仓库已下载到: {repo_path}")
+```
+
+### 使用过滤器下载
+
+```python
+from xiaoshiai_hub import snapshot_download
+
+# 只下载 YAML 和 Markdown 文件
+repo_path = snapshot_download(
+    repo_id="demo/demo",
+    allow_patterns=["*.yaml", "*.yml", "*.md"],
+    ignore_patterns=[".git*", "*.log"],
+    username="your-username",
+    password="your-password",
+)
+```
+
+### 上传文件
+
+```python
+from xiaoshiai_hub import upload_file
+
+# 上传单个文件
+commit_hash = upload_file(
+    path_or_fileobj="./config.yaml",
+    path_in_repo="config.yaml",
+    repo_id="demo/my-model",
+    repo_type="models",
+    commit_message="Upload config file",
+    username="your-username",
+    password="your-password",
+)
+print(f"提交哈希: {commit_hash}")
+```
+
+### 上传文件夹
+
+```python
+from xiaoshiai_hub import upload_folder
+
+# 上传整个文件夹
+commit_hash = upload_folder(
+    folder_path="./my_model",
+    repo_id="demo/my-model",
+    repo_type="models",
+    commit_message="Upload model files",
+    ignore_patterns=["*.log", ".git*"],  # 忽略这些文件
+    username="your-username",
+    password="your-password",
+)
+print(f"提交哈希: {commit_hash}")
+```
+
+### 加密上传和下载
+
+```python
+from xiaoshiai_hub import upload_file, moha_hub_download
+from xiaoshiai_hub.encryption import EncryptionAlgorithm
+
+# 上传加密文件
+commit_hash = upload_file(
+    path_or_fileobj="./secret.txt",
+    path_in_repo="secret.txt",
+    repo_id="demo/encrypted-repo",
+    encryption_key="your-32-character-secret-key",
+    encryption_algorithm=EncryptionAlgorithm.AES_256_CBC,
+    username="your-username",
+    password="your-password",
+)
+
+# 下载并解密文件
+file_path = moha_hub_download(
+    repo_id="demo/encrypted-repo",
+    filename="secret.txt",
+    decryption_key="your-32-character-secret-key",
+    decryption_algorithm=EncryptionAlgorithm.AES_256_CBC,
+    username="your-username",
+    password="your-password",
+)
+```
+
+### 使用 HubClient API
+
+```python
+from xiaoshiai_hub import HubClient
+
+# 创建客户端
+client = HubClient(
+    username="your-username",
+    password="your-password",
+)
+
+# 获取仓库信息
+repo_info = client.get_repository_info("demo", "models", "my-model")
+print(f"仓库名称: {repo_info.name}")
+print(f"组织: {repo_info.organization}")
+
+# 列出分支
+branches = client.list_branches("demo", "models", "my-model")
+for branch in branches:
+    print(f"分支: {branch.name} (commit: {branch.commit_sha})")
+
+# 浏览仓库内容
+content = client.get_repository_content("demo", "models", "my-model", "main")
+for entry in content.entries:
+    print(f"{entry.type}: {entry.name}")
+```
+
+## 🔐 加密功能
+
+SDK 支持多种加密算法：
+
+### 对称加密
+
+- **AES-256-CBC** - 标准 AES 加密
+- **AES-256-GCM** - AES 加密（带认证）
+- **SM4-CBC** - 国密 SM4 加密
+- **SM4-GCM** - 国密 SM4 加密（带认证）
+
+### 非对称加密
+
+- **RSA-OAEP** - RSA 加密（推荐）
+- **RSA-PKCS1V15** - RSA 加密（兼容性）
+- **SM2** - 国密 SM2 加密
+
+### 生成加密密钥
+
+```python
+import secrets
+import string
+
+# 生成对称加密密钥（32 字符）
+symmetric_key = ''.join(secrets.choice(string.ascii_letters + string.digits) for _ in range(32))
+print(f"对称密钥: {symmetric_key}")
+
+# 生成 RSA 密钥对
+from cryptography.hazmat.primitives.asymmetric import rsa
+from cryptography.hazmat.primitives import serialization
+
+private_key = rsa.generate_private_key(
+    public_exponent=65537,
+    key_size=2048,
+)
+public_key = private_key.public_key()
+
+# 序列化公钥
+public_pem = public_key.public_bytes(
+    encoding=serialization.Encoding.PEM,
+    format=serialization.PublicFormat.SubjectPublicKeyInfo
+).decode()
+
+# 序列化私钥
+private_pem = private_key.private_bytes(
+    encoding=serialization.Encoding.PEM,
+    format=serialization.PrivateFormat.PKCS8,
+    encryption_algorithm=serialization.NoEncryption()
+).decode()
+```
+
+### 部分文件加密
+
+```python
+from xiaoshiai_hub import upload_folder
+from xiaoshiai_hub.encryption import EncryptionAlgorithm
+
+# 上传文件夹，只加密部分文件
+commit_hash = upload_folder(
+    folder_path="./my_model",
+    repo_id="demo/my-model",
+    encryption_key="your-secret-key",
+    encryption_algorithm=EncryptionAlgorithm.AES_256_CBC,
+    encryption_exclude=["README.md", "*.yaml"],  # 这些文件不加密
+    username="your-username",
+    password="your-password",
+)
+```
+
+## ⚙️ 配置
+
+### 环境变量
+
+```bash
+# Hub 服务端点
+export MOHA_ENDPOINT="https://your-hub-url.com/moha"
+
+# 认证信息
+export MOHA_USERNAME="your-username"
+export MOHA_PASSWORD="your-password"
+
+# 加密密钥（可选）
+export ENCRYPTION_KEY="your-32-character-encryption-key"
+export DECRYPTION_KEY="your-32-character-decryption-key"
+```
+
+### 使用 .env 文件
+
+创建 `.env` 文件：
+
+```bash
+MOHA_ENDPOINT=https://your-hub-url.com/moha
+MOHA_USERNAME=your-username
+MOHA_PASSWORD=your-password
+ENCRYPTION_KEY=your-encryption-key
+```
+
+然后在代码中加载：
+
+```python
+from dotenv import load_dotenv
+load_dotenv()
+
+from xiaoshiai_hub import moha_hub_download
+
+# 自动使用环境变量中的认证信息
+file_path = moha_hub_download(
+    repo_id="demo/demo",
+    filename="config.yaml",
+)
+```
+
+## 📚 示例代码
+
+项目包含丰富的示例代码，位于 `examples/` 目录：
+
+- **01_basic_usage.py** - 基础使用示例
+- **02_download_file.py** - 下载文件示例
+- **03_download_repository.py** - 下载仓库示例
+- **04_upload_file.py** - 上传文件示例
+- **05_upload_folder.py** - 上传文件夹示例
+- **06_encryption.py** - 加密功能示例
+- **07_complete_workflow.py** - 完整工作流示例
+
+运行示例：
+
+```bash
+# 设置环境变量
+export MOHA_USERNAME="your-username"
+export MOHA_PASSWORD="your-password"
+
+# 运行示例
+python examples/01_basic_usage.py
+python examples/07_complete_workflow.py
+```
+
+详细说明请查看 [examples/README.md](examples/README.md)
+
+## 🧪 测试
+
+运行测试：
+
+```bash
+# 安装开发依赖
+pip install xiaoshiai-hub[dev]
+
+# 运行所有测试
+pytest
+
+# 运行特定测试文件
+pytest tests/test_client.py
+
+# 运行测试并显示覆盖率
+pytest --cov=xiaoshiai_hub --cov-report=html
+
+# 运行测试（详细输出）
+pytest -v
+```
+
+## 📖 API 文档
+
+### 核心函数
+
+#### `moha_hub_download()`
+
+下载单个文件。
+
+**参数：**
+- `repo_id` (str): 仓库 ID，格式为 "organization/repo_name"
+- `filename` (str): 要下载的文件名
+- `repo_type` (str): 仓库类型，"models" 或 "datasets"，默认 "models"
+- `revision` (str, optional): 分支、标签或提交哈希，默认 "main"
+- `local_dir` (str, optional): 本地保存目录
+- `username` (str, optional): 用户名
+- `password` (str, optional): 密码
+- `token` (str, optional): 认证 Token
+- `decryption_key` (str, optional): 解密密钥
+- `decryption_algorithm` (str, optional): 解密算法
+
+**返回：** 下载文件的本地路径
+
+#### `snapshot_download()`
+
+下载整个仓库。
+
+**参数：**
+- `repo_id` (str): 仓库 ID
+- `repo_type` (str): 仓库类型，默认 "models"
+- `revision` (str, optional): 分支、标签或提交哈希，默认 "main"
+- `local_dir` (str, optional): 本地保存目录
+- `allow_patterns` (List[str], optional): 允许下载的文件模式
+- `ignore_patterns` (List[str], optional): 忽略的文件模式
+- `username` (str, optional): 用户名
+- `password` (str, optional): 密码
+- `token` (str, optional): 认证 Token
+- `decryption_key` (str, optional): 解密密钥
+- `decryption_algorithm` (str, optional): 解密算法
+
+**返回：** 下载仓库的本地路径
+
+#### `upload_file()`
+
+上传单个文件。
+
+**参数：**
+- `path_or_fileobj` (str | Path | bytes): 文件路径或文件对象
+- `path_in_repo` (str): 仓库中的文件路径
+- `repo_id` (str): 仓库 ID
+- `repo_type` (str): 仓库类型，默认 "models"
+- `revision` (str): 分支名称，默认 "main"
+- `commit_message` (str, optional): 提交信息
+- `commit_description` (str, optional): 提交描述
+- `username` (str, optional): 用户名
+- `password` (str, optional): 密码
+- `token` (str, optional): 认证 Token
+- `encryption_key` (str, optional): 加密密钥
+- `encryption_algorithm` (str, optional): 加密算法
+
+**返回：** 提交哈希
+
+#### `upload_folder()`
+
+上传整个文件夹。
+
+**参数：**
+- `folder_path` (str | Path): 文件夹路径
+- `repo_id` (str): 仓库 ID
+- `repo_type` (str): 仓库类型，默认 "models"
+- `revision` (str): 分支名称，默认 "main"
+- `commit_message` (str, optional): 提交信息
+- `commit_description` (str, optional): 提交描述
+- `ignore_patterns` (List[str], optional): 忽略的文件模式
+- `username` (str, optional): 用户名
+- `password` (str, optional): 密码
+- `token` (str, optional): 认证 Token
+- `encryption_key` (str, optional): 加密密钥
+- `encryption_algorithm` (str, optional): 加密算法
+- `encryption_exclude` (List[str], optional): 不加密的文件模式
+
+**返回：** 提交哈希
+
+### HubClient 类
+
+#### 初始化
+
+```python
+client = HubClient(
+    base_url="https://hub.example.com/moha",
+    username="your-username",
+    password="your-password",
+)
+```
+
+#### 方法
+
+- `get_repository_info(org, repo_type, repo_name)` - 获取仓库信息
+- `list_branches(org, repo_type, repo_name)` - 列出分支
+- `list_tags(org, repo_type, repo_name)` - 列出标签
+- `get_repository_content(org, repo_type, repo_name, branch, path)` - 获取仓库内容
+- `download_file(org, repo_type, repo_name, branch, file_path, local_path)` - 下载文件
+- `get_moha_encryption(org, repo_type, repo_name, branch)` - 获取加密元数据
+
+## 🔧 开发
+
+### 设置开发环境
+
+```bash
+# 克隆仓库
+git clone https://github.com/poxiaoyun/XiaoShi-Moha.git
+cd XiaoShi-Moha/python-sdk
+
+# 创建虚拟环境
+python -m venv venv
+source venv/bin/activate  # Linux/macOS
+# 或
+venv\Scripts\activate  # Windows
+
+# 安装开发依赖
+pip install -e .[dev]
+```
+
+### 代码格式化
+
+```bash
+# 使用 black 格式化代码
+black xiaoshiai_hub tests examples
+
+# 使用 flake8 检查代码
+flake8 xiaoshiai_hub tests
+
+# 使用 mypy 检查类型
+mypy xiaoshiai_hub
+```
+
+### 构建和发布
+
+```bash
+# 构建包
+python -m build
+
+# 检查包
+twine check dist/*
+
+# 上传到 TestPyPI
+twine upload --repository testpypi dist/*
+
+# 上传到 PyPI
+twine upload dist/*
+```
+