mimic-kit 0.1.0__tar.gz

mimic_kit-0.1.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Mimic-Kit Contributors
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

mimic_kit-0.1.0/PKG-INFO

@@ -0,0 +1,149 @@
+Metadata-Version: 2.4
+Name: mimic-kit
+Version: 0.1.0
+Summary: Add your description here
+Requires-Python: >=3.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click>=8.0.0
+Requires-Dist: deepspeed>=0.18.7
+Requires-Dist: diskcache>=5.6.3
+Requires-Dist: liger-kernel>=0.7.0
+Requires-Dist: mpi4py>=4.1.1
+Requires-Dist: ms-swift>=4.0.1
+Requires-Dist: openai>=1.0.0
+Requires-Dist: peft>=0.6.2
+Requires-Dist: pydantic>=2.12.5
+Requires-Dist: pyyaml>=6.0.3
+Requires-Dist: torch>=2.10.0
+Requires-Dist: transformers[torch]>=5.2.0
+Dynamic: license-file
+
+# Mimic-Kit
+
+[![Python 3.13+](https://img.shields.io/badge/python-3.13+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
+[中文文档](README-ZH.md) | **English**
+
+Mimic-Kit is a **black-box knowledge distillation scaffolding** that helps you distill capabilities from powerful teacher models into smaller, more efficient student models.
+
+## Features
+
+- 🤖 **Teacher Model Generation**: Use OpenAI API (or compatible APIs) to generate high-quality training data
+- 🚀 **Efficient Training**: Built on [ms-swift](https://github.com/modelscope/swift) for streamlined model training
+- 🧩 **Flexible Tuning**: Support both LoRA (parameter-efficient) and full fine-tuning
+- ⚡ **Performance Optimized**: Integrated with DeepSpeed and Liger Kernel for accelerated training
+- 💾 **Smart Caching**: Automatically cache API responses to save costs
+- 📊 **Multiple Data Formats**: Support both chat and text completion formats
+
+## Quick Start
+
+### 1. Installation
+
+```bash
+# Clone the repository
+git clone <repository-url>
+cd mimic-kit
+
+# Install dependencies using UV
+uv sync
+
+# Or with dev dependencies
+uv sync --group dev
+```
+
+### 2. Initialize Configuration
+
+```bash
+uv run mimic init
+```
+
+This creates a `config.yaml` template. Edit it to configure your teacher model, student model, and training parameters.
+
+### 3. Prepare Your Data
+
+Create a JSONL file with your prompts. Supports two formats:
+
+**Chat Format:**
+```jsonl
+{"messages": [{"role": "user", "content": "Explain Python decorators"}]}
+{"messages": [{"role": "user", "content": "How to reverse a linked list?"}]}
+```
+
+**Text Completion Format:**
+```jsonl
+{"text": "Python decorators are a powerful feature..."}
+{"text": "To reverse a linked list, you need to..."}
+```
+
+### 4. Generate Training Data
+
+```bash
+uv run mimic generate
+```
+
+This sends your prompts to the teacher model and saves the generated responses as training data.
+
+### 5. Train Your Student Model
+
+```bash
+uv run mimic train
+```
+
+The student model will be fine-tuned on the generated data using the configured method (LoRA or full fine-tuning).
+
+## Configuration
+
+See `config.yaml` for all available options. Key sections:
+
+- `data`: Input/output paths, system prompts, templates
+- `teacher`: API provider, model, generation parameters
+- `student`: Base model, tuning method, hyperparameters
+- `training`: Batch size, learning rate, saving strategy
+
+## Project Structure
+
+```
+mimic-kit/
+├── mimic/
+│   ├── cli.py              # CLI entry point
+│   ├── config.py           # Configuration models (Pydantic v2)
+│   ├── generator/          # Teacher model data generation
+│   └── trainer/            # Student model training (ms-swift)
+├── data/                   # Training data directory
+├── config.yaml             # Your configuration file
+└── output/                 # Model outputs and checkpoints
+```
+
+## Requirements
+
+- Python 3.13+
+- CUDA-capable GPU (for training)
+- OpenAI API key (or compatible API)
+
+## Development
+
+```bash
+# Run tests
+uv run pytest
+
+# Run single test
+uv run pytest tests/test_cli.py::test_function
+
+# Format code
+uv run ruff format .
+
+# Check linting
+uv run ruff check .
+```
+
+## License
+
+MIT License - see [LICENSE](LICENSE) file for details.
+
+## Acknowledgments
+
+- Built with [ms-swift](https://github.com/modelscope/swift) for model training
+- Uses [Pydantic](https://docs.pydantic.dev/) for configuration validation
+- Powered by [Click](https://click.palletsprojects.com/) for CLI

mimic_kit-0.1.0/README.md

@@ -0,0 +1,128 @@
+# Mimic-Kit
+
+[![Python 3.13+](https://img.shields.io/badge/python-3.13+-blue.svg)](https://www.python.org/downloads/)
+[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
+
+[中文文档](README-ZH.md) | **English**
+
+Mimic-Kit is a **black-box knowledge distillation scaffolding** that helps you distill capabilities from powerful teacher models into smaller, more efficient student models.
+
+## Features
+
+- 🤖 **Teacher Model Generation**: Use OpenAI API (or compatible APIs) to generate high-quality training data
+- 🚀 **Efficient Training**: Built on [ms-swift](https://github.com/modelscope/swift) for streamlined model training
+- 🧩 **Flexible Tuning**: Support both LoRA (parameter-efficient) and full fine-tuning
+- ⚡ **Performance Optimized**: Integrated with DeepSpeed and Liger Kernel for accelerated training
+- 💾 **Smart Caching**: Automatically cache API responses to save costs
+- 📊 **Multiple Data Formats**: Support both chat and text completion formats
+
+## Quick Start
+
+### 1. Installation
+
+```bash
+# Clone the repository
+git clone <repository-url>
+cd mimic-kit
+
+# Install dependencies using UV
+uv sync
+
+# Or with dev dependencies
+uv sync --group dev
+```
+
+### 2. Initialize Configuration
+
+```bash
+uv run mimic init
+```
+
+This creates a `config.yaml` template. Edit it to configure your teacher model, student model, and training parameters.
+
+### 3. Prepare Your Data
+
+Create a JSONL file with your prompts. Supports two formats:
+
+**Chat Format:**
+```jsonl
+{"messages": [{"role": "user", "content": "Explain Python decorators"}]}
+{"messages": [{"role": "user", "content": "How to reverse a linked list?"}]}
+```
+
+**Text Completion Format:**
+```jsonl
+{"text": "Python decorators are a powerful feature..."}
+{"text": "To reverse a linked list, you need to..."}
+```
+
+### 4. Generate Training Data
+
+```bash
+uv run mimic generate
+```
+
+This sends your prompts to the teacher model and saves the generated responses as training data.
+
+### 5. Train Your Student Model
+
+```bash
+uv run mimic train
+```
+
+The student model will be fine-tuned on the generated data using the configured method (LoRA or full fine-tuning).
+
+## Configuration
+
+See `config.yaml` for all available options. Key sections:
+
+- `data`: Input/output paths, system prompts, templates
+- `teacher`: API provider, model, generation parameters
+- `student`: Base model, tuning method, hyperparameters
+- `training`: Batch size, learning rate, saving strategy
+
+## Project Structure
+
+```
+mimic-kit/
+├── mimic/
+│   ├── cli.py              # CLI entry point
+│   ├── config.py           # Configuration models (Pydantic v2)
+│   ├── generator/          # Teacher model data generation
+│   └── trainer/            # Student model training (ms-swift)
+├── data/                   # Training data directory
+├── config.yaml             # Your configuration file
+└── output/                 # Model outputs and checkpoints
+```
+
+## Requirements
+
+- Python 3.13+
+- CUDA-capable GPU (for training)
+- OpenAI API key (or compatible API)
+
+## Development
+
+```bash
+# Run tests
+uv run pytest
+
+# Run single test
+uv run pytest tests/test_cli.py::test_function
+
+# Format code
+uv run ruff format .
+
+# Check linting
+uv run ruff check .
+```
+
+## License
+
+MIT License - see [LICENSE](LICENSE) file for details.
+
+## Acknowledgments
+
+- Built with [ms-swift](https://github.com/modelscope/swift) for model training
+- Uses [Pydantic](https://docs.pydantic.dev/) for configuration validation
+- Powered by [Click](https://click.palletsprojects.com/) for CLI

mimic_kit-0.1.0/mimic/__init__.py

@@ -0,0 +1,3 @@
+"""Mimic - A command line tool"""
+
+__version__ = "0.1.0"

mimic_kit-0.1.0/mimic/cli.py

@@ -0,0 +1,115 @@
+"""CLI commands for mimic."""
+
+from pathlib import Path
+
+import click
+from mimic import __version__
+from mimic.config import load_config
+from mimic.generator import generate_dataset, save_dataset
+from mimic.templates import write_config_template
+from mimic.trainer import train_with_swift
+
+
+@click.group()
+@click.version_option(version=__version__)
+def cli():
+    """Mimic - A command line tool to distill models"""
+    pass
+
+
+@cli.command()
+@click.option(
+    "--output",
+    "-o",
+    default="config.yaml",
+    help="Output file path (default: config.yaml)",
+)
+@click.option("--force", "-f", is_flag=True, help="Overwrite existing file")
+def init(output, force):
+    """Initialize a new config.yaml template file"""
+    try:
+        write_config_template(output, force=force)
+        click.echo(f"Created config template: {output}")
+    except FileExistsError as e:
+        click.echo(f"Error: {e}. Use --force to overwrite.", err=True)
+        raise click.Abort()
+
+
+@cli.command()
+@click.option(
+    "--config",
+    "-c",
+    default="config.yaml",
+    help="Path to configuration file (default: config.yaml)",
+)
+def generate(config):
+    """Generate training data using teacher model"""
+    try:
+        # Load configuration
+        cfg = load_config(config)
+        click.echo(f"Loaded configuration from {config}")
+
+        # Generate dataset
+        data = generate_dataset(cfg)
+
+        # Save results
+        if data:
+            save_dataset(data, cfg.data.dataset_path)
+        else:
+            click.echo("Warning: No data was generated", err=True)
+            raise click.Abort()
+
+    except FileNotFoundError as e:
+        click.echo(f"Error: {e}", err=True)
+        raise click.Abort()
+    except Exception as e:
+        click.echo(f"Error: {e}", err=True)
+        raise click.Abort()
+
+
+@cli.command()
+@click.option(
+    "--config",
+    "-c",
+    default="config.yaml",
+    help="Path to configuration file (default: config.yaml)",
+)
+def train(config):
+    """Train student model using ms-swift"""
+    try:
+        # Load configuration
+        cfg = load_config(config)
+        click.echo(f"Loaded configuration from {config}")
+
+        # Check if dataset exists
+        dataset_path = Path(cfg.data.dataset_path)
+        if not dataset_path.exists():
+            click.echo(
+                f"Error: Dataset not found at {dataset_path}. "
+                "Run 'mimic generate' first.",
+                err=True,
+            )
+            raise click.Abort()
+
+        click.echo(f"Starting training with ms-swift...")
+        click.echo(f"Base model: {cfg.student.base_model}")
+        click.echo(f"Dataset: {cfg.data.dataset_path}")
+
+        # Run training
+        train_with_swift(cfg)
+
+        click.echo("Training completed successfully!")
+
+    except FileNotFoundError as e:
+        click.echo(f"Error: {e}", err=True)
+        raise click.Abort()
+    except RuntimeError as e:
+        click.echo(f"Training error: {e}", err=True)
+        raise click.Abort()
+    except Exception as e:
+        click.echo(f"Error: {e}", err=True)
+        raise click.Abort()
+
+
+if __name__ == "__main__":
+    cli()