synkro 0.1.4__tar.gz

synkro-0.1.4/.gitignore

@@ -0,0 +1,183 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+.github/workflows/
+
+# PyInstaller
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+Pipfile.lock
+
+# poetry
+poetry.lock
+
+# pdm
+.pdm.toml
+
+# PEP 582
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# IDEs
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+.DS_Store
+
+# Project specific
+*.jsonl
+*.json
+!examples/*.json
+!tests/*.json
+output/
+outputs/
+training_data/
+*.pdf
+!examples/*.pdf
+*.docx
+!examples/*.docx
+
+# API keys and secrets
+.env
+.env.local
+.env.*.local
+*.key
+*.pem
+
+# Model outputs (but not synkro/models/ Python package)
+/models/
+checkpoints/
+*.pt
+*.pth
+*.ckpt
+
+# Logs
+*.log
+logs/
+
+# Temporary files
+tmp/
+temp/
+*.tmp
+

synkro-0.1.4/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Murtaza Meerza
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

synkro-0.1.4/MANIFEST.in

@@ -0,0 +1,3 @@
+include README.md
+include LICENSE
+recursive-include examples *.py

synkro-0.1.4/PKG-INFO

@@ -0,0 +1,308 @@
+Metadata-Version: 2.4
+Name: synkro
+Version: 0.1.4
+Summary: Generate training datasets from any document
+Author: Murtaza Meerza
+License-Expression: MIT
+License-File: LICENSE
+Keywords: dataset-generation,fine-tuning,llm,synthetic-data,training-data
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Requires-Dist: beautifulsoup4>=4.12
+Requires-Dist: html2text>=2020.1
+Requires-Dist: httpx>=0.25
+Requires-Dist: instructor>=1.0
+Requires-Dist: litellm>=1.40
+Requires-Dist: mammoth>=1.6
+Requires-Dist: marker-pdf>=0.2
+Requires-Dist: pydantic>=2.0
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: rich>=13.0
+Requires-Dist: typer>=0.9
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Requires-Dist: ruff>=0.1; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# Synkro
+
+**Generate training datasets from any document.**
+
+Turn policies, handbooks, and documentation into high-quality training data for fine-tuning LLMs.
+
+## Features
+
+- **Quality Evaluation** - Each response is graded and automatically refined if it fails
+- **Multiple Formats** - SFT (chat), QA (question-answer), DPO (preference pairs)
+- **Any LLM Provider** - OpenAI, Anthropic, Google, Ollama, Groq, and more
+- **File Support** - PDF, DOCX, TXT, Markdown, URLs
+- **CLI Included** - Generate datasets from the command line
+
+## Installation
+
+```bash
+pip install synkro
+```
+
+## Quick Start
+
+```python
+from synkro.pipelines import create_pipeline
+from synkro.models.google import Google
+from synkro.types import DatasetType
+
+pipeline = create_pipeline(
+    model=Google.GEMINI_25_FLASH,          # Fast generation
+    grading_model=Google.GEMINI_25_PRO,    # Quality grading
+    dataset_type=DatasetType.SFT,
+)
+
+dataset = pipeline.generate(
+    "All expenses over $50 require manager approval.",
+    traces=50,
+)
+dataset.save("training.jsonl")
+```
+
+### From Files
+
+```python
+from synkro.pipelines import create_pipeline
+from synkro.core.policy import Policy
+
+policy = Policy.from_file("handbook.pdf")  # PDF, DOCX, TXT, MD
+pipeline = create_pipeline()
+dataset = pipeline.generate(policy, traces=100)
+dataset.save()
+```
+
+### From URLs
+
+```python
+from synkro.core.policy import Policy
+
+policy = Policy.from_url("https://example.com/terms")
+dataset = pipeline.generate(policy)
+```
+
+## Dataset Types
+
+| Format | Output | Best For |
+|--------|--------|----------|
+| **SFT** | Chat messages | Fine-tuning chat models |
+| **QA** | Question-answer pairs | RAG systems, knowledge bases |
+| **DPO** | Preference pairs | RLHF, alignment training |
+
+### SFT (Default)
+
+```python
+from synkro.types import DatasetType
+
+pipeline = create_pipeline(dataset_type=DatasetType.SFT)
+dataset = pipeline.generate(policy)
+```
+
+Output:
+```json
+{"messages": [
+  {"role": "system", "content": "You are a policy expert..."},
+  {"role": "user", "content": "What's the approval process for $350?"},
+  {"role": "assistant", "content": "For a $350 expense, you need..."}
+]}
+```
+
+### QA
+
+```python
+pipeline = create_pipeline(dataset_type=DatasetType.QA)
+```
+
+Output:
+```json
+{"question": "What's the approval process?", "answer": "You need...", "context": "..."}
+```
+
+### DPO
+
+```python
+pipeline = create_pipeline(dataset_type=DatasetType.DPO)
+```
+
+Output:
+```json
+{"prompt": "What's the process?", "chosen": "Good answer...", "rejected": "Bad answer..."}
+```
+
+## Evaluation & Grading
+
+Every response is graded on policy compliance, citations, and reasoning. Failed responses are automatically refined (up to 3 iterations).
+
+```python
+from synkro.pipelines import create_pipeline
+from synkro.models.openai import OpenAI
+
+pipeline = create_pipeline(
+    model=OpenAI.GPT_4O_MINI,       # Fast generation
+    grading_model=OpenAI.GPT_4O,    # Quality grading
+    max_iterations=3,               # Refinement attempts
+)
+
+dataset = pipeline.generate(policy, traces=100)
+
+# Check quality
+print(f"Pass rate: {dataset.passing_rate:.1%}")
+
+# Filter to only passing traces
+high_quality = dataset.filter(passed=True)
+high_quality.save("training.jsonl")
+```
+
+### Custom Graders
+
+```python
+from synkro.models.anthropic import Anthropic
+from synkro.models.ollama import Ollama
+
+# Use Claude for grading
+pipeline = create_pipeline(grading_model=Anthropic.CLAUDE_35_SONNET)
+
+# Or free local grading
+pipeline = create_pipeline(grading_model=Ollama.QWEN_25_32B)
+```
+
+## Models & Providers
+
+### OpenAI
+
+```python
+from synkro.models.openai import OpenAI
+
+pipeline = create_pipeline(model=OpenAI.GPT_4O_MINI)
+```
+
+| Model | Use Case |
+|-------|----------|
+| `OpenAI.GPT_4O` | Best quality (grading) |
+| `OpenAI.GPT_4O_MINI` | Fast & cheap (generation) |
+| `OpenAI.O1` | Reasoning tasks |
+
+**Env:** `OPENAI_API_KEY`
+
+### Anthropic
+
+```python
+from synkro.models.anthropic import Anthropic
+
+pipeline = create_pipeline(model=Anthropic.CLAUDE_35_HAIKU)
+```
+
+| Model | Use Case |
+|-------|----------|
+| `Anthropic.CLAUDE_35_SONNET` | High quality |
+| `Anthropic.CLAUDE_35_HAIKU` | Fast & cheap |
+
+**Env:** `ANTHROPIC_API_KEY`
+
+### Google
+
+```python
+from synkro.models.google import Google
+
+pipeline = create_pipeline(model=Google.GEMINI_25_FLASH)
+```
+
+| Model | Use Case |
+|-------|----------|
+| `Google.GEMINI_3_PRO` | Most intelligent |
+| `Google.GEMINI_25_FLASH` | Best price-performance |
+| `Google.GEMINI_2_FLASH_LITE` | Cheapest |
+
+**Env:** `GEMINI_API_KEY`
+
+### Ollama (Free, Local)
+
+```python
+from synkro.models.ollama import Ollama
+
+# First: ollama pull llama3.1:8b
+pipeline = create_pipeline(model=Ollama.LLAMA_31_8B)
+```
+
+| Model | Use Case |
+|-------|----------|
+| `Ollama.LLAMA_31_8B` | Fast local |
+| `Ollama.QWEN_25_32B` | High quality local |
+
+**No API key needed**
+
+### Groq (Ultra Fast)
+
+```python
+from synkro.models.groq import Groq
+
+pipeline = create_pipeline(model=Groq.LLAMA_33_70B)
+```
+
+| Model | Use Case |
+|-------|----------|
+| `Groq.LLAMA_33_70B` | Fastest inference |
+| `Groq.LLAMA_31_8B` | Budget option |
+
+**Env:** `GROQ_API_KEY`
+
+### Model Selection Tips
+
+**For Generation (fast, cheap):**
+- `OpenAI.GPT_4O_MINI` - Best balance
+- `Groq.LLAMA_33_70B` - Ultra fast
+- `Ollama.LLAMA_31_8B` - Free
+
+**For Grading (high quality):**
+- `OpenAI.GPT_4O` - Best quality
+- `Anthropic.CLAUDE_35_SONNET` - Great alternative
+- `Google.GEMINI_3_PRO` - Most intelligent
+
+## CLI
+
+### Generate
+
+```bash
+# From file
+synkro generate policy.pdf --traces 50 --format sft
+
+# From text
+synkro generate "All expenses over $50 need approval" -n 20
+
+# From URL
+synkro generate https://example.com/policy -o training.jsonl
+```
+
+**Options:**
+- `--traces, -n` - Number of traces (default: 20)
+- `--format, -f` - Output format: sft, qa, dpo (default: sft)
+- `--output, -o` - Output file path
+- `--model, -m` - Model for generation
+
+### Demo
+
+```bash
+synkro demo  # Quick demo with example policy
+```
+
+### Version
+
+```bash
+synkro version
+```
+
+## License
+
+MIT