synkro 0.4.30__tar.gz

synkro-0.4.30/.gitignore

@@ -0,0 +1,183 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# C extensions
+*.so
+
+# Distribution / packaging
+.Python
+build/
+develop-eggs/
+dist/
+downloads/
+eggs/
+.eggs/
+lib/
+lib64/
+parts/
+sdist/
+var/
+wheels/
+share/python-wheels/
+*.egg-info/
+.installed.cfg
+*.egg
+MANIFEST
+
+.github/workflows/
+
+# PyInstaller
+*.manifest
+*.spec
+
+# Installer logs
+pip-log.txt
+pip-delete-this-directory.txt
+
+# Unit test / coverage reports
+htmlcov/
+.tox/
+.nox/
+.coverage
+.coverage.*
+.cache
+nosetests.xml
+coverage.xml
+*.cover
+*.py,cover
+.hypothesis/
+.pytest_cache/
+cover/
+
+# Translations
+*.mo
+*.pot
+
+# Django stuff:
+*.log
+local_settings.py
+db.sqlite3
+db.sqlite3-journal
+
+# Flask stuff:
+instance/
+.webassets-cache
+
+# Scrapy stuff:
+.scrapy
+
+# Sphinx documentation
+docs/_build/
+
+# PyBuilder
+.pybuilder/
+target/
+
+# Jupyter Notebook
+.ipynb_checkpoints
+
+# IPython
+profile_default/
+ipython_config.py
+
+# pyenv
+.python-version
+
+# pipenv
+Pipfile.lock
+
+# poetry
+poetry.lock
+
+# pdm
+.pdm.toml
+
+# PEP 582
+__pypackages__/
+
+# Celery stuff
+celerybeat-schedule
+celerybeat.pid
+
+# SageMath parsed files
+*.sage.py
+
+# Environments
+.env
+.venv
+env/
+venv/
+ENV/
+env.bak/
+venv.bak/
+
+# Spyder project settings
+.spyderproject
+.spyproject
+
+# Rope project settings
+.ropeproject
+
+# mkdocs documentation
+/site
+
+# mypy
+.mypy_cache/
+.dmypy.json
+dmypy.json
+
+# Pyre type checker
+.pyre/
+
+# pytype static type analyzer
+.pytype/
+
+# Cython debug symbols
+cython_debug/
+
+# IDEs
+.vscode/
+.idea/
+*.swp
+*.swo
+*~
+.DS_Store
+
+# Project specific
+*.jsonl
+*.json
+!examples/*.json
+!tests/*.json
+output/
+outputs/
+training_data/
+*.pdf
+!examples/*.pdf
+*.docx
+!examples/*.docx
+
+# API keys and secrets
+.env
+.env.local
+.env.*.local
+*.key
+*.pem
+
+# Model outputs (but not synkro/models/ Python package)
+/models/
+checkpoints/
+*.pt
+*.pth
+*.ckpt
+
+# Logs
+*.log
+logs/
+
+# Temporary files
+tmp/
+temp/
+*.tmp
+

synkro-0.4.30/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Murtaza Meerza
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

synkro-0.4.30/PKG-INFO

@@ -0,0 +1,294 @@
+Metadata-Version: 2.4
+Name: synkro
+Version: 0.4.30
+Summary: Generate training datasets from any document
+Author: Murtaza Meerza
+License-Expression: MIT
+License-File: LICENSE
+Keywords: dataset-generation,fine-tuning,llm,synthetic-data,training-data
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
+Requires-Python: >=3.10
+Requires-Dist: beautifulsoup4>=4.12
+Requires-Dist: html2text>=2020.1
+Requires-Dist: httpx>=0.25
+Requires-Dist: litellm>=1.40
+Requires-Dist: mammoth>=1.6
+Requires-Dist: marker-pdf>=0.2
+Requires-Dist: pydantic>=2.0
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: rich>=13.0
+Requires-Dist: typer>=0.9
+Provides-Extra: dev
+Requires-Dist: pytest-asyncio>=0.21; extra == 'dev'
+Requires-Dist: pytest>=7.0; extra == 'dev'
+Requires-Dist: ruff>=0.1; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# Synkro
+
+Turn policies, handbooks, and documentation into high-quality training data for fine-tuning LLMs.
+
+## Features
+
+- **Quality Evaluation** - Each response is graded and automatically refined if it fails
+- **Multiple Formats** - Conversation (multi-turn), Instruction (single-turn), and Tool Calling
+- **Tool Call Training** - Generate OpenAI function calling format for teaching models to use custom tools
+- **Top LLM Providers** - OpenAI, Anthropic, and Google
+- **File Support** - PDF, DOCX, TXT, Markdown, URLs
+- **CLI Included** - Generate datasets from the command line
+
+## Installation
+
+```bash
+pip install synkro
+```
+
+## Quick Start
+
+```python
+from synkro.pipelines import create_pipeline
+from synkro.models.google import Google
+from synkro.types import DatasetType
+
+pipeline = create_pipeline(
+    model=Google.GEMINI_25_FLASH,          # Fast generation
+    grading_model=Google.GEMINI_25_PRO,    # Quality grading
+    dataset_type=DatasetType.CONVERSATION,
+)
+
+dataset = pipeline.generate(
+    "All expenses over $50 require manager approval.",
+    traces=50,
+)
+dataset.save("training.jsonl")
+```
+
+### From Files
+
+```python
+from synkro.pipelines import create_pipeline
+from synkro.core.policy import Policy
+
+policy = Policy.from_file("handbook.pdf")  # PDF, DOCX, TXT, MD
+pipeline = create_pipeline()
+dataset = pipeline.generate(policy, traces=100)
+dataset.save()
+```
+
+### From URLs
+
+```python
+from synkro.core.policy import Policy
+
+policy = Policy.from_url("https://example.com/terms")
+dataset = pipeline.generate(policy)
+```
+
+## Dataset Types
+
+| Type | Turns | Output Formats | Best For |
+|------|-------|----------------|----------|
+| **CONVERSATION** | Multi | messages | Fine-tuning chat models |
+| **INSTRUCTION** | 1 | messages | Instruction-following models |
+| **TOOL_CALL** | Multi | OpenAI function calling, ChatML | Teaching tool use |
+
+### Conversation (Default)
+
+```python
+from synkro.types import DatasetType
+
+pipeline = create_pipeline(dataset_type=DatasetType.CONVERSATION)
+dataset = pipeline.generate(policy)
+```
+
+Output (multi-turn):
+```json
+{"messages": [
+  {"role": "user", "content": "What's the approval process for $350?"},
+  {"role": "assistant", "content": "For a $350 expense, you need manager approval..."},
+  {"role": "user", "content": "What if my manager is unavailable?"},
+  {"role": "assistant", "content": "You can request approval from..."}
+]}
+```
+
+### Instruction
+
+```python
+pipeline = create_pipeline(dataset_type=DatasetType.INSTRUCTION)
+dataset = pipeline.generate(policy)
+```
+
+Output (single-turn):
+```json
+{"messages": [
+  {"role": "user", "content": "What's the approval process for $350?"},
+  {"role": "assistant", "content": "For a $350 expense, you need manager approval. Submit the expense report with receipt..."}
+]}
+```
+
+### Tool Calling
+
+Generate training data for teaching models when and how to use your custom tools:
+
+```python
+from synkro import create_pipeline, ToolDefinition, DatasetType
+
+# Define your tools
+web_search = ToolDefinition(
+    name="web_search",
+    description="Search the web for current information",
+    parameters={
+        "type": "object",
+        "properties": {
+            "query": {"type": "string", "description": "Search query"}
+        },
+        "required": ["query"]
+    },
+    mock_responses=["NYC: 72°F, sunny", "BTC: $67,234"]
+)
+
+# Create pipeline with tools
+pipeline = create_pipeline(
+    dataset_type=DatasetType.TOOL_CALL,
+    tools=[web_search],
+)
+
+# Generate from tool usage guidelines
+dataset = pipeline.generate("""
+Use web_search for real-time data like weather, prices.
+Answer general questions directly without tools.
+""", traces=20)
+
+dataset.save("tool_training.jsonl", format="tool_call")  # OpenAI format
+dataset.save("tool_training.jsonl", format="chatml")     # ChatML with XML tags
+```
+
+**Output Formats:**
+
+OpenAI function calling (`format="tool_call"`):
+```json
+{"messages": [
+  {"role": "user", "content": "What's the weather in NYC?"},
+  {"role": "assistant", "content": null, "tool_calls": [
+    {"id": "call_abc", "type": "function", "function": {"name": "web_search", "arguments": "{\"query\": \"weather NYC\"}"}}
+  ]},
+  {"role": "tool", "tool_call_id": "call_abc", "content": "NYC: 72°F, sunny"},
+  {"role": "assistant", "content": "The weather in NYC is 72°F and sunny."}
+]}
+```
+
+ChatML with XML tags (`format="chatml"`):
+```json
+{"messages": [
+  {"role": "user", "content": "What's the weather in NYC?"},
+  {"role": "assistant", "content": "<tool_call>\n{\"name\": \"web_search\", \"arguments\": {\"query\": \"weather NYC\"}}\n</tool_call>"},
+  {"role": "tool", "content": "<tool_response>\nNYC: 72°F, sunny\n</tool_response>"},
+  {"role": "assistant", "content": "The weather in NYC is 72°F and sunny."}
+]}
+```
+
+## Evaluation & Grading
+
+Every response is graded on policy compliance, citations, and reasoning. Failed responses are automatically refined (up to N iterations).
+
+```python
+from synkro.pipelines import create_pipeline
+from synkro.models.openai import OpenAI
+
+pipeline = create_pipeline(
+    model=OpenAI.GPT_4O_MINI,       # Fast generation
+    grading_model=OpenAI.GPT_4O,    # Quality grading
+    max_iterations=3,               # Refinement attempts
+)
+
+dataset = pipeline.generate(policy, traces=100)
+
+# Check quality
+print(f"Pass rate: {dataset.passing_rate:.1%}")
+
+# Filter to only passing traces
+high_quality = dataset.filter(passed=True)
+high_quality.save("training.jsonl")
+```
+
+## CLI
+
+```bash
+# From file
+synkro generate policy.pdf --traces 50
+
+# From text
+synkro generate "All expenses over $50 need approval" -n 20
+
+# From URL
+synkro generate https://example.com/policy -o training.jsonl
+
+# Skip interactive mode
+synkro generate policy.pdf --no-interactive
+```
+
+**Options:**
+- `--traces, -n` - Number of traces (default: 20)
+- `--output, -o` - Output file path
+- `--model, -m` - Model for generation
+- `--interactive/-i, --no-interactive/-I` - Review/edit extracted rules before generation (default: on)
+
+## Interactive Mode
+
+By default, synkro extracts policy rules into a Logic Map and lets you review/edit them before generation. The interactive session also shows the recommended conversation turns based on policy complexity:
+
+```
+╭─────────────────────────── Conversation Settings ────────────────────────────╮
+│  Complexity:  Conditional                                                    │
+│  Turns:       3                                                              │
+╰──────────────────────────────────────────────────────────────────────────────╯
+
+╭────────────────────────── 📜 Logic Map (3 rules) ────────────────────────────╮
+│ ├── R001: Expenses over $50 require manager approval                         │
+│ ├── R002: Client meals limited to $75/person                                 │
+│ └── R003: Receipts required for all expenses                                 │
+╰──────────────────────────────────────────────────────────────────────────────╯
+
+Enter feedback: shorter conversations
+✓ Set to 2 turns (User requested shorter/simpler conversations)
+
+Enter feedback: add a rule for travel expenses
+✓ Added R004: Travel expenses over $500 require VP approval
+
+Enter feedback: done
+✅ Session complete - 1 rule change(s), 2 turns
+```
+
+You can adjust both **conversation turns** and **rules** using natural language:
+
+| Input | Action |
+|-------|--------|
+| `"shorter conversations"` | Reduce turns (1-2) |
+| `"I want 5 turns"` | Set specific turn count |
+| `"more thorough"` | Increase turns (5-6) |
+| `"remove R002"` | Delete a rule |
+| `"add a rule for..."` | Add new rule |
+
+Commands: `done`, `undo`, `reset`, `show R001`, `help`
+
+## Logic Map Inspection
+
+Access the extracted rules programmatically:
+
+```python
+result = pipeline.generate(policy, traces=50, return_logic_map=True)
+
+# Inspect extracted rules
+for rule in result.logic_map.rules:
+    print(f"{rule.rule_id}: {rule.text}")
+
+# Get the dataset
+dataset = result.dataset
+```