synth-ai 0.2.13.dev2__tar.gz → 0.2.16__tar.gz

synth_ai-0.2.16/PKG-INFO

@@ -0,0 +1,193 @@
+Metadata-Version: 2.4
+Name: synth-ai
+Version: 0.2.16
+Summary: RL as a service SDK - Core AI functionality and tracing
+Author-email: Synth AI <josh@usesynth.ai>
+License-Expression: MIT
+Project-URL: Homepage, https://github.com/synth-laboratories/synth-ai
+Project-URL: Repository, https://github.com/synth-laboratories/synth-ai
+Project-URL: Issues, https://github.com/synth-laboratories/synth-ai/issues
+Requires-Python: >=3.11
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: python-dotenv>=1.0.1
+Requires-Dist: requests>=2.32.3
+Requires-Dist: urllib3>=2.3.0
+Requires-Dist: certifi>=2024.8.30
+Requires-Dist: tqdm>=4.66.4
+Requires-Dist: jsonschema>=4.23.0
+Requires-Dist: backoff>=2.0.0
+Requires-Dist: typing_extensions>=4.0.0
+Requires-Dist: openai>=1.99.0
+Requires-Dist: anthropic>=0.42.0
+Requires-Dist: langfuse<3.0.0,>=2.53.9
+Requires-Dist: opentelemetry-api<1.27.0,>=1.26.0
+Requires-Dist: opentelemetry-sdk<1.27.0,>=1.26.0
+Requires-Dist: diskcache>=5.6.3
+Requires-Dist: groq>=0.30.0
+Requires-Dist: google-genai>=1.26.0
+Requires-Dist: together>=1.5.21
+Requires-Dist: mistralai>=1.9.2
+Requires-Dist: fastapi>=0.115.12
+Requires-Dist: uvicorn>=0.34.2
+Requires-Dist: numpy>=2.2.3
+Requires-Dist: networkx>=3.4.2
+Requires-Dist: redis>=6.2.0
+Requires-Dist: duckdb>=1.0.0
+Requires-Dist: ty>=0.0.1a5
+Requires-Dist: toml>=0.10.2
+Requires-Dist: sqlalchemy>=2.0.42
+Requires-Dist: aiosqlite>=0.21.0
+Requires-Dist: greenlet>=3.2.3
+Requires-Dist: libsql>=0.1.8
+Requires-Dist: pynacl>=1.5.0
+Requires-Dist: google-api-core>=2.25.1
+Requires-Dist: google-generativeai>=0.8.5
+Requires-Dist: crafter>=1.8.3
+Requires-Dist: click<8.2,>=8.1.7
+Requires-Dist: textual>=1.1.0
+Requires-Dist: openai-harmony>=0.0.1
+Requires-Dist: asyncpg>=0.30.0
+Requires-Dist: aiohttp>=3.8.0
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: datasets>=4.0.0
+Requires-Dist: transformers>=4.56.1
+Requires-Dist: modal==1.1.4
+Requires-Dist: pyboy>=2.6.0
+Requires-Dist: setuptools>=80.9.0
+Provides-Extra: dev
+Requires-Dist: build>=1.2.2.post1; extra == "dev"
+Requires-Dist: twine>=4.0.0; extra == "dev"
+Requires-Dist: keyring>=24.0.0; extra == "dev"
+Requires-Dist: pytest>=8.3.3; extra == "dev"
+Requires-Dist: pytest-xdist>=3.6.1; extra == "dev"
+Requires-Dist: pytest-timeout>=2.3.1; extra == "dev"
+Requires-Dist: pytest-asyncio>=0.24.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.1.0; extra == "dev"
+Requires-Dist: pyright>=1.1.350; extra == "dev"
+Requires-Dist: coverage[toml]>=7.3.0; extra == "dev"
+Requires-Dist: ruff>=0.1.0; extra == "dev"
+Provides-Extra: research
+Requires-Dist: crafter>=1.8.3; extra == "research"
+Requires-Dist: datasets>=4.0.0; extra == "research"
+Provides-Extra: all
+Requires-Dist: crafter>=1.8.3; extra == "all"
+Requires-Dist: datasets>=4.0.0; extra == "all"
+Provides-Extra: analytics
+Requires-Dist: pandas>=2.2.3; extra == "analytics"
+Dynamic: license-file
+
+# Synth-AI SDK
+
+[![Python](https://img.shields.io/badge/python-3.11+-blue)](https://www.python.org/)
+[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+[![PyPI](https://img.shields.io/badge/PyPI-0.2.10-orange)](https://pypi.org/project/synth-ai/)
+![Coverage](https://img.shields.io/badge/coverage-9.09%25-red)
+![Tests](https://img.shields.io/badge/tests-37%2F38%20passing-brightgreen)
+![Blacksmith CI](https://img.shields.io/badge/CI-Blacksmith%20Worker-blue)
+
+> **Synth-AI** — Reinforcement Learning-as-a-Service for agents.  
+> **Docs:** [Get Started →](https://docs.usesynth.ai/sdk/get-started)
+
+---
+
+## 🚀 Install
+
+```bash
+pip install synth-ai
+# or
+uv add synth-ai
+```
+
+**Import:**
+
+```python
+import synth_ai
+```
+
+**CLI (with uvx):**
+
+```bash
+uvx synth-ai setup
+uvx synth-ai demo
+uvx synth-ai deploy
+uvx synth-ai run
+```
+
+> Full quickstart: [https://docs.usesynth.ai/sdk/get-started](https://docs.usesynth.ai/sdk/get-started)
+
+---
+
+When you run `uvx synth-ai setup`, the SDK opens your browser to the Synth dashboard for a one‑time pairing (handshake) with your signed‑in session. The SDK will automatically:
+
+
+Fast and effective reinforcement learning for agents, via an API.  
+Easily scale GPU topologies, train multi-node, and integrate with existing agent software.
+
+### Highlights
+
+- Scale GPU topologies (A10Gs, H100s, multi-node available on request)
+- Thin FastAPI wrapper integration
+- Supports OSS models like **Qwen3** (GPT-OSS GA soon)
+- Own your trained models
+
+---
+
+## ⚙️ Getting Started
+
+Synth-AI ships with a built-in RL example: training **Qwen3-0.6B** on math reasoning.
+
+1. Create accounts at [Synth](https://usesynth.ai) and [Modal](https://modal.com)
+2. Then run:
+
+   ```bash
+   uvx synth-ai demo
+   uvx synth-ai setup
+   uvx synth-ai deploy
+   uvx synth-ai run
+   ```
+
+3. To walk through your first RL run, see  
+   👉 [Synth-AI SDK Docs](https://docs.usesynth.ai/sdk/get-started)
+
+---
+
+## 🔐 SDK → Dashboard Pairing
+
+When you run `uvx synth-ai setup` (or legacy `uvx synth-ai rl_demo setup`):
+
+- The SDK opens your browser to the Synth dashboard to pair your SDK with your signed-in session.
+- Automatically detects your **user + organization**
+- Ensures both **API keys** exist
+- Writes them to your project’s `.env` as:
+
+  ```
+  SYNTH_API_KEY=
+  ENVIRONMENT_API_KEY=
+  ```
+
+✅ No keys printed or requested interactively — all handled via browser pairing.
+
+### Environment overrides
+
+- `SYNTH_CANONICAL_ORIGIN` → override dashboard base URL (default: https://www.usesynth.ai/dashboard)
+- `SYNTH_CANONICAL_DEV` → `1|true|on` to use local dashboard (http://localhost:3000)
+
+---
+
+## 📚 Documentation
+
+- **SDK Docs:** [https://docs.usesynth.ai/sdk/get-started](https://docs.usesynth.ai/sdk/get-started)
+- **CLI Reference:** [https://docs.usesynth.ai/cli](https://docs.usesynth.ai/cli)
+- **API Reference:** [https://docs.usesynth.ai/api](https://docs.usesynth.ai/api)
+- **Changelog:** [https://docs.usesynth.ai/changelog](https://docs.usesynth.ai/changelog)
+
+---
+
+## 🧠 Meta
+
+- Package: [`synth-ai`](https://pypi.org/project/synth-ai)
+- Import: `synth_ai`
+- Source: [github.com/synth-laboratories/synth-ai](https://github.com/synth-laboratories/synth-ai)
+- License: MIT

synth_ai-0.2.16/README.md

@@ -0,0 +1,113 @@
+# Synth-AI SDK
+
+[![Python](https://img.shields.io/badge/python-3.11+-blue)](https://www.python.org/)
+[![License](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+[![PyPI](https://img.shields.io/badge/PyPI-0.2.10-orange)](https://pypi.org/project/synth-ai/)
+![Coverage](https://img.shields.io/badge/coverage-9.09%25-red)
+![Tests](https://img.shields.io/badge/tests-37%2F38%20passing-brightgreen)
+![Blacksmith CI](https://img.shields.io/badge/CI-Blacksmith%20Worker-blue)
+
+> **Synth-AI** — Reinforcement Learning-as-a-Service for agents.  
+> **Docs:** [Get Started →](https://docs.usesynth.ai/sdk/get-started)
+
+---
+
+## 🚀 Install
+
+```bash
+pip install synth-ai
+# or
+uv add synth-ai
+```
+
+**Import:**
+
+```python
+import synth_ai
+```
+
+**CLI (with uvx):**
+
+```bash
+uvx synth-ai setup
+uvx synth-ai demo
+uvx synth-ai deploy
+uvx synth-ai run
+```
+
+> Full quickstart: [https://docs.usesynth.ai/sdk/get-started](https://docs.usesynth.ai/sdk/get-started)
+
+---
+
+When you run `uvx synth-ai setup`, the SDK opens your browser to the Synth dashboard for a one‑time pairing (handshake) with your signed‑in session. The SDK will automatically:
+
+
+Fast and effective reinforcement learning for agents, via an API.  
+Easily scale GPU topologies, train multi-node, and integrate with existing agent software.
+
+### Highlights
+
+- Scale GPU topologies (A10Gs, H100s, multi-node available on request)
+- Thin FastAPI wrapper integration
+- Supports OSS models like **Qwen3** (GPT-OSS GA soon)
+- Own your trained models
+
+---
+
+## ⚙️ Getting Started
+
+Synth-AI ships with a built-in RL example: training **Qwen3-0.6B** on math reasoning.
+
+1. Create accounts at [Synth](https://usesynth.ai) and [Modal](https://modal.com)
+2. Then run:
+
+   ```bash
+   uvx synth-ai demo
+   uvx synth-ai setup
+   uvx synth-ai deploy
+   uvx synth-ai run
+   ```
+
+3. To walk through your first RL run, see  
+   👉 [Synth-AI SDK Docs](https://docs.usesynth.ai/sdk/get-started)
+
+---
+
+## 🔐 SDK → Dashboard Pairing
+
+When you run `uvx synth-ai setup` (or legacy `uvx synth-ai rl_demo setup`):
+
+- The SDK opens your browser to the Synth dashboard to pair your SDK with your signed-in session.
+- Automatically detects your **user + organization**
+- Ensures both **API keys** exist
+- Writes them to your project’s `.env` as:
+
+  ```
+  SYNTH_API_KEY=
+  ENVIRONMENT_API_KEY=
+  ```
+
+✅ No keys printed or requested interactively — all handled via browser pairing.
+
+### Environment overrides
+
+- `SYNTH_CANONICAL_ORIGIN` → override dashboard base URL (default: https://www.usesynth.ai/dashboard)
+- `SYNTH_CANONICAL_DEV` → `1|true|on` to use local dashboard (http://localhost:3000)
+
+---
+
+## 📚 Documentation
+
+- **SDK Docs:** [https://docs.usesynth.ai/sdk/get-started](https://docs.usesynth.ai/sdk/get-started)
+- **CLI Reference:** [https://docs.usesynth.ai/cli](https://docs.usesynth.ai/cli)
+- **API Reference:** [https://docs.usesynth.ai/api](https://docs.usesynth.ai/api)
+- **Changelog:** [https://docs.usesynth.ai/changelog](https://docs.usesynth.ai/changelog)
+
+---
+
+## 🧠 Meta
+
+- Package: [`synth-ai`](https://pypi.org/project/synth-ai)
+- Import: `synth_ai`
+- Source: [github.com/synth-laboratories/synth-ai](https://github.com/synth-laboratories/synth-ai)
+- License: MIT

synth_ai-0.2.16/examples/README.md

@@ -0,0 +1 @@
+### The instructions for how to create and configure a task app are documented at https://docs.usesynth.ai/sdk/task-apps

synth_ai-0.2.16/examples/multi_step/SFT_README.md

@@ -0,0 +1,147 @@
+# SFT Training for Qwen3-Coder-30B with LoRA
+
+Supervised Fine-Tuning configuration for the same 30B MoE model used in RL training.
+
+## Configuration Overview
+
+**Model:** `Qwen/Qwen3-Coder-30B-A3B-Instruct` (Mixture of Experts)
+
+**Hardware:** 4x H200 GPUs (561GB total VRAM)
+
+**Parallelism Strategy:**
+- **Tensor Parallel (TP)**: 2 GPUs - Splits the model across 2 GPUs for inference/forward pass
+- **Data Parallel (DP)**: 2 GPUs - Splits batches across 2 GPUs for training throughput
+
+**LoRA Configuration:**
+- Rank (r): 16
+- Alpha: 32
+- Dropout: 0.05
+- Target modules: `["all-linear"]` - Applies LoRA to all linear layers
+
+## Memory Breakdown per GPU
+
+With 4x H200 (141GB each):
+
+**Model Split (TP=2):**
+- 2 GPUs hold the base model (70GB each)
+- ~70GB free per GPU for activations and gradients
+
+**Training (DP=2):**
+- 2 GPUs process different batches
+- LoRA adapters: ~5-10GB per GPU
+- Gradients/optimizer states: ~20-30GB per GPU
+- **Total per training GPU: ~50-60GB** ✅
+
+## Quick Start
+
+### 1. Prepare Your Dataset
+
+Your dataset should be in JSONL format with conversation turns:
+
+```jsonl
+{"messages": [{"role": "system", "content": "..."}, {"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
+{"messages": [{"role": "user", "content": "..."}, {"role": "assistant", "content": "..."}]}
+```
+
+### 2. Run Training
+
+```bash
+# Using the helper script
+./examples/multi_step/run_sft_qwen30b.sh path/to/your/dataset.jsonl
+
+# Or directly with synth-ai CLI
+uvx synth-ai train \
+  --type sft \
+  --config examples/multi_step/configs/crafter_sft_qwen30b_lora.toml \
+  --dataset path/to/your/dataset.jsonl \
+  --env-file backend/.env.dev
+```
+
+### 3. Monitor Training
+
+Check the Synth dashboard for:
+- Training loss curve
+- Validation metrics (if validation set provided)
+- GPU utilization
+- Training throughput (tokens/sec)
+
+## Hyperparameters
+
+**Batch Configuration:**
+- Per-device batch size: 1
+- Gradient accumulation: 64 steps
+- **Effective global batch size: 128** (1 × 64 × 2 GPUs)
+
+**Learning Rate:**
+- Initial LR: 5e-6
+- Warmup ratio: 3%
+- Schedule: Linear decay
+
+**Sequence Length:** 4096 tokens
+
+**Training:**
+- Epochs: 1
+- Mixed precision: BF16
+- DeepSpeed: Stage 2 (optimizer state sharding)
+- Activation checkpointing: Enabled
+
+## Configuration File Structure
+
+```toml
+[algorithm]
+type = "offline"        # Supervised (not RL)
+method = "sft"          # Supervised fine-tuning
+variety = "lora"        # Using LoRA adapters
+
+[compute]
+gpu_type = "H200"
+gpu_count = 4
+
+[data.topology]
+tensor_parallel = 2     # Split model across 2 GPUs
+data_parallel = 2       # Split batches across 2 GPUs
+
+[training]
+mode = "lora"
+use_qlora = true        # Quantized LoRA (4-bit base model)
+
+[lora]
+r = 16                  # LoRA rank
+alpha = 32              # LoRA scaling
+dropout = 0.05
+target_modules = ["all-linear"]  # Apply to all linear layers
+```
+
+## Comparison with RL Config
+
+| Aspect | SFT | RL |
+|--------|-----|-----|
+| Purpose | Supervised learning | Reinforcement learning |
+| Data | Labeled examples | Environment interactions |
+| Topology | TP=2, DP=2 | Split: 2 inference + 2 training |
+| Batch size | 128 (effective) | Variable (episode-based) |
+| Training | Standard backprop | Policy gradient (GSPO) |
+
+## Tips
+
+1. **Start Small:** Test with a small dataset first to verify the pipeline
+2. **Validation:** Add a validation set to monitor overfitting
+3. **Checkpointing:** Training saves checkpoints every 100 steps
+4. **Resume:** Can resume from checkpoint if training is interrupted
+5. **Inference:** After training, use the LoRA adapter with the base model
+
+## Output
+
+After training completes, you'll get:
+- LoRA adapter weights (saved to volume)
+- Training metrics and logs
+- Best checkpoint (based on validation loss)
+- Model ready for inference or RL initialization
+
+## Next Steps
+
+1. **Evaluate:** Test your fine-tuned model on held-out data
+2. **RL Training:** Use this as initialization for RL (`init_from_sft = true`)
+3. **Deploy:** Load LoRA adapter for inference
+4. **Iterate:** Adjust hyperparameters based on performance
+

synth_ai-0.2.16/examples/multi_step/configs/README_verilog_rl.md

@@ -0,0 +1,77 @@
+# Verilog RL with LoRA (Qwen3-0.6B)
+
+## Quick Start
+
+1. **Deploy Verilog Task App**:
+```bash
+cd synth-ai
+uvx synth-ai modal-serve grpo-verilog
+```
+Note the Modal URL and update `task_url` in `verilog_rl_lora.toml`.
+
+2. **Run Training**:
+```bash
+uvx synth-ai rl run --config examples/multi_step/configs/verilog_rl_lora.toml
+```
+
+## Configuration Overview
+
+### **Key Adaptations from Crafter**:
+
+- **Model**: `Qwen/Qwen3-0.6B` (✅ proven in SFT configs)
+- **Environment**: `verilog` instead of `crafter`
+- **Steps**: 15 turns (vs Crafter's 10) for compilation workflows
+- **Rewards**: Adjusted for sparser Verilog rewards (0.5 vs 1.0 indicator_lambda)
+- **Rubrics**: Verilog-specific judging criteria
+
+### **Hardware Requirements** (Standard RL setup):
+- ✅ **2x H100 GPUs** (vLLM inference + LoRA training split)
+- ✅ **No tensor parallelism** needed for 0.6B model
+- ✅ **4x faster inference** than 32B model
+- ✅ **Same compute pattern** as Crafter (just smaller model)
+
+### **Expected Workflow**:
+1. Agent writes Verilog code (`write_file`)
+2. Compiles to check syntax (`compile`)
+3. Simulates to verify behavior (`simulate`)
+4. Submits if tests pass (`submit`)
+5. **Rewards**: +1.0 for compilation success, +10.0 for passing tests
+
+## Rubric Design
+
+### **Event Rewards** (per decision):
+- **Compilation Success**: 70% weight (1.0 for success, 0.0 for errors)
+- **Process Efficiency**: 30% weight (penalizes redundant operations)
+
+### **Outcome Rewards** (final score):
+- **Tests Passed**: 80% weight (full credit when all tests pass)
+- **Design Quality**: 20% weight (code clarity, documentation)
+
+## Troubleshooting
+
+### **If training fails**:
+1. Check Modal URL in `task_url` field
+2. Verify `GROQ_API_KEY` for inference
+3. Ensure `OPENAI_API_KEY` for judging
+
+### **Memory issues** (unlikely with 0.6B):
+- Reduce `batch_size` to 2
+- Set `gradient_accumulation_steps = 2`
+- Verify 2x GPU split is working (vLLM on GPU 0, training on GPU 1)
+
+### **Slow training**:
+- Increase `episodes_per_batch` to 6-8
+- Check network latency to Modal task app
+
+## Expected Results
+
+- **Convergence**: Should learn basic compilation workflow in 1-2 hours
+- **Success Rate**: 20-40% initial test pass rate (improves with training)
+- **Learning**: Agent learns to debug compilation errors and write correct Verilog
+
+## Next Steps
+
+1. **Monitor reward progression** in training logs
+2. **Adjust rubrics** if agent struggles with compilation errors
+3. **Scale to 8B model** once 0.6B baseline works
+4. **Add domain-specific fine-tuning** for Verilog syntax

synth_ai-0.2.16/examples/multi_step/configs/VERILOG_REWARDS.md

@@ -0,0 +1,90 @@
+# Verilog Reward Structure (Normalized to 1.0)
+
+## Overview
+All rewards in the Verilog task app are normalized so the maximum possible reward is **1.0**.
+
+## Reward Components
+
+### 1. Step Penalty: **-0.001** per step
+- Applied to every action taken
+- Encourages efficient solutions
+- Normalized from `-0.01` (original)
+
+### 2. Compile Success: **+0.01**
+- Awarded when `iverilog` compilation succeeds (returncode 0)
+- Validates syntax correctness
+- Normalized from `+0.1` (original)
+
+### 3. Simulation Pass: **+0.1**
+- Awarded when `vvp` simulation passes all tests
+- Validates behavioral correctness
+- Normalized from `+1.0` (original)
+
+### 4. Submit Success: **+1.0** (maximum reward)
+- Awarded when final submission passes all verification tests
+- This is the goal state
+- Normalized from `+10.0` (original)
+
+## Typical Reward Trajectories
+
+### ✅ Optimal Path (3 steps)
+```
+Step 1: write_file        → -0.001
+Step 2: compile (success) → +0.01 - 0.001 = +0.009
+Step 3: simulate (pass)   → +0.1 - 0.001 = +0.099
+Total:                      ~0.107
+```
+
+### ✅ Good Path (4 steps with submit)
+```
+Step 1: write_file        → -0.001
+Step 2: compile (success) → +0.009
+Step 3: simulate (pass)   → +0.099
+Step 4: submit (success)  → +1.0 - 0.001 = +0.999
+Total:                      ~1.106
+```
+*Note: Can exceed 1.0 if intermediate rewards stack with final submit*
+
+### ❌ Failure Path (compilation errors)
+```
+Step 1: write_file        → -0.001
+Step 2: compile (fail)    → -0.001
+Step 3: write_file (fix)  → -0.001
+Step 4: compile (success) → +0.009
+Step 5: simulate (pass)   → +0.099
+Total:                      ~0.105
+```
+
+## Implementation Details
+
+### Location
+- **Reward components**: `synth_ai/environments/examples/verilog/engine.py`
+  - `VerilogCompileSuccessComponent`: +0.01
+  - `VerilogSimulationPassComponent`: +0.1
+  - `VerilogSubmitSuccessComponent`: +1.0
+  - `VerilogStepPenaltyComponent`: -0.001
+
+### Normalization Ratio
+All rewards were divided by **10.0** to normalize:
+- Original max: ~10.0
+- Normalized max: ~1.0
+- Ratio: 10.0
+
+## Why Normalize?
+
+1. **Consistency**: Makes it easier to compare rewards across different task types
+2. **RL Training**: Standard reward scales improve learning stability
+3. **Interpretability**: Rewards as percentages (0.0 to 1.0) are intuitive
+4. **Judge Compatibility**: Rubric scores typically range 0-1, making blending easier
+
+## Testing
+```bash
+# Run eval to verify normalized rewards
+uvx synth-ai eval --config examples/multi_step/configs/verilog_eval_groq_qwen32b.toml
+```
+
+Expected output for successful rollout:
+- `mean_return` ≈ 0.1 (if only compile+simulate)
+- `mean_return` ≈ 1.0+ (if full submit success)
+
+