@synsci/cli-darwin-x64 1.1.97 → 1.1.99

package/bin/skills/colab-finetuning/SKILL.md

@@ -1,153 +0,0 @@
----
-name: colab-finetuning
-description: Fine-tune LLMs on Google Colab GPUs directly from synsc. Connects to Colab runtimes via WebSocket bridge for remote training with Unsloth. Supports SFT, GRPO, DPO, vision, and TTS workflows on free T4 to Pro A100 GPUs.
-version: 1.0.0
-author: Synthetic Sciences
-license: MIT
-tags: [Fine-Tuning, Google Colab, GPU, Remote Training, Unsloth, LoRA, GRPO, Cloud GPU]
-dependencies: [unsloth, torch, transformers, trl, datasets]
----
-
-# Google Colab Fine-Tuning
-
-Fine-tune LLMs using Google Colab GPUs directly from the synsc CLI. Connect to free or paid Colab runtimes and run Unsloth training workflows remotely.
-
-## When to Use Colab Fine-Tuning
-
-**Use Colab when:**
-- You don't have a local GPU but need to fine-tune a model
-- You want free GPU access (T4 with 15GB VRAM on Colab Free)
-- Training models up to ~14B parameters (4-bit QLoRA)
-- Quick experiments and prototyping before scaling to cloud
-- Colab Pro/Pro+ for A100 (40-80GB) access
-
-**Don't use Colab when:**
-- You need persistent long-running jobs (>12h) — use Tinker or cloud providers
-- Training 70B+ models — use Lambda, RunPod, or multi-GPU cloud
-- You need guaranteed uptime — Colab may disconnect idle sessions
-- Production training pipelines — use managed services
-
-**Colab vs Alternatives:**
-
-| Need | Use |
-|------|-----|
-| Free GPU, quick experiments | **Google Colab** |
-| Managed cloud training (any size) | Tinker |
-| Persistent multi-GPU training | Lambda / RunPod |
-| Local GPU available | Unsloth directly |
-| Enterprise with SLA | Colab Enterprise (Vertex AI) |
-
-## Quick Start
-
-### Step 1: Generate Bridge Notebook
-
-```
-Use colab_notebook tool with workflow="bridge"
-```
-
-This creates a `synsc-bridge.ipynb` file that establishes a WebSocket tunnel between synsc and the Colab GPU.
-
-### Step 2: Open in Colab
-
-1. Go to [colab.research.google.com](https://colab.research.google.com)
-2. Upload the bridge notebook (File → Upload notebook)
-3. Select GPU runtime (Runtime → Change runtime type → T4 GPU)
-4. Run all cells
-5. Copy the WebSocket URL that appears
-
-### Step 3: Connect from synsc
-
-```
-Use colab_connect tool with connection_url="wss://..."
-```
-
-### Step 4: Run Training
-
-```
-Use colab_finetune tool with:
-  workflow: "sft"
-  model: "unsloth/Qwen3-4B-unsloth-bnb-4bit"
-  dataset: "mlabonne/FineTome-100k"
-```
-
-Or execute individual cells:
-```
-Use colab_execute tool with code="import torch; print(torch.cuda.get_device_name(0))"
-```
-
-## GPU Tiers
-
-| Tier | GPU | VRAM | Max Model (QLoRA) | Session Limit |
-|------|-----|------|-------------------|---------------|
-| Free | T4 | 15 GB | ~14B | 12h, may disconnect |
-| Pro ($10/mo) | T4/V100/A100 | 16-40 GB | ~32B | 24h, priority |
-| Pro+ ($50/mo) | A100 (80GB) | 80 GB | ~72B | 24h, guaranteed |
-| Enterprise | Configurable | Any | Any | No limit |
-
-See [references/gpu-tiers.md](references/gpu-tiers.md) for detailed VRAM requirements.
-
-## Available Tools
-
-| Tool | Purpose |
-|------|---------|
-| `colab_connect` | Connect to a Colab runtime (standard bridge or enterprise) |
-| `colab_execute` | Run arbitrary Python code on the connected GPU |
-| `colab_status` | Check GPU, memory, disk, connection status |
-| `colab_finetune` | Run complete Unsloth training workflow (SFT/GRPO/DPO/vision/TTS) |
-| `colab_notebook` | Generate .ipynb notebooks for Colab |
-
-## Training Workflows
-
-### SFT (Supervised Fine-Tuning)
-Standard instruction tuning. Best for: chat models, domain adaptation, format learning.
-
-```
-colab_finetune workflow=sft model="unsloth/Qwen3-4B-unsloth-bnb-4bit" dataset="mlabonne/FineTome-100k"
-```
-
-### GRPO (Reinforcement Learning)
-Train reasoning models with reward functions. Best for: math, coding, structured output.
-
-```
-colab_finetune workflow=grpo model="unsloth/Qwen3-4B-unsloth-bnb-4bit" dataset="your-dataset"
-```
-
-### DPO (Direct Preference Optimization)
-Align models with human preferences. Requires chosen/rejected pairs.
-
-```
-colab_finetune workflow=dpo model="unsloth/Llama-3.1-8B-unsloth-bnb-4bit" dataset="HuggingFaceH4/ultrafeedback_binarized"
-```
-
-### Vision Fine-Tuning
-Fine-tune vision-language models (Qwen3-VL, Gemma 3, Llama 3.2 Vision).
-
-```
-colab_finetune workflow=vision model="unsloth/Qwen3-VL-2B-unsloth-bnb-4bit" dataset="your-vision-dataset"
-```
-
-### TTS Fine-Tuning
-Fine-tune text-to-speech models (Orpheus, Sesame-CSM).
-
-```
-colab_finetune workflow=tts model="unsloth/orpheus-3b-0.1-ft-unsloth-bnb-4bit" dataset="your-tts-dataset"
-```
-
-## Troubleshooting
-
-### Connection Issues
-- **"WebSocket connection failed"**: Ensure the bridge notebook is still running in Colab. Re-run the tunnel cell if the URL expired.
-- **"Execution timeout"**: Colab may have disconnected due to inactivity. Run the keep-alive cell.
-- **"Connection closed during execution"**: Colab Free disconnects after idle time. Upgrade to Pro for more stability.
-
-### Training Issues
-- **CUDA OOM**: Reduce batch_size, max_seq_length, or lora_rank. Use 4-bit quantization.
-- **Slow training**: Ensure GPU runtime is selected. Check with `colab_status detail=gpu`.
-- **Package not found**: Unsloth is installed automatically by `colab_finetune`. For custom packages, use `colab_execute` with pip install.
-
-### Colab Enterprise
-- Requires GCP project with Vertex AI API enabled
-- Use `colab_connect mode=enterprise project_id="your-project"`
-- GPU quotas apply — check GCP console for availability
-
-See [references/troubleshooting.md](references/troubleshooting.md) for more solutions.

package/bin/skills/colab-finetuning/references/bridge-setup.md

@@ -1,68 +0,0 @@
-# Bridge Notebook Setup
-
-## How the Bridge Works
-
-The bridge notebook creates a WebSocket tunnel from a Google Colab runtime to your local machine:
-
-```
-synsc CLI ←→ WebSocket ←→ Cloudflare Tunnel ←→ Jupyter Server (on Colab GPU)
-```
-
-1. A Jupyter notebook server starts on the Colab VM (port 8888)
-2. `jupyter_http_over_ws` extension enables WebSocket connections
-3. `cloudflared` creates a public tunnel URL
-4. synsc connects via WebSocket and sends Jupyter kernel messages
-
-## Step-by-Step Setup
-
-### 1. Generate the Bridge Notebook
-```
-Use colab_notebook tool with workflow="bridge"
-```
-This creates `synsc-bridge.ipynb` in your project directory.
-
-### 2. Upload to Google Colab
-- Go to [colab.research.google.com](https://colab.research.google.com)
-- File → Upload notebook → select `synsc-bridge.ipynb`
-
-### 3. Select GPU Runtime
-- Runtime → Change runtime type
-- Hardware accelerator: GPU
-- GPU type: T4 (free) or A100 (pro)
-
-### 4. Run All Cells
-Run cells in order:
-1. **GPU check** — confirms GPU is available
-2. **Install dependencies** — installs `jupyter_http_over_ws`
-3. **Install cloudflared** — downloads the tunnel binary
-4. **Start bridge** — starts Jupyter + tunnel, prints WebSocket URL
-
-### 5. Copy the URL
-The output will show:
-```
-============================================================
-SYNSC BRIDGE READY
-============================================================
-
-Paste this URL into synsc:
-
-  wss://random-name.trycloudflare.com/api/kernels/default/channels?token=...
-
-============================================================
-```
-
-### 6. Connect from synsc
-```
-Use colab_connect tool with connection_url="wss://..."
-```
-
-### 7. Keep Alive
-Run the keep-alive cell to prevent Colab from disconnecting due to inactivity.
-
-## Security
-
-- The bridge uses a random token for authentication
-- The Cloudflare tunnel URL is unique and expires when the notebook stops
-- No Google account credentials are transmitted through the bridge
-- The tunnel is read/write — anyone with the URL can execute code on the Colab VM
-- **Do not share the WebSocket URL**

package/bin/skills/colab-finetuning/references/gpu-tiers.md

@@ -1,54 +0,0 @@
-# GPU Tiers & VRAM Requirements
-
-## Colab GPU Options
-
-### Free Tier
-- **T4**: 15 GB VRAM, Compute Capability 7.5
-- **P100**: 16 GB VRAM, Compute Capability 6.0 (less common)
-- Session limit: ~12 hours, may disconnect during idle
-
-### Pro Tier ($10/month)
-- **T4**: 15 GB VRAM (priority access)
-- **V100**: 16 GB VRAM, Compute Capability 7.0
-- **L4**: 24 GB VRAM, Compute Capability 8.9
-- **A100**: 40 GB VRAM, Compute Capability 8.0
-- Session limit: ~24 hours, priority GPU allocation
-
-### Pro+ Tier ($50/month)
-- **A100 (80GB)**: 80 GB VRAM
-- Session limit: ~24 hours, guaranteed GPU access
-- Background execution available
-
-### Enterprise (Vertex AI)
-- Any GPU type configured in GCP
-- No session limits
-- Full API control
-- Requires GCP project and billing
-
-## Model VRAM Requirements (4-bit QLoRA)
-
-| Model Size | VRAM Required | Fits on |
-|-----------|---------------|---------|
-| 1B | ~3 GB | T4 (free) |
-| 3B | ~5 GB | T4 (free) |
-| 4B | ~6 GB | T4 (free) |
-| 7-8B | ~9 GB | T4 (free) |
-| 13-14B | ~15 GB | T4 (free, tight) |
-| 32B | ~22 GB | L4, A100 (Pro) |
-| 70-72B | ~44 GB | A100 80GB (Pro+) |
-
-## Recommended Configurations
-
-### Free Tier (T4, 15 GB)
-- **SFT**: Qwen3-4B, Llama-3.2-3B, Gemma-3-4B — batch_size=2, max_seq=2048
-- **GRPO**: Qwen3-4B — batch_size=1, max_seq=1024, num_generations=4
-- **Vision**: Qwen3-VL-2B — batch_size=1
-
-### Pro Tier (A100, 40 GB)
-- **SFT**: Qwen3-14B, Llama-3.1-8B — batch_size=4, max_seq=4096
-- **GRPO**: 8B models — batch_size=2, num_generations=8
-- **DPO**: 8B models — batch_size=4
-
-### Pro+ Tier (A100 80GB)
-- **SFT**: Qwen3-32B, Llama-3.3-70B (tight) — batch_size=1-2
-- **GRPO**: 32B models — batch_size=1

package/bin/skills/colab-finetuning/references/troubleshooting.md

@@ -1,79 +0,0 @@
-# Troubleshooting
-
-## Connection Problems
-
-### "WebSocket connection failed"
-- **Cause**: Bridge notebook stopped or tunnel URL expired
-- **Fix**: Re-run the bridge cells in Colab to get a fresh URL
-
-### "Connection timeout after 30 seconds"
-- **Cause**: Firewall blocking WebSocket connections, or Colab VM is unresponsive
-- **Fix**: Try a different network. Check if Colab runtime is still active.
-
-### "Connection closed during execution"
-- **Cause**: Colab disconnected the session (idle timeout or resource limits)
-- **Fix**: Run the keep-alive cell. Upgrade to Colab Pro for longer sessions.
-
-### Bridge notebook won't start
-- **Cause**: Colab Free tier quota exhausted
-- **Fix**: Wait for quota reset (resets daily) or upgrade to Pro.
-
-## Training Problems
-
-### CUDA Out of Memory (OOM)
-- Reduce `batch_size` to 1
-- Reduce `max_seq_length` (try 1024 or 512)
-- Reduce `lora_rank` (try 8 instead of 16)
-- Use `quantization="4bit"` (default)
-- For GRPO: reduce `num_generations` to 2
-
-### Training is very slow
-- Check GPU: `colab_status detail=gpu` — ensure a GPU is connected
-- If using T4, training 8B+ models will be slower than A100
-- Ensure `packing=True` for SFT (default in templates)
-- Check for CPU bottleneck: `colab_status detail=full`
-
-### "ModuleNotFoundError: No module named 'unsloth'"
-- The `colab_finetune` tool installs Unsloth automatically
-- If running manual code, first run: `!pip install unsloth`
-
-### Dataset loading fails
-- Verify the dataset ID exists on HuggingFace
-- For gated datasets, authenticate: `!huggingface-cli login`
-- For local datasets, upload to Colab first with `colab_execute`
-
-### Loss is NaN or not decreasing
-- Reduce learning rate (try 1e-5 for GRPO, 5e-5 for SFT)
-- Check dataset format matches the template expectations
-- Ensure the dataset isn't empty: `colab_execute code="print(len(dataset))"`
-
-## Colab-Specific Issues
-
-### "You have exhausted your GPU quota"
-- Colab Free limits GPU usage per day/week
-- **Fix**: Wait for reset, or use Colab Pro
-
-### Runtime disconnects after ~30 minutes
-- Run the keep-alive cell in the bridge notebook
-- Interact with the Colab tab occasionally (Colab detects browser activity)
-
-### "Cannot connect to GPU backend"
-- Colab may be overloaded — try again later
-- Change runtime type and change back (Runtime → Change runtime type)
-
-### Files disappear after disconnect
-- Colab VMs are ephemeral — files are deleted when the runtime stops
-- Save important files to Google Drive or push to HuggingFace Hub
-- Use `push_to_hub` parameter in `colab_finetune` to auto-upload
-
-## Enterprise (Vertex AI) Issues
-
-### "Not authenticated"
-- Run `/connect google-colab` to authenticate with Google OAuth
-
-### "Insufficient quota"
-- Check GCP quotas in the Cloud Console
-- Request quota increase for the desired GPU type and region
-
-### "Vertex AI API not enabled"
-- Enable the API: `gcloud services enable aiplatform.googleapis.com`

package/bin/skills/constitutional-ai/SKILL.md

@@ -1,290 +0,0 @@
----
-name: constitutional-ai
-description: Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful outputs without human labels. Powers Claude's safety system.
-version: 1.0.0
-author: Synthetic Sciences
-license: MIT
-tags: [Safety Alignment, Constitutional AI, RLAIF, Self-Critique, Harmlessness, Anthropic, AI Safety, RL From AI Feedback, Claude]
-dependencies: [transformers, torch, trl]
----
-
-# Constitutional AI - Harmlessness from AI Feedback
-
-## Quick start
-
-Constitutional AI (CAI) trains models to be harmless through self-critique and AI feedback, without requiring human labels for harmful outputs.
-
-**Key concept**: Models learn to critique and revise their own responses using a "constitution" (set of principles).
-
-**Two phases**:
-1. **Supervised Learning (SL)**: Self-critique + revision
-2. **Reinforcement Learning (RL)**: RLAIF (RL from AI Feedback)
-
-**Constitution example**:
-```
-Principles:
-1. Choose the response that is most helpful, honest, and harmless
-2. Avoid responses that are toxic, racist, or sexist
-3. Prefer responses that explain objections rather than refuse
-4. Choose responses that are thoughtful and nuanced
-```
-
-## Common workflows
-
-### Workflow 1: Supervised learning phase (self-critique + revision)
-
-**Step 1: Generate initial responses**:
-```python
-from transformers import pipeline
-
-generator = pipeline("text-generation", model="base-model")
-
-prompts = [
-    "How do I hack a website?",
-    "Write something offensive about a group.",
-    "Help me cheat on my exam."
-]
-
-initial_responses = generator(prompts, max_length=200)
-```
-
-**Step 2: Self-critique with constitution**:
-```python
-critique_prompt = """
-Given the question:
-{question}
-
-And the response:
-{response}
-
-Constitution:
-1. Choose responses that are helpful, honest, and harmless
-2. Avoid toxic, racist, or sexist content
-3. Explain objections rather than refuse
-
-Identify any ways this response violates the constitution:
-"""
-
-critiques = generator(
-    [critique_prompt.format(q=q, r=r) for q, r in zip(prompts, initial_responses)]
-)
-```
-
-**Step 3: Revision based on critique**:
-```python
-revision_prompt = """
-Question: {question}
-Original response: {response}
-Critique: {critique}
-
-Please revise the response to better align with the constitution:
-"""
-
-revised_responses = generator(
-    [revision_prompt.format(q=q, r=r, c=c)
-     for q, r, c in zip(prompts, initial_responses, critiques)]
-)
-```
-
-**Step 4: Fine-tune on revised responses**:
-```python
-from trl import SFTTrainer
-
-# Create dataset of (prompt, revised_response) pairs
-dataset = create_dataset(prompts, revised_responses)
-
-trainer = SFTTrainer(
-    model=model,
-    train_dataset=dataset,
-    max_seq_length=1024
-)
-trainer.train()
-```
-
-### Workflow 2: RL phase (RLAIF - RL from AI Feedback)
-
-**Step 1: Generate comparison pairs**:
-```python
-# Sample multiple responses per prompt
-responses_a = generator(prompts, num_return_sequences=2, do_sample=True, temperature=0.8)
-responses_b = generator(prompts, num_return_sequences=2, do_sample=True, temperature=0.8)
-```
-
-**Step 2: AI preference evaluation**:
-```python
-preference_prompt = """
-Question: {question}
-
-Response A: {response_a}
-Response B: {response_b}
-
-Constitution:
-{constitution}
-
-Which response better follows the constitution? Explain your reasoning, then choose A or B.
-"""
-
-# Get AI preferences (no human labels needed!)
-preferences = generator(
-    [preference_prompt.format(q=q, ra=ra, rb=rb, constitution=CONSTITUTION)
-     for q, ra, rb in zip(prompts, responses_a, responses_b)]
-)
-
-# Parse preferences (A or B)
-chosen, rejected = parse_preferences(preferences, responses_a, responses_b)
-```
-
-**Step 3: Train preference model (reward model)**:
-```python
-from trl import RewardTrainer, RewardConfig
-
-preference_dataset = create_preference_dataset(prompts, chosen, rejected)
-
-reward_config = RewardConfig(
-    output_dir="constitutional-reward-model",
-    learning_rate=1e-5,
-    num_train_epochs=1
-)
-
-reward_trainer = RewardTrainer(
-    model=model,
-    args=reward_config,
-    train_dataset=preference_dataset,
-    processing_class=tokenizer
-)
-reward_trainer.train()
-```
-
-**Step 4: RL training with RLAIF**:
-```python
-from trl import PPOTrainer, PPOConfig
-
-ppo_config = PPOConfig(
-    reward_model_path="constitutional-reward-model",
-    learning_rate=1e-6,
-    kl_coef=0.05
-)
-
-ppo_trainer = PPOTrainer(
-    model=model,
-    config=ppo_config,
-    reward_model=reward_model
-)
-ppo_trainer.train()
-```
-
-### Workflow 3: Chain-of-thought critique
-
-**Enable reasoning transparency**:
-```python
-cot_critique_prompt = """
-Question: {question}
-Response: {response}
-
-Let's think step-by-step about whether this response follows our principles:
-
-1. Is it helpful? [Yes/No and reasoning]
-2. Is it honest? [Yes/No and reasoning]
-3. Is it harmless? [Yes/No and reasoning]
-4. Does it avoid toxicity? [Yes/No and reasoning]
-
-Based on this analysis, suggest a revision if needed.
-"""
-
-cot_critiques = generator(
-    [cot_critique_prompt.format(q=q, r=r) for q, r in zip(prompts, responses)]
-)
-```
-
-## When to use vs alternatives
-
-**Use Constitutional AI when**:
-- Want safety alignment without human labels
-- Need explainable AI decisions
-- Want to avoid evasive refusals
-- Have a clear set of principles/constitution
-- Need scalable safety training
-
-**Principles**:
-- **RLAIF**: AI-generated preferences (scalable, no human labels)
-- **RLHF**: Human preferences (more accurate, expensive)
-- **Self-critique**: Iterative improvement
-- **Chain-of-thought**: Reasoning transparency
-
-**Use alternatives instead**:
-- **RLHF (PPO)**: Need human-validated safety
-- **DPO/SimPO**: Have human preference data
-- **NeMo Guardrails**: Need runtime content filtering
-- **LlamaGuard**: Need pre-trained moderation model
-
-## Common issues
-
-**Issue: Model refuses too much (evasive)**
-
-Add constitution principle:
-```
-Prefer responses that engage thoughtfully with questions rather than
-refusing to answer. Explain concerns while still being helpful.
-```
-
-**Issue: Self-critiques are weak**
-
-Use stronger critique prompts:
-```
-Critically analyze this response for ANY potential issues, however minor.
-Be thorough and specific in identifying problems.
-```
-
-**Issue: Revisions don't improve quality**
-
-Iterate multiple times:
-```python
-for _ in range(3):  # 3 rounds of critique/revision
-    critique = generate_critique(response)
-    response = generate_revision(response, critique)
-```
-
-**Issue: RLAIF preferences are noisy**
-
-Use multiple AI evaluators:
-```python
-# Get preferences from 3 different models
-prefs_1 = model_1.evaluate(responses)
-prefs_2 = model_2.evaluate(responses)
-prefs_3 = model_3.evaluate(responses)
-
-# Majority vote
-final_preference = majority_vote(prefs_1, prefs_2, prefs_3)
-```
-
-## Advanced topics
-
-**Constitution design**: See [references/constitution-design.md](references/constitution-design.md) for principle selection, trade-offs between helpfulness and harmlessness, and domain-specific constitutions.
-
-**RLAIF vs RLHF**: See [references/rlaif-comparison.md](references/rlaif-comparison.md) for performance comparison, cost analysis, and when to use AI feedback vs human feedback.
-
-**Chain-of-thought reasoning**: See [references/cot-critique.md](references/cot-critique.md) for prompt engineering for critiques, multi-step reasoning, and transparency improvements.
-
-## Hardware requirements
-
-- **GPU**: NVIDIA A100/H100 recommended
-- **VRAM**:
-  - SL phase (7B): 1× A100 40GB
-  - RL phase (7B): 2× A100 40GB (policy + reward model)
-- **Single-node**: Sufficient for most use cases
-- **Mixed precision**: BF16 recommended
-
-**Compute requirements**:
-- **SL phase**: Similar to standard SFT
-- **RL phase**: Similar to PPO (higher than DPO)
-- **AI evaluation**: Additional inference for critique/preference generation
-
-## Resources
-
-- Paper: https://arxiv.org/abs/2212.08073 (Dec 2022)
-- Anthropic blog: https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback
-- Implementation: TRL (PPOTrainer + RewardTrainer)
-- Claude: Uses Constitutional AI for safety
-
-
-