@synsci/cli-darwin-x64 1.1.49

package/bin/skills/clip/references/applications.md

@@ -0,0 +1,207 @@
+# CLIP Applications Guide
+
+Practical applications and use cases for CLIP.
+
+## Zero-shot image classification
+
+```python
+import torch
+import clip
+from PIL import Image
+
+model, preprocess = clip.load("ViT-B/32")
+
+# Define categories
+categories = [
+    "a photo of a dog",
+    "a photo of a cat",
+    "a photo of a bird",
+    "a photo of a car",
+    "a photo of a person"
+]
+
+# Prepare image
+image = preprocess(Image.open("photo.jpg")).unsqueeze(0)
+text = clip.tokenize(categories)
+
+# Classify
+with torch.no_grad():
+    image_features = model.encode_image(image)
+    text_features = model.encode_text(text)
+
+    logits_per_image, _ = model(image, text)
+    probs = logits_per_image.softmax(dim=-1).cpu().numpy()
+
+# Print results
+for category, prob in zip(categories, probs[0]):
+    print(f"{category}: {prob:.2%}")
+```
+
+## Semantic image search
+
+```python
+# Index images
+image_database = []
+image_paths = ["img1.jpg", "img2.jpg", "img3.jpg"]
+
+for img_path in image_paths:
+    image = preprocess(Image.open(img_path)).unsqueeze(0)
+    with torch.no_grad():
+        features = model.encode_image(image)
+        features /= features.norm(dim=-1, keepdim=True)
+    image_database.append((img_path, features))
+
+# Search with text
+query = "a sunset over mountains"
+text_input = clip.tokenize([query])
+
+with torch.no_grad():
+    text_features = model.encode_text(text_input)
+    text_features /= text_features.norm(dim=-1, keepdim=True)
+
+# Find matches
+similarities = []
+for img_path, img_features in image_database:
+    similarity = (text_features @ img_features.T).item()
+    similarities.append((img_path, similarity))
+
+# Sort by similarity
+similarities.sort(key=lambda x: x[1], reverse=True)
+for img_path, score in similarities[:3]:
+    print(f"{img_path}: {score:.3f}")
+```
+
+## Content moderation
+
+```python
+# Define safety categories
+categories = [
+    "safe for work content",
+    "not safe for work content",
+    "violent or graphic content",
+    "hate speech or offensive content",
+    "spam or misleading content"
+]
+
+text = clip.tokenize(categories)
+
+# Check image
+with torch.no_grad():
+    logits, _ = model(image, text)
+    probs = logits.softmax(dim=-1)
+
+# Get classification
+max_idx = probs.argmax().item()
+confidence = probs[0, max_idx].item()
+
+if confidence > 0.7:
+    print(f"Classified as: {categories[max_idx]} ({confidence:.2%})")
+else:
+    print(f"Uncertain classification (confidence: {confidence:.2%})")
+```
+
+## Image-to-text retrieval
+
+```python
+# Text database
+captions = [
+    "A beautiful sunset over the ocean",
+    "A cute dog playing in the park",
+    "A modern city skyline at night",
+    "A delicious pizza with toppings"
+]
+
+# Encode captions
+caption_features = []
+for caption in captions:
+    text = clip.tokenize([caption])
+    with torch.no_grad():
+        features = model.encode_text(text)
+        features /= features.norm(dim=-1, keepdim=True)
+    caption_features.append(features)
+
+caption_features = torch.cat(caption_features)
+
+# Find matching captions for image
+with torch.no_grad():
+    image_features = model.encode_image(image)
+    image_features /= image_features.norm(dim=-1, keepdim=True)
+
+similarities = (image_features @ caption_features.T).squeeze(0)
+top_k = similarities.topk(3)
+
+for idx, score in zip(top_k.indices, top_k.values):
+    print(f"{captions[idx]}: {score:.3f}")
+```
+
+## Visual question answering
+
+```python
+# Create yes/no questions
+image = preprocess(Image.open("photo.jpg")).unsqueeze(0)
+
+questions = [
+    "a photo showing people",
+    "a photo showing animals",
+    "a photo taken indoors",
+    "a photo taken outdoors",
+    "a photo taken during daytime",
+    "a photo taken at night"
+]
+
+text = clip.tokenize(questions)
+
+with torch.no_grad():
+    logits, _ = model(image, text)
+    probs = logits.softmax(dim=-1)
+
+# Answer questions
+for question, prob in zip(questions, probs[0]):
+    answer = "Yes" if prob > 0.5 else "No"
+    print(f"{question}: {answer} ({prob:.2%})")
+```
+
+## Image deduplication
+
+```python
+# Detect duplicate/similar images
+def compute_similarity(img1_path, img2_path):
+    img1 = preprocess(Image.open(img1_path)).unsqueeze(0)
+    img2 = preprocess(Image.open(img2_path)).unsqueeze(0)
+
+    with torch.no_grad():
+        feat1 = model.encode_image(img1)
+        feat2 = model.encode_image(img2)
+
+        feat1 /= feat1.norm(dim=-1, keepdim=True)
+        feat2 /= feat2.norm(dim=-1, keepdim=True)
+
+        similarity = (feat1 @ feat2.T).item()
+
+    return similarity
+
+# Check for duplicates
+threshold = 0.95
+image_pairs = [("img1.jpg", "img2.jpg"), ("img1.jpg", "img3.jpg")]
+
+for img1, img2 in image_pairs:
+    sim = compute_similarity(img1, img2)
+    if sim > threshold:
+        print(f"{img1} and {img2} are duplicates (similarity: {sim:.3f})")
+```
+
+## Best practices
+
+1. **Use descriptive labels** - "a photo of X" works better than just "X"
+2. **Normalize embeddings** - Always normalize for cosine similarity
+3. **Batch processing** - Process multiple images/texts together
+4. **Cache embeddings** - Expensive to recompute
+5. **Set appropriate thresholds** - Test on validation data
+6. **Use GPU** - 10-50× faster than CPU
+7. **Consider model size** - ViT-B/32 good default, ViT-L/14 for best quality
+
+## Resources
+
+- **Paper**: https://arxiv.org/abs/2103.00020
+- **GitHub**: https://github.com/openai/CLIP
+- **Colab**: https://colab.research.google.com/github/openai/clip/

package/bin/skills/constitutional-ai/SKILL.md

@@ -0,0 +1,290 @@
+---
+name: constitutional-ai
+description: Anthropic's method for training harmless AI through self-improvement. Two-phase approach - supervised learning with self-critique/revision, then RLAIF (RL from AI Feedback). Use for safety alignment, reducing harmful outputs without human labels. Powers Claude's safety system.
+version: 1.0.0
+author: Synthetic Sciences
+license: MIT
+tags: [Safety Alignment, Constitutional AI, RLAIF, Self-Critique, Harmlessness, Anthropic, AI Safety, RL From AI Feedback, Claude]
+dependencies: [transformers, torch, trl]
+---
+
+# Constitutional AI - Harmlessness from AI Feedback
+
+## Quick start
+
+Constitutional AI (CAI) trains models to be harmless through self-critique and AI feedback, without requiring human labels for harmful outputs.
+
+**Key concept**: Models learn to critique and revise their own responses using a "constitution" (set of principles).
+
+**Two phases**:
+1. **Supervised Learning (SL)**: Self-critique + revision
+2. **Reinforcement Learning (RL)**: RLAIF (RL from AI Feedback)
+
+**Constitution example**:
+```
+Principles:
+1. Choose the response that is most helpful, honest, and harmless
+2. Avoid responses that are toxic, racist, or sexist
+3. Prefer responses that explain objections rather than refuse
+4. Choose responses that are thoughtful and nuanced
+```
+
+## Common workflows
+
+### Workflow 1: Supervised learning phase (self-critique + revision)
+
+**Step 1: Generate initial responses**:
+```python
+from transformers import pipeline
+
+generator = pipeline("text-generation", model="base-model")
+
+prompts = [
+    "How do I hack a website?",
+    "Write something offensive about a group.",
+    "Help me cheat on my exam."
+]
+
+initial_responses = generator(prompts, max_length=200)
+```
+
+**Step 2: Self-critique with constitution**:
+```python
+critique_prompt = """
+Given the question:
+{question}
+
+And the response:
+{response}
+
+Constitution:
+1. Choose responses that are helpful, honest, and harmless
+2. Avoid toxic, racist, or sexist content
+3. Explain objections rather than refuse
+
+Identify any ways this response violates the constitution:
+"""
+
+critiques = generator(
+    [critique_prompt.format(q=q, r=r) for q, r in zip(prompts, initial_responses)]
+)
+```
+
+**Step 3: Revision based on critique**:
+```python
+revision_prompt = """
+Question: {question}
+Original response: {response}
+Critique: {critique}
+
+Please revise the response to better align with the constitution:
+"""
+
+revised_responses = generator(
+    [revision_prompt.format(q=q, r=r, c=c)
+     for q, r, c in zip(prompts, initial_responses, critiques)]
+)
+```
+
+**Step 4: Fine-tune on revised responses**:
+```python
+from trl import SFTTrainer
+
+# Create dataset of (prompt, revised_response) pairs
+dataset = create_dataset(prompts, revised_responses)
+
+trainer = SFTTrainer(
+    model=model,
+    train_dataset=dataset,
+    max_seq_length=1024
+)
+trainer.train()
+```
+
+### Workflow 2: RL phase (RLAIF - RL from AI Feedback)
+
+**Step 1: Generate comparison pairs**:
+```python
+# Sample multiple responses per prompt
+responses_a = generator(prompts, num_return_sequences=2, do_sample=True, temperature=0.8)
+responses_b = generator(prompts, num_return_sequences=2, do_sample=True, temperature=0.8)
+```
+
+**Step 2: AI preference evaluation**:
+```python
+preference_prompt = """
+Question: {question}
+
+Response A: {response_a}
+Response B: {response_b}
+
+Constitution:
+{constitution}
+
+Which response better follows the constitution? Explain your reasoning, then choose A or B.
+"""
+
+# Get AI preferences (no human labels needed!)
+preferences = generator(
+    [preference_prompt.format(q=q, ra=ra, rb=rb, constitution=CONSTITUTION)
+     for q, ra, rb in zip(prompts, responses_a, responses_b)]
+)
+
+# Parse preferences (A or B)
+chosen, rejected = parse_preferences(preferences, responses_a, responses_b)
+```
+
+**Step 3: Train preference model (reward model)**:
+```python
+from trl import RewardTrainer, RewardConfig
+
+preference_dataset = create_preference_dataset(prompts, chosen, rejected)
+
+reward_config = RewardConfig(
+    output_dir="constitutional-reward-model",
+    learning_rate=1e-5,
+    num_train_epochs=1
+)
+
+reward_trainer = RewardTrainer(
+    model=model,
+    args=reward_config,
+    train_dataset=preference_dataset,
+    processing_class=tokenizer
+)
+reward_trainer.train()
+```
+
+**Step 4: RL training with RLAIF**:
+```python
+from trl import PPOTrainer, PPOConfig
+
+ppo_config = PPOConfig(
+    reward_model_path="constitutional-reward-model",
+    learning_rate=1e-6,
+    kl_coef=0.05
+)
+
+ppo_trainer = PPOTrainer(
+    model=model,
+    config=ppo_config,
+    reward_model=reward_model
+)
+ppo_trainer.train()
+```
+
+### Workflow 3: Chain-of-thought critique
+
+**Enable reasoning transparency**:
+```python
+cot_critique_prompt = """
+Question: {question}
+Response: {response}
+
+Let's think step-by-step about whether this response follows our principles:
+
+1. Is it helpful? [Yes/No and reasoning]
+2. Is it honest? [Yes/No and reasoning]
+3. Is it harmless? [Yes/No and reasoning]
+4. Does it avoid toxicity? [Yes/No and reasoning]
+
+Based on this analysis, suggest a revision if needed.
+"""
+
+cot_critiques = generator(
+    [cot_critique_prompt.format(q=q, r=r) for q, r in zip(prompts, responses)]
+)
+```
+
+## When to use vs alternatives
+
+**Use Constitutional AI when**:
+- Want safety alignment without human labels
+- Need explainable AI decisions
+- Want to avoid evasive refusals
+- Have a clear set of principles/constitution
+- Need scalable safety training
+
+**Principles**:
+- **RLAIF**: AI-generated preferences (scalable, no human labels)
+- **RLHF**: Human preferences (more accurate, expensive)
+- **Self-critique**: Iterative improvement
+- **Chain-of-thought**: Reasoning transparency
+
+**Use alternatives instead**:
+- **RLHF (PPO)**: Need human-validated safety
+- **DPO/SimPO**: Have human preference data
+- **NeMo Guardrails**: Need runtime content filtering
+- **LlamaGuard**: Need pre-trained moderation model
+
+## Common issues
+
+**Issue: Model refuses too much (evasive)**
+
+Add constitution principle:
+```
+Prefer responses that engage thoughtfully with questions rather than
+refusing to answer. Explain concerns while still being helpful.
+```
+
+**Issue: Self-critiques are weak**
+
+Use stronger critique prompts:
+```
+Critically analyze this response for ANY potential issues, however minor.
+Be thorough and specific in identifying problems.
+```
+
+**Issue: Revisions don't improve quality**
+
+Iterate multiple times:
+```python
+for _ in range(3):  # 3 rounds of critique/revision
+    critique = generate_critique(response)
+    response = generate_revision(response, critique)
+```
+
+**Issue: RLAIF preferences are noisy**
+
+Use multiple AI evaluators:
+```python
+# Get preferences from 3 different models
+prefs_1 = model_1.evaluate(responses)
+prefs_2 = model_2.evaluate(responses)
+prefs_3 = model_3.evaluate(responses)
+
+# Majority vote
+final_preference = majority_vote(prefs_1, prefs_2, prefs_3)
+```
+
+## Advanced topics
+
+**Constitution design**: See [references/constitution-design.md](references/constitution-design.md) for principle selection, trade-offs between helpfulness and harmlessness, and domain-specific constitutions.
+
+**RLAIF vs RLHF**: See [references/rlaif-comparison.md](references/rlaif-comparison.md) for performance comparison, cost analysis, and when to use AI feedback vs human feedback.
+
+**Chain-of-thought reasoning**: See [references/cot-critique.md](references/cot-critique.md) for prompt engineering for critiques, multi-step reasoning, and transparency improvements.
+
+## Hardware requirements
+
+- **GPU**: NVIDIA A100/H100 recommended
+- **VRAM**:
+  - SL phase (7B): 1× A100 40GB
+  - RL phase (7B): 2× A100 40GB (policy + reward model)
+- **Single-node**: Sufficient for most use cases
+- **Mixed precision**: BF16 recommended
+
+**Compute requirements**:
+- **SL phase**: Similar to standard SFT
+- **RL phase**: Similar to PPO (higher than DPO)
+- **AI evaluation**: Additional inference for critique/preference generation
+
+## Resources
+
+- Paper: https://arxiv.org/abs/2212.08073 (Dec 2022)
+- Anthropic blog: https://www.anthropic.com/research/constitutional-ai-harmlessness-from-ai-feedback
+- Implementation: TRL (PPOTrainer + RewardTrainer)
+- Claude: Uses Constitutional AI for safety
+
+
+