fine 0.1.0

data/docs/tutorials/siglip2-object-recognition.md

@@ -0,0 +1,203 @@
+# Fine-tuning SigLIP2 for New Object Recognition
+
+Teach the model to recognize specific objects, products, or concepts it hasn't seen before.
+
+## When to Use This
+
+- You want to detect your specific products, logos, or items
+- You need to recognize custom objects not in standard datasets
+- Examples: your product catalog, brand logos, custom equipment, specific animals/plants
+
+## How It Works
+
+SigLIP2 learns visual concepts from image-text pairs. For object recognition, we fine-tune it to associate your specific objects with labels, enabling it to recognize them in new images.
+
+## Dataset Structure
+
+Create folders for each object you want to recognize:
+
+```
+data/
+  train/
+    my_product_a/
+      product_a_01.jpg
+      product_a_02.jpg
+      product_a_angle1.jpg
+      product_a_lighting2.jpg
+      ...
+    my_product_b/
+      product_b_01.jpg
+      ...
+    background/           # Optional: negative examples
+      random_scene_01.jpg
+      ...
+```
+
+**Tips for Object Recognition:**
+- Capture objects from multiple angles
+- Vary lighting conditions
+- Include different backgrounds
+- Show objects at different scales
+- 30-100 images per object works well
+
+## Training
+
+```ruby
+require "fine"
+
+# Use a larger model for better object recognition
+classifier = Fine::ImageClassifier.new("google/siglip2-base-patch16-384") do |config|
+  config.epochs = 5
+  config.batch_size = 16
+  config.learning_rate = 1e-4     # Slightly lower for fine-grained recognition
+  config.freeze_encoder = false   # Full fine-tuning for new concepts
+end
+
+classifier.fit(
+  train_dir: "data/train",
+  val_dir: "data/val"
+)
+
+classifier.save("models/product_recognizer")
+```
+
+## Recognizing Objects in New Images
+
+```ruby
+recognizer = Fine::ImageClassifier.load("models/product_recognizer")
+
+# Check what object is in an image
+results = recognizer.predict("customer_photo.jpg")
+
+# Get top prediction
+top = results.first
+if top[:score] > 0.7
+  puts "Detected: #{top[:label]} (#{(top[:score] * 100).round}% confident)"
+else
+  puts "No confident match found"
+end
+
+# See all possibilities
+results.each do |pred|
+  puts "#{pred[:label]}: #{(pred[:score] * 100).round}%"
+end
+```
+
+## Including Negative Examples
+
+For better precision, include a "background" or "other" class:
+
+```
+data/train/
+  product_a/
+  product_b/
+  other/          # Images without your products
+```
+
+This helps the model learn what your objects are NOT.
+
+## Multi-Object Detection Strategy
+
+If images might contain multiple objects, train separate binary classifiers:
+
+```ruby
+# Train one model per object type
+products = ["product_a", "product_b", "product_c"]
+
+products.each do |product|
+  classifier = Fine::ImageClassifier.new("google/siglip2-base-patch16-224")
+
+  # Binary classification: this_product vs everything_else
+  classifier.fit(train_dir: "data/binary/#{product}")
+  classifier.save("models/detect_#{product}")
+end
+
+# At inference time, run all detectors
+def detect_all_products(image_path)
+  detected = []
+
+  products.each do |product|
+    detector = Fine::ImageClassifier.load("models/detect_#{product}")
+    results = detector.predict(image_path)
+
+    if results.first[:label] == product && results.first[:score] > 0.8
+      detected << product
+    end
+  end
+
+  detected
+end
+```
+
+## Best Practices
+
+### Image Collection
+
+1. **Variety is key**: Same object, different conditions
+2. **Real-world context**: Objects in actual use, not just product shots
+3. **Scale variation**: Close-ups and distant shots
+4. **Partial visibility**: Objects partially obscured (if that's realistic)
+
+### Data Augmentation
+
+Enable augmentation for more robust recognition:
+
+```ruby
+classifier = Fine::ImageClassifier.new("google/siglip2-base-patch16-384") do |config|
+  config.augmentation do |aug|
+    aug.random_horizontal_flip = true
+    aug.random_rotation = 15          # Degrees
+    aug.color_jitter = { brightness: 0.2, contrast: 0.2 }
+  end
+end
+```
+
+### Confidence Thresholds
+
+Set appropriate thresholds based on your use case:
+
+```ruby
+results = recognizer.predict(image)
+confidence = results.first[:score]
+
+case
+when confidence > 0.9
+  # High confidence - safe to act automatically
+when confidence > 0.7
+  # Medium confidence - show to user for confirmation
+else
+  # Low confidence - likely not a known object
+end
+```
+
+## Example: Product Catalog Recognition
+
+```ruby
+# Train on your product catalog
+catalog_recognizer = Fine::ImageClassifier.new("google/siglip2-base-patch16-384") do |config|
+  config.epochs = 10
+  config.batch_size = 8
+  config.on_epoch_end do |epoch, metrics|
+    puts "Epoch #{epoch + 1}: val_accuracy=#{(metrics[:val_accuracy] * 100).round(1)}%"
+  end
+end
+
+catalog_recognizer.fit(
+  train_dir: "catalog/train",
+  val_dir: "catalog/val"
+)
+
+catalog_recognizer.save("models/catalog_v1")
+
+# Use in production
+def identify_product(photo_path)
+  recognizer = Fine::ImageClassifier.load("models/catalog_v1")
+  results = recognizer.predict(photo_path, top_k: 3)
+
+  {
+    product_id: results.first[:label],
+    confidence: results.first[:score],
+    alternatives: results[1..2]
+  }
+end
+```

data/docs/tutorials/siglip2-similarity-search.md

@@ -0,0 +1,152 @@
+# Using SigLIP2 for Image Similarity Search
+
+Find visually similar images using learned embeddings.
+
+## When to Use This
+
+- Find similar products in a catalog
+- Detect near-duplicates
+- Build "more like this" features
+- Visual search engines
+
+## How It Works
+
+Instead of classifying images, we extract the embedding vectors from SigLIP2 and compare them. Similar images have similar embeddings.
+
+## Extracting Embeddings
+
+```ruby
+require "fine"
+
+# Load a pre-trained model (no fine-tuning needed for general similarity)
+# Or load your fine-tuned model for domain-specific similarity
+model = Fine::Models::SigLIP2ForImageClassification.from_pretrained(
+  "google/siglip2-base-patch16-224",
+  num_labels: 1  # Dummy value, we won't use the classifier
+)
+
+# Get the encoder only
+encoder = model.encoder
+
+# Prepare image transform
+transforms = Fine::Transforms::Compose.new([
+  Fine::Transforms::Resize.new(224),
+  Fine::Transforms::ToTensor.new,
+  Fine::Transforms::Normalize.new
+])
+
+def get_embedding(encoder, transforms, image_path)
+  image = Vips::Image.new_from_file(image_path, access: :sequential)
+  tensor = transforms.call(image).unsqueeze(0)  # Add batch dimension
+
+  encoder.eval
+  Torch.no_grad do
+    encoder.call(tensor).squeeze.to_a
+  end
+end
+
+# Extract embeddings
+embedding1 = get_embedding(encoder, transforms, "image1.jpg")
+embedding2 = get_embedding(encoder, transforms, "image2.jpg")
+```
+
+## Computing Similarity
+
+```ruby
+def cosine_similarity(a, b)
+  dot = a.zip(b).sum { |x, y| x * y }
+  norm_a = Math.sqrt(a.sum { |x| x * x })
+  norm_b = Math.sqrt(b.sum { |x| x * x })
+  dot / (norm_a * norm_b)
+end
+
+similarity = cosine_similarity(embedding1, embedding2)
+puts "Similarity: #{(similarity * 100).round(1)}%"
+# > 90% = very similar
+# 70-90% = somewhat similar
+# < 70% = different
+```
+
+## Building a Search Index
+
+For searching through many images:
+
+```ruby
+class ImageSearchIndex
+  def initialize(encoder, transforms)
+    @encoder = encoder
+    @transforms = transforms
+    @embeddings = {}
+  end
+
+  def add(image_path)
+    @embeddings[image_path] = get_embedding(image_path)
+  end
+
+  def search(query_path, top_k: 5)
+    query_emb = get_embedding(query_path)
+
+    results = @embeddings.map do |path, emb|
+      { path: path, score: cosine_similarity(query_emb, emb) }
+    end
+
+    results.sort_by { |r| -r[:score] }.first(top_k)
+  end
+
+  private
+
+  def get_embedding(path)
+    image = Vips::Image.new_from_file(path, access: :sequential)
+    tensor = @transforms.call(image).unsqueeze(0)
+
+    @encoder.eval
+    Torch.no_grad do
+      @encoder.call(tensor).squeeze.to_a
+    end
+  end
+
+  def cosine_similarity(a, b)
+    dot = a.zip(b).sum { |x, y| x * y }
+    norm_a = Math.sqrt(a.sum { |x| x * x })
+    norm_b = Math.sqrt(b.sum { |x| x * x })
+    dot / (norm_a * norm_b)
+  end
+end
+
+# Usage
+index = ImageSearchIndex.new(encoder, transforms)
+
+# Index your images
+Dir.glob("catalog/*.jpg").each { |path| index.add(path) }
+
+# Search
+results = index.search("query_image.jpg", top_k: 10)
+results.each do |result|
+  puts "#{result[:path]}: #{(result[:score] * 100).round}% similar"
+end
+```
+
+## Domain-Specific Similarity
+
+For better results on your specific domain, fine-tune first:
+
+```ruby
+# Fine-tune on your domain
+classifier = Fine::ImageClassifier.new("google/siglip2-base-patch16-224")
+classifier.fit(train_dir: "my_domain/train", epochs: 3)
+classifier.save("models/my_domain")
+
+# Load the encoder from fine-tuned model
+model = Fine::Models::SigLIP2ForImageClassification.load("models/my_domain")
+encoder = model.encoder
+
+# Now use this encoder for similarity search
+# It will be better at distinguishing items in your domain
+```
+
+## Performance Tips
+
+For large catalogs (10k+ images):
+- Pre-compute and cache embeddings
+- Use approximate nearest neighbor libraries (e.g., Annoy, Faiss via FFI)
+- Batch embedding computation for faster indexing

data/docs/tutorials/text-classification.md

@@ -0,0 +1,233 @@
+# Fine-tuning Text Classification Models
+
+Classify text into categories—sentiment, spam, intent, topics.
+
+## When to Use This
+
+- Sentiment analysis (positive/negative/neutral reviews)
+- Spam detection
+- Intent classification for chatbots
+- Support ticket routing
+- Content moderation
+- Topic categorization
+
+## Supported Models
+
+| Model | Parameters | Speed | Quality |
+|-------|------------|-------|---------|
+| `distilbert-base-uncased` | 66M | Fast | Good |
+| `bert-base-uncased` | 110M | Medium | Better |
+| `roberta-base` | 125M | Medium | Better |
+| `microsoft/deberta-v3-small` | 44M | Fast | Great |
+| `microsoft/deberta-v3-base` | 86M | Medium | Best |
+
+## Dataset Format
+
+JSONL file with `text` and `label` fields:
+
+```
+data/train.jsonl
+{"text": "This product exceeded my expectations!", "label": "positive"}
+{"text": "Terrible quality, broke after one day", "label": "negative"}
+{"text": "It's okay, nothing special", "label": "neutral"}
+```
+
+Or CSV:
+
+```
+data/train.csv
+text,label
+"This product exceeded my expectations!",positive
+"Terrible quality, broke after one day",negative
+```
+
+## Basic Training
+
+```ruby
+require "fine"
+
+classifier = Fine::TextClassifier.new("distilbert-base-uncased")
+
+classifier.fit(
+  train_file: "data/train.jsonl",
+  val_file: "data/val.jsonl",
+  epochs: 3
+)
+
+classifier.save("models/sentiment")
+```
+
+## Making Predictions
+
+```ruby
+classifier = Fine::TextClassifier.load("models/sentiment")
+
+# Single text
+result = classifier.predict("I love this product!")
+puts result.first[:label]  # => "positive"
+puts result.first[:score]  # => 0.97
+
+# Batch prediction
+results = classifier.predict([
+  "Great service, highly recommend",
+  "Worst purchase ever",
+  "It works as described"
+])
+```
+
+## Configuration
+
+```ruby
+classifier = Fine::TextClassifier.new("microsoft/deberta-v3-small") do |config|
+  config.epochs = 5
+  config.batch_size = 16
+  config.learning_rate = 2e-5      # Lower than vision models
+  config.max_length = 256          # Max tokens per text
+  config.warmup_ratio = 0.1        # 10% of steps for warmup
+
+  config.on_epoch_end do |epoch, metrics|
+    puts "Epoch #{epoch}: val_accuracy=#{(metrics[:val_accuracy] * 100).round(1)}%"
+  end
+end
+```
+
+## Use Cases
+
+### Sentiment Analysis
+
+```ruby
+# Train on product reviews
+classifier = Fine::TextClassifier.new("distilbert-base-uncased")
+classifier.fit(train_file: "reviews.jsonl", epochs: 3)
+
+# Analyze new reviews
+def analyze_sentiment(review_text)
+  result = classifier.predict(review_text).first
+  {
+    sentiment: result[:label],
+    confidence: result[:score],
+    needs_attention: result[:label] == "negative" && result[:score] > 0.8
+  }
+end
+```
+
+### Spam Detection
+
+```ruby
+classifier = Fine::TextClassifier.new("distilbert-base-uncased")
+classifier.fit(train_file: "spam_ham.jsonl", epochs: 3)
+
+def is_spam?(message)
+  result = classifier.predict(message).first
+  result[:label] == "spam" && result[:score] > 0.7
+end
+```
+
+### Intent Classification
+
+```ruby
+# For chatbot / support routing
+# Labels: billing, technical, shipping, general, cancel
+
+classifier = Fine::TextClassifier.new("microsoft/deberta-v3-small")
+classifier.fit(train_file: "support_intents.jsonl", epochs: 5)
+
+def route_ticket(message)
+  result = classifier.predict(message).first
+
+  case result[:label]
+  when "billing"
+    assign_to_billing_team(message)
+  when "cancel"
+    assign_to_retention_team(message)
+  when "technical"
+    assign_to_tech_support(message)
+  else
+    assign_to_general_queue(message)
+  end
+end
+```
+
+### Multi-label Classification
+
+For texts that can have multiple labels:
+
+```ruby
+classifier = Fine::TextClassifier.new("distilbert-base-uncased") do |config|
+  config.multi_label = true
+  config.threshold = 0.5  # Predict labels above this confidence
+end
+
+# Data format for multi-label
+# {"text": "Server crashed and lost data", "labels": ["technical", "urgent", "data_loss"]}
+
+classifier.fit(train_file: "tickets_multilabel.jsonl")
+
+result = classifier.predict("Payment failed and I can't login")
+# => [{ label: "billing", score: 0.89 }, { label: "technical", score: 0.72 }]
+```
+
+## Data Preparation Tips
+
+### Minimum Data
+
+- 100+ examples per class for decent results
+- 500+ examples per class for good results
+- Balance classes or use class weights
+
+### Class Imbalance
+
+```ruby
+classifier = Fine::TextClassifier.new("distilbert-base-uncased") do |config|
+  config.class_weights = :balanced  # Auto-compute from data
+  # Or manually: config.class_weights = { "positive" => 1.0, "negative" => 2.5 }
+end
+```
+
+### Text Preprocessing
+
+The tokenizer handles most preprocessing, but you might want to:
+
+```ruby
+def clean_text(text)
+  text
+    .gsub(/<[^>]+>/, ' ')     # Remove HTML
+    .gsub(/https?:\S+/, '')   # Remove URLs
+    .gsub(/\s+/, ' ')         # Normalize whitespace
+    .strip
+end
+```
+
+## Evaluation
+
+```ruby
+# Load test set
+test_data = File.readlines("data/test.jsonl").map { |l| JSON.parse(l) }
+
+predictions = classifier.predict(test_data.map { |d| d["text"] })
+actuals = test_data.map { |d| d["label"] }
+
+# Calculate accuracy
+correct = predictions.zip(actuals).count { |pred, actual| pred.first[:label] == actual }
+accuracy = correct.to_f / predictions.size
+puts "Test accuracy: #{(accuracy * 100).round(1)}%"
+```
+
+## Troubleshooting
+
+**Low accuracy:**
+- Add more training data
+- Check for mislabeled examples
+- Try a larger model (deberta-v3-base)
+- Increase epochs
+
+**Overfitting:**
+- Add more data
+- Use dropout (increase in config)
+- Reduce epochs
+- Use a smaller model
+
+**Slow training:**
+- Reduce `max_length`
+- Use a smaller model (distilbert)
+- Reduce batch size if memory-constrained