ruby_llm 0.1.0.pre37 → 0.1.0.pre39

data/docs/guides/embeddings.md

@@ -1,325 +0,0 @@
----
-layout: default
-title: Embeddings
-parent: Guides
-nav_order: 7
-permalink: /guides/embeddings
----
-
-# Embeddings
-
-Text embeddings are numerical representations of text that capture semantic meaning. RubyLLM makes it easy to generate embeddings for a variety of applications, including semantic search, clustering, and recommendation systems.
-
-## Basic Embedding Generation
-
-The simplest way to create an embedding is with the global `embed` method:
-
-```ruby
-# Create an embedding for a single text
-embedding = RubyLLM.embed("Ruby is a programmer's best friend")
-
-# The vector representation
-vector = embedding.vectors
-puts "Vector dimension: #{vector.length}"  # => 1536 for text-embedding-3-small
-```
-
-## Embedding Multiple Texts
-
-You can efficiently embed multiple texts at once:
-
-```ruby
-# Create embeddings for multiple texts
-texts = ["Ruby", "Python", "JavaScript"]
-embeddings = RubyLLM.embed(texts)
-
-# Each text gets its own vector
-puts "Number of vectors: #{embeddings.vectors.length}"  # => 3
-puts "First vector dimensions: #{embeddings.vectors.first.length}"
-```
-
-## Choosing Models
-
-By default, RubyLLM uses OpenAI's `text-embedding-3-small`, but you can specify a different model:
-
-```ruby
-# Use a specific model
-embedding = RubyLLM.embed(
-  "This is a test sentence",
-  model: "text-embedding-3-large"
-)
-
-# Or use a Google model
-google_embedding = RubyLLM.embed(
-  "This is a test sentence",
-  model: "text-embedding-004"
-)
-```
-
-You can configure the default embedding model globally:
-
-```ruby
-RubyLLM.configure do |config|
-  config.default_embedding_model = "text-embedding-3-large"
-end
-```
-
-## Using Embedding Results
-
-### Vector Properties
-
-The embedding result contains useful information:
-
-```ruby
-embedding = RubyLLM.embed("Example text")
-
-# The vector representation
-puts embedding.vectors.class  # => Array
-puts embedding.vectors.first.class  # => Float
-
-# The model used
-puts embedding.model  # => "text-embedding-3-small"
-
-# Token usage
-puts embedding.input_tokens  # => 3
-```
-
-### Calculating Similarity
-
-Embeddings are commonly used to calculate similarity between texts:
-
-```ruby
-require 'matrix'
-
-# Create embeddings for two texts
-embedding1 = RubyLLM.embed("I love Ruby programming")
-embedding2 = RubyLLM.embed("Ruby is my favorite language")
-
-# Convert to Vector objects
-vector1 = Vector.elements(embedding1.vectors)
-vector2 = Vector.elements(embedding2.vectors)
-
-# Calculate cosine similarity
-similarity = vector1.inner_product(vector2) / (vector1.norm * vector2.norm)
-puts "Similarity: #{similarity}"  # Higher values (closer to 1) mean more similar
-```
-
-### Simple Semantic Search
-
-```ruby
-# Create a simple search index
-class SearchIndex
-  def initialize(texts, model: nil)
-    @texts = texts
-    @embeddings = RubyLLM.embed(texts, model: model).vectors
-  end
-
-  def search(query, top_k: 3)
-    query_embedding = RubyLLM.embed(query).vectors
-    query_vector = Vector.elements(query_embedding)
-
-    # Calculate similarities
-    similarities = @embeddings.map.with_index do |embedding, idx|
-      vector = Vector.elements(embedding)
-      similarity = query_vector.inner_product(vector) / (query_vector.norm * vector.norm)
-      [idx, similarity]
-    end
-
-    # Return top results
-    similarities.sort_by { |_, similarity| -similarity }
-                .take(top_k)
-                .map { |idx, similarity| { text: @texts[idx], similarity: similarity } }
-  end
-end
-
-# Create an index
-documents = [
-  "Ruby is a dynamic, interpreted language",
-  "Python is known for its readability",
-  "JavaScript runs in the browser",
-  "Ruby on Rails is a web framework",
-  "Django is a Python web framework"
-]
-
-index = SearchIndex.new(documents)
-
-# Search for similar documents
-results = index.search("web development frameworks")
-results.each do |result|
-  puts "#{result[:text]} (Similarity: #{result[:similarity].round(4)})"
-end
-```
-
-## Error Handling
-
-Handle errors that may occur during embedding generation:
-
-```ruby
-begin
-  embedding = RubyLLM.embed("Example text")
-rescue RubyLLM::UnauthorizedError
-  puts "Please check your API key"
-rescue RubyLLM::BadRequestError => e
-  puts "Invalid request: #{e.message}"
-rescue RubyLLM::Error => e
-  puts "Error generating embedding: #{e.message}"
-end
-```
-
-## Performance Considerations
-
-When working with embeddings, keep these best practices in mind:
-
-1. **Batch processing** - Embedding multiple texts at once is more efficient than making separate calls
-2. **Caching** - Store embeddings in your database rather than regenerating them
-3. **Dimensionality** - Different models produce embeddings with different dimensions
-4. **Normalization** - Consider normalizing vectors to improve similarity calculations
-
-## Working with Large Datasets
-
-For larger datasets, process embeddings in batches:
-
-```ruby
-def embed_in_batches(texts, batch_size: 100, model: nil)
-  all_embeddings = []
-
-  texts.each_slice(batch_size) do |batch|
-    batch_embeddings = RubyLLM.embed(batch, model: model).vectors
-    all_embeddings.concat(batch_embeddings)
-
-    # Optional: add a small delay to avoid rate limiting
-    sleep(0.1)
-  end
-
-  all_embeddings
-end
-
-# Usage
-documents = File.readlines("documents.txt", chomp: true)
-embeddings = embed_in_batches(documents)
-```
-
-## Rails Integration
-
-In a Rails application, you might integrate embeddings like this:
-
-```ruby
-class Document < ApplicationRecord
-  serialize :embedding, Array
-
-  before_save :generate_embedding, if: -> { content_changed? }
-
-  def self.search(query, limit: 10)
-    # Generate query embedding
-    query_embedding = RubyLLM.embed(query).vectors
-
-    # Convert to SQL for similarity search
-    where.not(embedding: nil)
-         .select("*, (embedding <=> ?) AS similarity", query_embedding)
-         .order("similarity DESC")
-         .limit(limit)
-  end
-
-  private
-
-  def generate_embedding
-    return if content.blank?
-
-    self.embedding = RubyLLM.embed(content).vectors
-  rescue RubyLLM::Error => e
-    errors.add(:base, "Failed to generate embedding: #{e.message}")
-    throw :abort
-  end
-end
-```
-
-Note: The above example assumes you're using PostgreSQL with the `pgvector` extension for vector similarity search.
-
-## Example Use Cases
-
-### Document Classification
-
-```ruby
-# Train a simple classifier
-class SimpleClassifier
-  def initialize
-    @categories = {}
-  end
-
-  def train(text, category)
-    @categories[category] ||= []
-    @categories[category] << RubyLLM.embed(text).vectors
-  end
-
-  def classify(text)
-    # Get embedding for the query text
-    query_embedding = RubyLLM.embed(text).vectors
-    query_vector = Vector.elements(query_embedding)
-
-    # Find the closest category
-    best_similarity = -1
-    best_category = nil
-
-    @categories.each do |category, embeddings|
-      # Calculate average similarity to this category
-      similarity = embeddings.map do |embedding|
-        vector = Vector.elements(embedding)
-        query_vector.inner_product(vector) / (query_vector.norm * vector.norm)
-      end.sum / embeddings.size
-
-      if similarity > best_similarity
-        best_similarity = similarity
-        best_category = category
-      end
-    end
-
-    { category: best_category, confidence: best_similarity }
-  end
-end
-
-# Usage
-classifier = SimpleClassifier.new
-
-# Train with examples
-classifier.train("How do I install Ruby?", :installation)
-classifier.train("Setting up Ruby environment", :installation)
-classifier.train("What are blocks in Ruby?", :language_features)
-classifier.train("Understanding Ruby modules", :language_features)
-
-# Classify new queries
-puts classifier.classify("How to install Ruby on Ubuntu?")
-# => {:category=>:installation, :confidence=>0.92}
-```
-
-### Content Recommendation
-
-```ruby
-def recommend_similar_content(content_id, library, count: 3)
-  # Get the target content
-  target = library.find(content_id)
-  target_embedding = RubyLLM.embed(target.description).vectors
-  target_vector = Vector.elements(target_embedding)
-
-  # Compare with all other content
-  similarities = library.reject { |item| item.id == content_id }.map do |item|
-    next if item.embedding.nil?
-
-    item_vector = Vector.elements(item.embedding)
-    similarity = target_vector.inner_product(item_vector) / (target_vector.norm * item_vector.norm)
-
-    [item, similarity]
-  end.compact
-
-  # Return top matches
-  similarities.sort_by { |_, similarity| -similarity }
-              .take(count)
-              .map { |item, similarity| { item: item, similarity: similarity } }
-end
-```
-
-## Next Steps
-
-Now that you understand embeddings, you might want to explore:
-
-- [Chat]({% link guides/chat.md %}) for interactive AI conversations
-- [Tools]({% link guides/tools.md %}) to extend AI capabilities
-- [Error Handling]({% link guides/error-handling.md %}) for robust applications

data/docs/guides/error-handling.md

@@ -1,301 +0,0 @@
----
-layout: default
-title: Error Handling
-parent: Guides
-nav_order: 8
-permalink: /guides/error-handling
----
-
-# Error Handling
-
-Proper error handling is crucial when working with AI services. RubyLLM provides a comprehensive error handling system that helps you build robust applications.
-
-## Error Hierarchy
-
-RubyLLM uses a structured error hierarchy:
-
-```ruby
-RubyLLM::Error                    # Base error class
-    RubyLLM::BadRequestError      # Invalid request parameters (400)
-    RubyLLM::UnauthorizedError    # API key issues (401)
-    RubyLLM::PaymentRequiredError # Billing issues (402)
-    RubyLLM::RateLimitError       # Rate limit exceeded (429)
-    RubyLLM::ServerError          # Provider server error (500)
-    RubyLLM::ServiceUnavailableError # Service unavailable (503)
-    RubyLLM::ModelNotFoundError   # Invalid model ID
-    RubyLLM::InvalidRoleError     # Invalid message role
-```
-
-## Basic Error Handling
-
-Wrap your AI interactions in `begin/rescue` blocks:
-
-```ruby
-begin
-  chat = RubyLLM.chat
-  response = chat.ask "What's the capital of France?"
-  puts response.content
-rescue RubyLLM::Error => e
-  puts "AI interaction failed: #{e.message}"
-end
-```
-
-## Handling Specific Errors
-
-Target specific error types for more precise handling:
-
-```ruby
-begin
-  chat = RubyLLM.chat
-  response = chat.ask "Generate a detailed analysis"
-rescue RubyLLM::UnauthorizedError
-  puts "Please check your API credentials"
-rescue RubyLLM::PaymentRequiredError
-  puts "Payment required - please check your account balance"
-rescue RubyLLM::RateLimitError
-  puts "Rate limit exceeded - please try again later"
-rescue RubyLLM::ServiceUnavailableError
-  puts "Service temporarily unavailable - please try again later"
-rescue RubyLLM::BadRequestError => e
-  puts "Bad request: #{e.message}"
-rescue RubyLLM::Error => e
-  puts "Other error: #{e.message}"
-end
-```
-
-## API Response Details
-
-The `Error` class contains the original response, allowing for detailed error inspection:
-
-```ruby
-begin
-  chat = RubyLLM.chat
-  chat.ask "Some question"
-rescue RubyLLM::Error => e
-  puts "Error: #{e.message}"
-  puts "Status: #{e.response.status}"
-  puts "Body: #{e.response.body}"
-end
-```
-
-## Error Handling with Streaming
-
-When using streaming, errors can occur during the stream:
-
-```ruby
-begin
-  chat = RubyLLM.chat
-  chat.ask "Generate a long response" do |chunk|
-    print chunk.content
-  end
-rescue RubyLLM::Error => e
-  puts "\nStreaming error: #{e.message}"
-end
-```
-
-## Handling Tool Errors
-
-When using tools, errors can be handled within the tool or in the calling code:
-
-```ruby
-# Error handling within tools
-class Calculator < RubyLLM::Tool
-  description "Performs calculations"
-
-  param :expression,
-    type: :string,
-    desc: "Math expression to evaluate"
-
-  def execute(expression:)
-    eval(expression).to_s
-  rescue StandardError => e
-    # Return error as structured data
-    { error: "Calculation error: #{e.message}" }
-  end
-end
-
-# Error handling when using tools
-begin
-  chat = RubyLLM.chat.with_tool(Calculator)
-  chat.ask "What's 1/0?"
-rescue RubyLLM::Error => e
-  puts "Error using tools: #{e.message}"
-end
-```
-
-## Automatic Retries
-
-RubyLLM automatically retries on certain transient errors:
-
-```ruby
-# Configure retry behavior
-RubyLLM.configure do |config|
-  config.max_retries = 5 # Maximum number of retries
-end
-```
-
-The following errors trigger automatic retries:
-- Network timeouts
-- Connection failures
-- Rate limit errors (429)
-- Server errors (500, 502, 503, 504)
-
-## Provider-Specific Errors
-
-Each provider may return slightly different error messages. RubyLLM normalizes these into standard error types, but the original error details are preserved:
-
-```ruby
-begin
-  chat = RubyLLM.chat
-  chat.ask "Some question"
-rescue RubyLLM::Error => e
-  if e.response.body.include?("organization_quota_exceeded")
-    puts "Your organization's quota has been exceeded"
-  else
-    puts "Error: #{e.message}"
-  end
-end
-```
-
-## Error Handling in Rails
-
-When using RubyLLM in a Rails application, you can handle errors at different levels:
-
-### Controller Level
-
-```ruby
-class ChatController < ApplicationController
-  rescue_from RubyLLM::Error, with: :handle_ai_error
-
-  def create
-    @chat = Chat.create!(chat_params)
-    @chat.ask(params[:message])
-    redirect_to @chat
-  end
-
-  private
-
-  def handle_ai_error(exception)
-    flash[:error] = "AI service error: #{exception.message}"
-    redirect_to chats_path
-  end
-end
-```
-
-### Background Job Level
-
-```ruby
-class AiChatJob < ApplicationJob
-  retry_on RubyLLM::RateLimitError, RubyLLM::ServiceUnavailableError,
-           wait: :exponentially_longer, attempts: 5
-
-  discard_on RubyLLM::UnauthorizedError, RubyLLM::BadRequestError
-
-  def perform(chat_id, message)
-    chat = Chat.find(chat_id)
-    chat.ask(message)
-  rescue RubyLLM::Error => e
-    # Log error and notify user
-    ErrorNotifier.notify(chat.user, "AI chat error: #{e.message}")
-  end
-end
-```
-
-## Monitoring Errors
-
-For production applications, monitor AI service errors:
-
-```ruby
-# Custom error handler
-module AiErrorMonitoring
-  def self.track_error(error, context = {})
-    # Record error in your monitoring system
-    Sentry.capture_exception(error, extra: context)
-
-    # Log details
-    Rails.logger.error "[AI Error] #{error.class}: #{error.message}"
-    Rails.logger.error "Context: #{context.inspect}"
-
-    # Return or re-raise as needed
-    error
-  end
-end
-
-# Usage
-begin
-  chat.ask "Some question"
-rescue RubyLLM::Error => e
-  AiErrorMonitoring.track_error(e, {
-    model: chat.model.id,
-    tokens: chat.messages.sum(&:input_tokens)
-  })
-
-  # Show appropriate message to user
-  flash[:error] = "Sorry, we encountered an issue with our AI service"
-end
-```
-
-## Graceful Degradation
-
-For critical applications, implement fallback strategies:
-
-```ruby
-def get_ai_response(question, fallback_message = nil)
-  begin
-    chat = RubyLLM.chat
-    response = chat.ask(question)
-    response.content
-  rescue RubyLLM::Error => e
-    Rails.logger.error "AI error: #{e.message}"
-
-    # Fallback to alternative model
-    begin
-      fallback_chat = RubyLLM.chat(model: 'gpt-3.5-turbo')
-      fallback_response = fallback_chat.ask(question)
-      fallback_response.content
-    rescue RubyLLM::Error => e2
-      Rails.logger.error "Fallback AI error: #{e2.message}"
-      fallback_message || "Sorry, our AI service is currently unavailable"
-    end
-  end
-end
-```
-
-## Best Practices
-
-1. **Always wrap AI calls in error handling** - Don't assume AI services will always be available
-2. **Implement timeouts** - Configure appropriate request timeouts
-3. **Use background jobs** - Process AI requests asynchronously when possible
-4. **Set up monitoring** - Track error rates and response times
-5. **Have fallback content** - Prepare fallback responses when AI services fail
-6. **Gracefully degrade** - Implement multiple fallback strategies
-7. **Communicate to users** - Provide clear error messages when AI services are unavailable
-
-## Error Recovery
-
-When dealing with errors, consider recovery strategies:
-
-```ruby
-MAX_RETRIES = 3
-
-def ask_with_recovery(chat, question, retries = 0)
-  chat.ask(question)
-rescue RubyLLM::RateLimitError, RubyLLM::ServiceUnavailableError => e
-  if retries < MAX_RETRIES
-    # Exponential backoff
-    sleep_time = 2 ** retries
-    puts "Error: #{e.message}. Retrying in #{sleep_time} seconds..."
-    sleep sleep_time
-    ask_with_recovery(chat, question, retries + 1)
-  else
-    raise e
-  end
-end
-```
-
-## Next Steps
-
-Now that you understand error handling in RubyLLM, you might want to explore:
-
-- [Rails Integration]({% link guides/rails.md %}) for using RubyLLM in Rails applications
-- [Tools]({% link guides/tools.md %}) for using tools with error handling