ruby_llm 0.1.0.pre36 → 0.1.0.pre37

data/docs/guides/rails.md

@@ -0,0 +1,401 @@
+---
+layout: default
+title: Rails Integration
+parent: Guides
+nav_order: 5
+permalink: /guides/rails
+---
+
+# Rails Integration
+
+RubyLLM provides seamless integration with Rails through ActiveRecord models. This allows you to easily persist chats, messages, and tool calls in your database.
+
+## Setup
+
+### 1. Create Migrations
+
+First, create the necessary tables in your database:
+
+```ruby
+# db/migrate/YYYYMMDDHHMMSS_create_chats.rb
+class CreateChats < ActiveRecord::Migration[8.0]
+  def change
+    create_table :chats do |t|
+      t.string :model_id
+      t.timestamps
+    end
+  end
+end
+
+# db/migrate/YYYYMMDDHHMMSS_create_messages.rb
+class CreateMessages < ActiveRecord::Migration[8.0]
+  def change
+    create_table :messages do |t|
+      t.references :chat, null: false, foreign_key: true
+      t.string :role
+      t.text :content
+      t.string :model_id
+      t.integer :input_tokens
+      t.integer :output_tokens
+      t.references :tool_call
+      t.timestamps
+    end
+  end
+end
+
+# db/migrate/YYYYMMDDHHMMSS_create_tool_calls.rb
+class CreateToolCalls < ActiveRecord::Migration[8.0]
+  def change
+    create_table :tool_calls do |t|
+      t.references :message, null: false, foreign_key: true
+      t.string :tool_call_id, null: false
+      t.string :name, null: false
+      t.jsonb :arguments, default: {}
+      t.timestamps
+    end
+
+    add_index :tool_calls, :tool_call_id
+  end
+end
+```
+
+Run the migrations:
+
+```bash
+rails db:migrate
+```
+
+### 2. Set Up Models
+
+Create the model classes:
+
+```ruby
+# app/models/chat.rb
+class Chat < ApplicationRecord
+  acts_as_chat
+end
+
+# app/models/message.rb
+class Message < ApplicationRecord
+  acts_as_message
+end
+
+# app/models/tool_call.rb
+class ToolCall < ApplicationRecord
+  acts_as_tool_call
+end
+```
+
+### 3. Configure RubyLLM
+
+In an initializer (e.g., `config/initializers/ruby_llm.rb`):
+
+```ruby
+RubyLLM.configure do |config|
+  config.openai_api_key = ENV['OPENAI_API_KEY']
+  config.anthropic_api_key = ENV['ANTHROPIC_API_KEY']
+  config.gemini_api_key = ENV['GEMINI_API_KEY']
+  config.deepseek_api_key = ENV['DEEPSEEK_API_KEY']
+end
+```
+
+## Basic Usage
+
+Once your models are set up, you can use them like any other Rails model:
+
+```ruby
+# Create a new chat
+chat = Chat.create!(model_id: 'gpt-4o-mini')
+
+# Ask a question
+chat.ask "What's the capital of France?"
+
+# The response is automatically persisted
+puts chat.messages.last.content
+
+# Continue the conversation
+chat.ask "Tell me more about that city"
+
+# All messages are stored in the database
+chat.messages.order(:created_at).each do |message|
+  puts "#{message.role}: #{message.content}"
+end
+```
+
+## Streaming Responses
+
+You can stream responses while still persisting the final result:
+
+```ruby
+chat = Chat.create!(model_id: 'gpt-4o-mini')
+
+chat.ask "Write a short poem about Ruby" do |chunk|
+  # Stream content to the user
+  ActionCable.server.broadcast "chat_#{chat.id}", { content: chunk.content }
+end
+
+# The complete message is saved in the database
+puts chat.messages.last.content
+```
+
+## Using with Hotwire
+
+RubyLLM's Rails integration works seamlessly with Hotwire for real-time updates:
+
+```ruby
+# app/models/chat.rb
+class Chat < ApplicationRecord
+  acts_as_chat
+
+  # Add broadcast capabilities
+  broadcasts_to ->(chat) { "chat_#{chat.id}" }
+end
+```
+
+In your controller:
+
+```ruby
+# app/controllers/chats_controller.rb
+class ChatsController < ApplicationController
+  def show
+    @chat = Chat.find(params[:id])
+  end
+
+  def ask
+    @chat = Chat.find(params[:id])
+
+    # Use a background job to avoid blocking
+    ChatJob.perform_later(@chat.id, params[:message])
+
+    # Let the user know we're working on it
+    respond_to do |format|
+      format.turbo_stream
+      format.html { redirect_to @chat }
+    end
+  end
+end
+```
+
+Create a background job:
+
+```ruby
+# app/jobs/chat_job.rb
+class ChatJob < ApplicationJob
+  queue_as :default
+
+  def perform(chat_id, message)
+    chat = Chat.find(chat_id)
+
+    # Start with a "typing" indicator
+    Turbo::StreamsChannel.broadcast_append_to(
+      chat,
+      target: "messages",
+      partial: "messages/typing"
+    )
+
+    chat.ask(message) do |chunk|
+      # Remove typing indicator after first chunk
+      if chunk == chat.messages.last.to_llm.content[0...chunk.content.length]
+        Turbo::StreamsChannel.broadcast_remove_to(
+          chat,
+          target: "typing"
+        )
+      end
+
+      # Update the streaming message
+      Turbo::StreamsChannel.broadcast_replace_to(
+        chat,
+        target: "assistant_message_#{chat.messages.last.id}",
+        partial: "messages/message",
+        locals: { message: chat.messages.last, content: chunk.content }
+      )
+    end
+  end
+end
+```
+
+In your views:
+
+```erb
+<!-- app/views/chats/show.html.erb -->
+<%= turbo_stream_from @chat %>
+
+<div id="messages">
+  <%= render @chat.messages %>
+</div>
+
+<%= form_with(url: ask_chat_path(@chat), method: :post) do |f| %>
+  <%= f.text_area :message %>
+  <%= f.submit "Send" %>
+<% end %>
+```
+
+## Using Tools
+
+Tools work seamlessly with Rails integration:
+
+```ruby
+class Calculator < RubyLLM::Tool
+  description "Performs arithmetic calculations"
+
+  param :expression,
+    type: :string,
+    desc: "Math expression to evaluate"
+
+  def execute(expression:)
+    eval(expression).to_s
+  rescue StandardError => e
+    "Error: #{e.message}"
+  end
+end
+
+# Add the tool to your chat
+chat = Chat.create!(model_id: 'gpt-4o-mini')
+chat.with_tool(Calculator)
+
+# Ask a question that requires calculation
+chat.ask "What's 123 * 456?"
+
+# Tool calls are persisted
+tool_call = chat.messages.find_by(role: 'assistant').tool_calls.first
+puts "Tool: #{tool_call.name}"
+puts "Arguments: #{tool_call.arguments}"
+```
+
+## Customizing Models
+
+You can customize the behavior of your models:
+
+```ruby
+class Chat < ApplicationRecord
+  acts_as_chat
+
+  # Add custom behavior
+  belongs_to :user
+  has_many :tags
+
+  # Add custom scopes
+  scope :recent, -> { order(created_at: :desc).limit(10) }
+  scope :by_model, ->(model_id) { where(model_id: model_id) }
+
+  # Add custom methods
+  def summarize
+    self.ask "Please summarize our conversation so far."
+  end
+
+  def token_count
+    messages.sum { |m| (m.input_tokens || 0) + (m.output_tokens || 0) }
+  end
+end
+```
+
+## Message Content Customization
+
+You can customize how message content is stored or extracted:
+
+```ruby
+class Message < ApplicationRecord
+  acts_as_message
+
+  # Override content handling
+  def extract_content
+    # For example, compress or expand content
+    JSON.parse(content) rescue content
+  end
+end
+```
+
+## Advanced Integration
+
+### User Association
+
+Associate chats with users:
+
+```ruby
+# Migration
+add_reference :chats, :user, foreign_key: true
+
+# Model
+class Chat < ApplicationRecord
+  acts_as_chat
+  belongs_to :user
+end
+
+# Usage
+user.chats.create!(model_id: 'gpt-4o-mini').ask("Hello!")
+```
+
+### Metadata and Tagging
+
+Add metadata to chats:
+
+```ruby
+# Migration
+add_column :chats, :metadata, :jsonb, default: {}
+
+# Model
+class Chat < ApplicationRecord
+  acts_as_chat
+end
+
+# Usage
+chat = Chat.create!(
+  model_id: 'gpt-4o-mini',
+  metadata: {
+    purpose: 'customer_support',
+    category: 'billing',
+    priority: 'high'
+  }
+)
+```
+
+### Scoping and Filtering
+
+Create scopes for easier querying:
+
+```ruby
+class Chat < ApplicationRecord
+  acts_as_chat
+
+  scope :using_gpt, -> { where("model_id LIKE ?", "gpt-%") }
+  scope :using_claude, -> { where("model_id LIKE ?", "claude-%") }
+  scope :recent, -> { order(created_at: :desc).limit(10) }
+  scope :with_high_token_count, -> {
+    joins(:messages)
+    .group(:id)
+    .having("SUM(messages.input_tokens + messages.output_tokens) > ?", 10000)
+  }
+end
+```
+
+## Performance Considerations
+
+For high-volume applications:
+
+1. **Background Processing**: Use background jobs for AI requests
+2. **Connection Pooling**: Ensure your database connection pool is sized appropriately
+3. **Pagination**: Use pagination when showing chat histories
+4. **Archiving**: Consider archiving old chats to maintain performance
+
+```ruby
+# Example background job
+class AskAiJob < ApplicationJob
+  queue_as :ai_requests
+
+  def perform(chat_id, message)
+    chat = Chat.find(chat_id)
+    chat.ask(message)
+  end
+end
+
+# Usage
+AskAiJob.perform_later(chat.id, "Tell me about Ruby")
+```
+
+## Next Steps
+
+Now that you've integrated RubyLLM with Rails, you might want to explore:
+
+- [Using Tools]({% link guides/tools.md %}) to add capabilities to your chats
+- [Streaming Responses]({% link guides/streaming.md %}) for a better user experience
+- [Error Handling]({% link guides/error-handling.md %}) to handle AI service issues gracefully

data/docs/guides/streaming.md

@@ -0,0 +1,242 @@
+---
+layout: default
+title: Streaming
+parent: Guides
+nav_order: 4
+permalink: /guides/streaming
+---
+
+# Streaming Responses
+
+RubyLLM provides streaming capabilities that allow you to receive AI responses in real-time as they're being generated, rather than waiting for the complete response. This creates a more interactive experience and is especially useful for long responses or applications with real-time UI updates.
+
+## Basic Streaming
+
+To stream responses, simply provide a block to the `ask` method:
+
+```ruby
+chat = RubyLLM.chat
+
+chat.ask "Write a short story about a programmer" do |chunk|
+  # Each chunk contains a portion of the response
+  print chunk.content
+  $stdout.flush  # Ensure content is displayed immediately
+end
+```
+
+## Understanding Chunks
+
+Each streamed chunk is an instance of `RubyLLM::Chunk` (which inherits from `RubyLLM::Message`) and provides:
+
+```ruby
+chunk.content       # The text fragment in this chunk
+chunk.role          # Always :assistant for streamed chunks
+chunk.model_id      # The model generating the response
+chunk.input_tokens  # Input token count (usually only in the final chunk)
+chunk.output_tokens # Output token count (usually only in the final chunk)
+```
+
+## Accumulated Response
+
+Even when streaming, RubyLLM still returns the complete final message:
+
+```ruby
+final_message = chat.ask "Write a poem" do |chunk|
+  print chunk.content
+end
+
+# You can use the final message as normal
+puts "\nFinal message length: #{final_message.content.length}"
+puts "Token usage: #{final_message.output_tokens} tokens"
+```
+
+## Web Application Integration
+
+### Rails with ActionCable
+
+```ruby
+# In your controller
+def ask
+  @chat = Chat.find(params[:id])
+
+  @chat.ask(params[:message]) do |chunk|
+    ActionCable.server.broadcast(
+      "chat_#{@chat.id}",
+      { content: chunk.content }
+    )
+  end
+
+  head :ok
+end
+
+# In your JavaScript
+const channel = consumer.subscriptions.create({ channel: "ChatChannel", id: chatId }, {
+  received(data) {
+    // Append incoming chunk to the display
+    document.getElementById('response').innerHTML += data.content;
+  }
+});
+```
+
+### Rails with Turbo Streams
+
+```ruby
+class ChatJob < ApplicationJob
+  queue_as :default
+
+  def perform(chat_id, message)
+    chat = Chat.find(chat_id)
+
+    chat.ask(message) do |chunk|
+      Turbo::StreamsChannel.broadcast_update_to(
+        "chat_#{chat.id}",
+        target: "response",
+        html: chunk.content,
+        append: true
+      )
+    end
+  end
+end
+```
+
+### Sinatra with Server-Sent Events (SSE)
+
+```ruby
+get '/chat/:id/ask' do
+  content_type 'text/event-stream'
+
+  chat = Chat.find(params[:id])
+
+  chat.ask(params[:message]) do |chunk|
+    # Send chunk as SSE event
+    out << "data: #{chunk.content}\n\n"
+  end
+
+  # Send completion signal
+  out << "event: complete\ndata: {}\n\n"
+end
+```
+
+## Error Handling
+
+Errors that occur during streaming need special handling:
+
+```ruby
+begin
+  chat.ask("Tell me a story") do |chunk|
+    print chunk.content
+  end
+rescue RubyLLM::Error => e
+  puts "\nError during streaming: #{e.message}"
+end
+```
+
+Common errors during streaming:
+
+- `ServiceUnavailableError` - The AI service is temporarily unavailable
+- `RateLimitError` - You've exceeded your API rate limit
+- `BadRequestError` - There was a problem with your request parameters
+
+## Provider-Specific Considerations
+
+### OpenAI
+
+OpenAI's streaming implementation provides small, frequent chunks for a smooth experience.
+
+### Anthropic
+
+Claude models may return slightly larger chunks with potentially longer pauses between them.
+
+### Google Gemini
+
+Gemini streaming is highly responsive but may show slightly different chunking behavior.
+
+## Streaming with Tools
+
+When using tools, streaming works a bit differently:
+
+```ruby
+chat.with_tool(Calculator)
+   .ask("What's 123 * 456?") do |chunk|
+     # Tool call execution isn't streamed
+     # You'll receive chunks after tool execution completes
+     print chunk.content
+   end
+```
+
+The tool call execution introduces a pause in the streaming, as the model waits for the tool response before continuing.
+
+## Performance Considerations
+
+Streaming typically uses the same number of tokens as non-streaming responses but establishes longer-lived connections to the AI provider. Consider these best practices:
+
+1. Set appropriate timeouts for streaming connections
+2. Handle network interruptions gracefully
+3. Consider background processing for long-running streams
+4. Implement rate limiting to avoid overwhelming your servers
+
+## Tracking Token Usage
+
+Token usage information is typically only available in the final chunk or completed message:
+
+```ruby
+total_tokens = 0
+
+chat.ask("Write a detailed explanation of quantum computing") do |chunk|
+  print chunk.content
+
+  # Only count tokens in the final chunk
+  if chunk.output_tokens
+    total_tokens = chunk.input_tokens + chunk.output_tokens
+  end
+end
+
+puts "\nTotal tokens: #{total_tokens}"
+```
+
+## Custom Processing of Streamed Content
+
+You can process streamed content in real-time:
+
+```ruby
+accumulated_text = ""
+
+chat.ask("Write a list of 10 fruits") do |chunk|
+  new_content = chunk.content
+  accumulated_text += new_content
+
+  # Count fruits as they come in
+  if new_content.include?("\n")
+    fruit_count = accumulated_text.scan(/\d+\./).count
+    print "\rFruits listed: #{fruit_count}/10"
+  end
+end
+```
+
+## Rails Integration
+
+When using RubyLLM's Rails integration with `acts_as_chat`, streaming still works seamlessly:
+
+```ruby
+class Chat < ApplicationRecord
+  acts_as_chat
+end
+
+chat = Chat.create!(model_id: 'gpt-4o-mini')
+
+# Stream responses while persisting the final result
+chat.ask("Tell me about Ruby") do |chunk|
+  ActionCable.server.broadcast("chat_#{chat.id}", { content: chunk.content })
+end
+
+# The complete message is saved in the database
+puts chat.messages.last.content
+```
+
+## Next Steps
+
+Now that you understand streaming, you might want to explore:
+
+- [Using Tools]({% link guides/tools.md %}) to add capabilities to your AI interactions
+- [Rails Integration]({% link guides/rails.md %}) to persist conversations
+- [Error Handling]({% link guides/error-handling.md %}) for reliable applications