rainbow_llm 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 0b7f668ffcf5f6b67b910f729dd10670908264762798119e3fb0d21b4e27995f
+  data.tar.gz: d0f42404d0d702e8c720b1e5febc8b17fcf02e626d867acd600cbc4ad1fedff0
+SHA512:
+  metadata.gz: fcffac6222cdefa8255eb457913e802621fa262cac2d72ecac56d8e4031b18d71fb345d260911ddebe44b326d5f348ba00f6c33f4cae9df20b3dad662ff686d8
+  data.tar.gz: dc7598c47b9cf858a4e41be22c5b268f8206206f8451f657fc0ef20db25b137e8d51a9fc3019aac44fc184a87ce05bfd40d55954c326cb1d4bb8604372be199b

data/.rspec

@@ -0,0 +1,3 @@
+--require spec_helper
+--format documentation
+--color

data/CHANGELOG.md

@@ -0,0 +1,5 @@
+## [Unreleased]
+
+## [0.1.0] - 2026-01-31
+
+- Initial release

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 a-chris
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,269 @@
+# 🌈 RainbowLLM
+
+Sponsored by
+<a href="https://coney.app" target="_blank"><img src="https://www.google.com/s2/favicons?domain=coney.app&sz=32" alt="Coney.app" width="20" height="20"> Coney.app</a> <a href="https://rubycon.it" target="_blank"><img src="https://www.google.com/s2/favicons?domain=rubycon.it&sz=32" alt="Rubycon.it" width="20" height="20"> Rubycon.it</a>
+
+[![Gem Version](https://badge.fury.io/rb/rainbow_llm.svg)](https://badge.fury.io/rb/rainbow_llm) [![MIT License](https://img.shields.io/badge/license-MIT-blue.svg)](https://opensource.org/licenses/MIT) [![Ruby](https://img.shields.io/badge/ruby-%3E%3D%203.0-red.svg)](https://www.ruby-lang.org/)
+
+RainbowLLM is your **AI resilience layer** - a smart routing gem that gives you:
+
+- **Automatic failover**: When one provider fails, it instantly tries the next. Your application stays online even when providers go down
+- **Maximize free tiers usage and avoid vendor lock-in**
+- **Easy integration**: Simple Ruby interface based on RubyLLM
+
+```ruby
+# If OpenAI fails, automatically tries Anthropic, then Ollama
+response = RainbowLLM.chat(
+  models: ["openai/gpt-5", "anthropic/claude-3.5-sonnet", "ollama/llama3.3"]
+).ask(user_question)
+```
+
+## 🚀 Quick Start
+
+```bash
+bundle add rainbow_llm
+```
+
+## 📖 Table of Contents
+
+- [Features](#-features)
+- [Installation](#-installation)
+- [Configuration](#-configuration)
+- [Usage Examples](#-usage-examples)
+- [Advanced Patterns](#-advanced-patterns)
+- [Development](#-development)
+- [Contributing](#-contributing)
+- [License](#-license)
+
+## Features
+
+- **Automatic failover** - Tries providers in your specified order with instant fallback when a provider fails, configurable retry logic and timeouts
+- **Cost optimization** - Route to the most cost-effective provider, maximize free tier usage across providers, and avoid rate limit surprises
+- **Flexible configuration** - Support for OpenAI-compatible endpoints, Basic Auth and API key authentication, custom endpoints and model mappings
+- **Production ready** - Built on the reliable **ruby_llm** foundation with comprehensive error handling and detailed logging and monitoring
+
+## 📦 Installation
+
+### Option 1: With Bundler (Recommended)
+
+```bash
+bundle add rainbow_llm
+```
+
+### Option 2: Direct Install
+
+```bash
+ gem install rainbow_llm
+```
+
+### Requirements
+
+- Ruby 3.2+
+
+## ⚙️ Configuration
+
+Configure your providers once, use them everywhere:
+
+```ruby
+# config/initializers/rainbow_llm.rb
+RainbowLLM.configure do |config|
+  # Local Ollama instance
+  config.provider :ollama, {
+    provider: "openai_basic",
+    uri_base: "http://localhost:11434/v1",
+    access_token: ENV['OLLAMA_API_KEY'],
+    assume_model_exists: true
+  }
+
+  # Cerebras cloud service
+  config.provider :cerebras, {
+    provider: "openai",
+    uri_base: "https://api.cerebras.ai/v1",
+    access_token: ENV['CEREBRAS_API_KEY'],
+    assume_model_exists: true
+  }
+
+  # Add as many providers as you need!
+  config.provider :openai, {
+    provider: "openai",
+    access_token: ENV['OPENAI_API_KEY']
+  }
+end
+```
+
+## 💡 Usage Examples
+
+### Basic Chat Completion
+
+```ruby
+response = RainbowLLM.chat(
+  models: ["ollama/llama3.3", "cerebras/llama-3.3-70b"]
+).ask("Explain quantum computing to a 5-year-old")
+
+puts response.content
+# => "Imagine you have a magical toy that can be in two places at once..."
+```
+
+### Fluent API Options
+
+Chain options together for fine-grained control:
+
+```ruby
+# Set temperature and timeout
+response = RainbowLLM.chat(models: ["ollama/llama3.3"])
+  .with_temperature(0.8)
+  .with_timeout(30)
+  .ask("Write a creative story")
+
+# Use JSON schema for structured output
+response = RainbowLLM.chat(models: ["cerebras/llama-3.3-70b"])
+  .with_schema(MySchema)
+  .ask("Extract data from this text")
+
+# Chain multiple options
+response = RainbowLLM.chat(models: ["openai/gpt-5", "cerebras/llama-3.3-70b"])
+  .with_temperature(0.7)
+  .with_timeout(45)
+  .with_schema(ResponseSchema)
+  .ask("Analyze this document")
+```
+
+**Response Object:**
+
+```ruby
+# Check which model succeeded
+response.model
+# => "cerebras/llama-3.3-70b" (or nil if all failed)
+
+# Get the content
+response.content
+# => "The analysis results..." (or nil if all failed)
+
+# Inspect detailed status for each model
+response.details
+# => {
+#      "ollama/llama3.3" => { status: :failed, error: "Connection refused" },
+#      "cerebras/llama-3.3-70b" => { status: :success, duration: 1.23 }
+#    }
+```
+
+### Error Handling
+
+RainbowLLM doesn't raise exceptions - it returns a Response with details:
+
+```ruby
+response = RainbowLLM.chat(
+  models: ["primary-model", "backup-model-1", "backup-model-2"]
+).ask("Important business question")
+
+if response.content
+  puts "Success: #{response.content}"
+  puts "Provided by: #{response.model}"
+else
+  # All providers failed - inspect details to understand why
+  puts "All providers failed!"
+
+  response.details.each do |model, info|
+    puts "#{model}: #{info[:status]} - #{info[:error]}"
+  end
+  # => primary-model: failed - Rate limit exceeded
+  # => backup-model-1: failed - Connection timeout
+  # => backup-model-2: failed - Invalid API key
+end
+```
+
+## 🎯 Advanced Patterns
+
+### Cost-Based Routing
+
+```ruby
+# Route to cheapest available provider first
+cheap_models = [
+  "ollama/llama3.2-3b",   # Free (local)
+  "cerebras/llama-3.3-70b", # Free tier
+  "openai/gpt-5"    # Paid fallback
+]
+
+response = RainbowLLM.chat(models: cheap_models)
+  .with_temperature(0.5)
+  .ask(user_input)
+```
+
+### Performance-Based Routing
+
+```ruby
+# Route to fastest providers for time-sensitive requests
+fast_models = [
+  "cerebras/llama-3.3-70b",  # Fast cloud
+  "openai/gpt-5",         # Fast but more expensive
+  "ollama/llama3.2"        # Local but slower
+]
+
+response = RainbowLLM.chat(models: fast_models)
+  .with_timeout(1)
+  .ask(time_sensitive_question)
+```
+
+### Multi-Provider Load Balancing
+
+```ruby
+# Distribute load across providers to avoid rate limits
+providers = [
+  "openai/model-1",
+  "anthropic/model-2",
+  "cerebras/model-3",
+  "ollama/model-4"
+]
+
+# RainbowLLM will try each in order until one succeeds
+response = RainbowLLM.chat(models: providers)
+  .with_temperature(0.7)
+  .ask(request)
+```
+
+## 🔧 Development
+
+Want to contribute or run tests locally?
+
+```bash
+# Clone the repo
+git clone https://github.com/a-chris/rainbow_llm.git
+cd rainbow_llm
+
+# Install dependencies
+bin/setup
+
+# Run tests
+rake test
+
+# Launch interactive console
+bin/console
+
+# Install locally
+bundle exec rake install
+```
+
+## 🤝 Contributing
+
+We welcome contributions! Here's how you can help:
+
+- **Report bugs**: Open an issue with detailed reproduction steps
+- **Suggest features**: What would make RainbowLLM even better?
+- **Submit pull requests**: Fix bugs, add features, improve docs
+- **Spread the word**: Star the repo, share with friends!
+
+**Development setup**:
+
+```bash
+# After cloning
+bin/setup  # Installs dependencies
+rake test # Runs the test suite
+```
+
+## 📜 License
+
+RainbowLLM is open source software licensed under the [MIT License](https://opensource.org/licenses/MIT).
+
+---
+
+**Need help?** Open an issue or contact [@a-chris](https://github.com/a-chris)

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rspec/core/rake_task"
+
+RSpec::Core::RakeTask.new
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+task default: %i[spec rubocop]

data/lib/rainbow_llm/chat_builder.rb

@@ -0,0 +1,110 @@
+# frozen_string_literal: true
+
+module RainbowLLM
+  # Fluent API builder for chat requests with automatic failover across models
+  class ChatBuilder
+    attr_reader :models, :temperature, :schema, :timeout
+
+    def initialize(models)
+      @models = models
+      @temperature = nil
+      @schema = nil
+      @timeout = nil
+    end
+
+    def with_temperature(value)
+      dup.tap { |builder| builder.instance_variable_set(:@temperature, value) }
+    end
+
+    def with_schema(value)
+      dup.tap { |builder| builder.instance_variable_set(:@schema, value) }
+    end
+
+    def with_timeout(seconds)
+      dup.tap { |builder| builder.instance_variable_set(:@timeout, seconds) }
+    end
+
+    def ask(prompt)
+      raise InvalidArgument, "prompt cannot be nil or empty" if prompt.nil? || prompt.to_s.strip.empty?
+
+      details = {}
+
+      models.each do |model_string|
+        # Split model string into provider and model parts
+        provider, model = model_string.split("/", 2)
+
+        if model.nil? || model.strip.empty?
+          details[model_string] = {
+            status: :error,
+            error: "Invalid model format: '#{model_string}'. Expected format: 'provider/model_name'"
+          }
+          next
+        end
+
+        context =
+          if RainbowLLM.configuration.providers.key?(provider.to_sym)
+            RubyLLM.context do |c|
+              c.request_timeout = @timeout if @timeout
+              c.openai_api_base = RainbowLLM.configuration.providers.dig(provider.to_sym, :uri_base)
+              c.openai_api_key = RainbowLLM.configuration.providers.dig(provider.to_sym, :access_token)
+            end
+          elsif @timeout
+            RubyLLM.context do |c|
+              c.request_timeout = @timeout if @timeout
+            end
+          end
+
+        provider_type = RainbowLLM.configuration.providers.dig(provider.to_sym, :provider) || provider
+        assume_model_exists = RainbowLLM.configuration.providers.dig(provider.to_sym, :assume_model_exists)
+
+        base = {model:, provider: provider_type, assume_model_exists:, context:}.compact
+
+        opts = {}
+        opts[:temperature] = temperature if temperature
+        opts[:schema] = schema if schema
+
+        result =
+          begin
+            chat = RubyLLM.chat(**base.merge(opts))
+            response = chat.ask(prompt)
+            content = RainbowLLM.check_content(RainbowLLM.extract_content(response))
+
+            if content
+              details[model_string] = {status: :success}
+              {content:, model: model_string, status: :success}
+            else
+              details[model_string] = {status: :empty}
+              nil
+            end
+          rescue Faraday::TimeoutError => e
+            warn "Provider #{provider} failed with timeout error: #{e.message}"
+            details[model_string] = {status: :timeout, error: e.message}
+            nil
+          rescue RubyLLM::Error => e
+            # Log the error for debugging purposes
+            warn "Provider #{provider} failed with error: #{e.message}"
+            details[model_string] = {status: :error, error: e.message}
+            nil
+          end
+
+        if result && result[:status] == :success
+          return Response.new(content: result[:content], model: result[:model], details:)
+        end
+      end
+
+      # If we get here, all providers failed
+      Response.new(content: nil, model: nil, details:)
+    end
+  end
+
+  def self.extract_content(response)
+    return nil if response.nil?
+
+    response.content.strip
+  end
+
+  def self.check_content(content)
+    return nil if content.nil? || content.to_s.strip.empty?
+    content
+  end
+end

data/lib/rainbow_llm/configuration.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+module RainbowLLM
+  class Configuration
+    attr_accessor :providers
+
+    def initialize
+      @providers = {}
+    end
+
+    def provider(name, config)
+      @providers[name.to_sym] = config
+    end
+  end
+end

data/lib/rainbow_llm/errors.rb

@@ -0,0 +1,22 @@
+module RainbowLLM
+  # Base error class for all RainbowLLM errors
+  class Error < StandardError; end
+
+  # Raised when provided arguments are invalid or missing
+  class InvalidArgument < Error; end
+
+  # Raised when provider configuration is missing or invalid
+  class ConfigurationError < Error; end
+
+  # Raised when a specified provider cannot be found
+  class ProviderNotFoundError < Error; end
+
+  # Raised when a specified model cannot be found
+  class ModelNotFoundError < Error; end
+
+  # Raised when all providers fail to respond
+  class AllProvidersFailedError < Error; end
+
+  # Raised when model format is invalid
+  class ModelError < Error; end
+end

data/lib/rainbow_llm/providers/basic_auth_openai.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+require "ruby_llm"
+
+module RainbowLLM
+  module Providers
+    class BasicAuthOpenAI < RubyLLM::Providers::OpenAI
+      def headers
+        {"Authorization" => "Basic #{@config.openai_api_key}"}
+      end
+    end
+  end
+end
+
+# Register the provider with RubyLLM
+RubyLLM::Provider.register :openai_basic, RainbowLLM::Providers::BasicAuthOpenAI

data/lib/rainbow_llm/response.rb

@@ -0,0 +1 @@
+Response = Data.define(:content, :model, :details)

data/lib/rainbow_llm/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module RainbowLLM
+  VERSION = "0.1.0"
+end

data/lib/rainbow_llm.rb

@@ -0,0 +1,29 @@
+# frozen_string_literal: true
+
+require "ruby_llm"
+require_relative "rainbow_llm/version"
+require_relative "rainbow_llm/configuration"
+require_relative "rainbow_llm/errors"
+require_relative "rainbow_llm/response"
+require_relative "rainbow_llm/chat_builder"
+require_relative "rainbow_llm/providers/basic_auth_openai"
+
+# RainbowLLM - A routing gem for multiple LLM providers with automatic failover
+module RainbowLLM
+  def self.configuration
+    @configuration ||= Configuration.new
+  end
+
+  def self.configuration=(config)
+    @configuration = config
+  end
+
+  def self.configure
+    yield(configuration)
+  end
+
+  def self.chat(models:)
+    raise InvalidArgument, "models cannot be nil or empty" if models.nil? || models.empty?
+    ChatBuilder.new(models)
+  end
+end

data/mise.toml

@@ -0,0 +1,2 @@
+[tools]
+ruby = "4.0.0"

metadata

@@ -0,0 +1,100 @@
+--- !ruby/object:Gem::Specification
+name: rainbow_llm
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- a-chris
+bindir: exe
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: ruby_llm
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+- !ruby/object:Gem::Dependency
+  name: rspec-mocks
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '3.12'
+description: RainbowLLM provides intelligent routing and failover capabilities for
+  multiple LLM providers. Automatically tries providers in sequence until one responds
+  successfully, ensuring reliable access to LLM services.
+email:
+- a.christian.toscano@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".rspec"
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- Rakefile
+- lib/rainbow_llm.rb
+- lib/rainbow_llm/chat_builder.rb
+- lib/rainbow_llm/configuration.rb
+- lib/rainbow_llm/errors.rb
+- lib/rainbow_llm/providers/basic_auth_openai.rb
+- lib/rainbow_llm/response.rb
+- lib/rainbow_llm/version.rb
+- mise.toml
+homepage: https://github.com/a-chris/rainbow_llm
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/a-chris/rainbow_llm
+  source_code_uri: https://github.com/a-chris/rainbow_llm
+  changelog_uri: https://github.com/a-chris/rainbow_llm/blob/main/CHANGELOG.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 3.2.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.3
+specification_version: 4
+summary: A routing gem for multiple LLM providers with automatic failover.
+test_files: []