ranked_llm 0.2.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: c9609e477e95f6b743ca320ed83d57e5c93fcc85bdb44fed154c61ef31e00509
+  data.tar.gz: af8579c4bea899322880e00582ff5276a1be17d6dfdea41bab04b203763d5d04
+SHA512:
+  metadata.gz: e2b73f9b1bf57fee78149c76242f576824746bd53ee05407d21c991ee5e6ec524214f0291b2c4d6afb46e33b7e73258448014313dcc34d638efc5747468f1477
+  data.tar.gz: f6d85e4dfb92374bc89faaa81a05b24456842a116bd18af4a82c7cddc6f24786e27b561621ca4b382e44abd19683574b0cd0a4082c51a40a67f979008b2006fa

data/CHANGELOG.md

@@ -0,0 +1,34 @@
+# Changelog
+
+All notable changes to this project are documented here. Format loosely follows
+[Keep a Changelog](https://keepachangelog.com/en/1.1.0/).
+
+## [0.2.0] - 2026-07-24
+
+### Added
+
+- Kimi (Moonshot AI) as a provider: `kimi-k3` (1M context, vision), `kimi-k2.7-code`
+  (262k context, coding-focused, no vision), `kimi-k2.6` (262k context, vision).
+  Model IDs, context windows, vision support, and pricing verified against the
+  official docs at platform.kimi.ai. Uses the existing `Llm::OpenAiCompatibleAdapter`
+  — no adapter changes needed, since Moonshot's API supports forced `tool_choice`
+  on a standard chat/completions endpoint.
+
+## [0.1.0] - 2026-07-23
+
+### Added
+
+- Initial extraction from Progentick: ranked, multi-provider AI API credentials
+  with automatic fallback and per-call cost tracking for Rails apps.
+- `Llm::Owned` concern — include in any tenant/owner model (Account, Team, User, ...).
+- `Llm::Client.for(owner).call_tool(...)` — tries ranked credentials in order,
+  falling back to the next on any failure; only vision-capable credentials are
+  attempted when the call includes an image.
+- `Llm::AnthropicAdapter` and `Llm::OpenAiCompatibleAdapter` (shared by OpenAI,
+  DeepSeek, Mistral, OpenRouter, and any other OpenAI-compatible provider).
+- `Llm::ProviderRegistry` — hand-maintained model catalog with per-model pricing
+  and vision flags.
+- `Llm::Credential` (encrypted `api_key`, positioned per-owner) and
+  `Llm::UsageRecord` (token counts + cost per call).
+- Rails generators: `ranked_llm:install` (migrations) and `ranked_llm:views`
+  (scaffolded settings UI — controller, view, drag-to-reorder Stimulus controller).

data/LICENSE.txt

@@ -0,0 +1,19 @@
+Copyright 2026 Jeremy
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,111 @@
+# RankedLlm
+
+A ranked list of AI API credentials — spanning any mix of OpenAI, Anthropic, DeepSeek, Mistral, OpenRouter, etc. —
+tried in order, falling back to the next on **any** failure from the current one. Tracks per-model pricing and logs
+real spend per call, so cost is visible instead of assumed.
+
+Extracted from [Progentick](https://github.com/ohheyjeremy/progentick).
+
+## What it replaces
+
+An app whose AI-calling features are pinned to one hardcoded provider/model/key. If that credential is down,
+rate-limited, or misconfigured, every feature that depends on it just breaks.
+
+## Install
+
+Add to your Gemfile:
+
+```ruby
+gem "ranked_llm", github: "ohheyjeremy/ranked_llm"
+```
+
+Then:
+
+```bash
+bundle install
+bin/rails generate ranked_llm:install
+bin/rails db:migrate
+```
+
+This creates `llm_credentials` and `llm_usage_records` tables, scoped **polymorphically** to whatever your app's
+tenant/owner concept is — the gem never assumes it's called `Account`.
+
+Wire up your owner model:
+
+```ruby
+class Account < ApplicationRecord # or Team, Organization, User, ...
+  include Llm::Owned
+end
+```
+
+If this app doesn't already use Active Record Encryption (`Llm::Credential#api_key` is encrypted at rest with it):
+
+```bash
+bin/rails db:encryption:init
+```
+
+## Usage
+
+```ruby
+result = Llm::Client.for(current_account).call_tool(
+  system: "You triage bug reports.",
+  tool: { name: "triage", description: "...", input_schema: { type: "object", properties: { ... } } },
+  max_tokens: 500,
+  messages: [ { role: "user", content: "..." } ]
+)
+# => { "priority" => "high", ... } — a plain Hash with string keys, the parsed tool-call arguments.
+```
+
+- `messages:` for a full multi-turn conversation, or `content_blocks:` (`{ kind: :text, text: "..." }` /
+  `{ kind: :image, media_type: "...", data: "<base64>" }`) for a single implicit user turn — pass exactly one.
+- If any `content_blocks` entry is `kind: :image`, only vision-capable ranked credentials are attempted.
+- On failure, the client automatically retries the next-ranked credential. Raises
+  `Llm::Client::AllProvidersFailedError` if every configured credential fails, or
+  `Llm::Client::NoCredentialsError` if none are configured (or none are vision-capable when an image is present).
+- Every **successful** call logs an `Llm::UsageRecord` (provider, model, token counts, cost at the time of the
+  call) — failed attempts that got skipped in the fallback chain are never logged.
+
+## Settings UI
+
+There's no mounted engine UI — every host app has its own auth boundary and design system. Instead, scaffold a
+starting point and adapt it:
+
+```bash
+bin/rails generate ranked_llm:views
+```
+
+This copies a controller, view, and Stimulus drag-to-reorder controller into your app. Read the generator's output
+for what to rename (`current_account` → your app's actual auth helper) and restyle.
+
+## Pricing data
+
+`Llm::ProviderRegistry::PROVIDERS` hand-seeds `input_price_per_million`/`output_price_per_million` (USD per 1M
+tokens) per model, current as of when this gem was last updated. **This drifts.** Verify against each provider's
+current pricing page before trusting it for real budgeting decisions. Models with genuinely variable pricing (e.g.
+`openrouter/auto`, a meta-router) have `nil` pricing — `Llm::ProviderRegistry.cost_for` returns `nil` rather than
+`0` for these, so a usage record's `cost_usd` can be `nil` and callers must not treat that as "free."
+
+Adding a new provider/model: edit `PROVIDERS` in `app/services/llm/provider_registry.rb`. Model names are
+deliberately not free-text on `Llm::Credential` — they're validated against this registry, since provider catalogs
+(especially OpenRouter's) move fast and typos are easy.
+
+## Adding a new provider
+
+- If it speaks the OpenAI-style chat-completions shape (function-calling via `tools`/`tool_choice`), just add it to
+  `PROVIDERS` with its `base_uri` — `Llm::OpenAiCompatibleAdapter` handles it.
+- Otherwise (like Anthropic), add a case to `Llm::Client#adapter_for` and a new adapter implementing
+  `#call_tool(system:, tool:, max_tokens:, content_blocks: nil, messages: nil) -> Llm::CallResult`.
+
+## Testing
+
+```bash
+bundle install
+bundle exec rake test
+```
+
+Uses [combustion](https://github.com/pat/combustion) to run the model/service specs against an in-memory dummy
+Rails app (`test/internal`) — no full generated dummy app checked in.
+
+## License
+
+MIT.

data/app/models/llm/credential.rb

@@ -0,0 +1,25 @@
+module Llm
+  # A ranked AI API credential. Llm::Client tries these in position order for
+  # every LLM call, falling back to the next rank on any failure from the
+  # current one.
+  class Credential < ApplicationRecord
+    belongs_to :owner, polymorphic: true
+
+    encrypts :api_key
+
+    positioned on: :owner
+
+    validates :provider, :model, :api_key, presence: true
+    validate :provider_and_model_are_known
+
+    scope :ranked, -> { order(:position) }
+
+    private
+
+    def provider_and_model_are_known
+      return if Llm::ProviderRegistry.valid?(provider, model)
+
+      errors.add(:model, "#{model.inspect} is not a supported model for provider #{provider.inspect}")
+    end
+  end
+end

data/app/models/llm/owned.rb

@@ -0,0 +1,20 @@
+module Llm
+  # Include in whatever model is the tenant/owner concept in the host app
+  # (Account, Team, Organization, User, ...). Credentials and usage records
+  # belong to that model polymorphically, so this gem never has to assume
+  # what the owner is called.
+  #
+  #   class Account < ApplicationRecord
+  #     include Llm::Owned
+  #   end
+  #
+  #   Llm::Client.for(current_account).call_tool(...)
+  module Owned
+    extend ActiveSupport::Concern
+
+    included do
+      has_many :llm_credentials, as: :owner, class_name: "Llm::Credential", dependent: :destroy
+      has_many :llm_usage_records, as: :owner, class_name: "Llm::UsageRecord", dependent: :destroy
+    end
+  end
+end

data/app/models/llm/usage_record.rb

@@ -0,0 +1,22 @@
+module Llm
+  # One row per successful Llm::Client call: which provider/model actually
+  # served it, how many tokens it used, and what that cost at the time (using
+  # Llm::ProviderRegistry's pricing at the moment of the call, not whatever
+  # the registry says later — prices drift, history shouldn't).
+  #
+  # Deliberately keyed by provider/model strings, not a credential_id — a
+  # credential can be reranked or removed, but its usage history shouldn't
+  # disappear with it.
+  class UsageRecord < ApplicationRecord
+    belongs_to :owner, polymorphic: true
+
+    validates :provider, :model, presence: true
+    validates :input_tokens, :output_tokens, numericality: { greater_than_or_equal_to: 0 }
+
+    scope :this_month, -> { where(created_at: Time.current.all_month) }
+
+    def self.total_cost(scope = all)
+      scope.sum(:cost_usd)
+    end
+  end
+end

data/app/models/llm.rb

@@ -0,0 +1,6 @@
+# Multi-provider LLM abstraction: a ranked list of API credentials tried in
+# order, falling back to the next on any failure, shared by every feature in
+# the host app that calls an LLM.
+module Llm
+  def self.table_name_prefix = "llm_"
+end

data/app/services/llm/anthropic_adapter.rb

@@ -0,0 +1,41 @@
+module Llm
+  class AnthropicAdapter
+    def initialize(credential)
+      @credential = credential
+    end
+
+    def call_tool(system:, tool:, max_tokens:, content_blocks: nil, messages: nil)
+      wire_messages = messages || [ { role: "user", content: content_blocks.map { |block| to_wire_block(block) } } ]
+      client = Anthropic::Client.new(api_key: @credential.api_key)
+      response = client.messages.create(
+        model: @credential.model,
+        max_tokens: max_tokens,
+        system: system,
+        messages: wire_messages,
+        tools: [ { name: tool[:name], description: tool[:description], input_schema: tool[:input_schema] } ],
+        tool_choice: { type: "tool", name: tool[:name] }
+      )
+      tool_use = response.content.find { |block| block.type == :tool_use }
+      raise "Llm::AnthropicAdapter: no #{tool[:name]} tool call in response" unless tool_use
+
+      # The anthropic gem parses response JSON with symbolize_names: true, so
+      # tool_use.input has symbol keys — stringify to match
+      # Llm::OpenAiCompatibleAdapter#call_tool's return shape, which callers
+      # rely on via string-key access.
+      CallResult.new(
+        data: tool_use.input.deep_stringify_keys,
+        input_tokens: response.usage.input_tokens,
+        output_tokens: response.usage.output_tokens
+      )
+    end
+
+    private
+
+    def to_wire_block(block)
+      case block[:kind]
+      when :text then { type: "text", text: block[:text] }
+      when :image then { type: "image", source: { type: "base64", media_type: block[:media_type], data: block[:data] } }
+      end
+    end
+  end
+end

data/app/services/llm/call_result.rb

@@ -0,0 +1,7 @@
+module Llm
+  # What an adapter's #call_tool returns internally: the tool-call data a
+  # caller wants, plus token counts so Llm::Client can log spend before
+  # unwrapping to just `data` for the caller (existing callers are unaffected
+  # by this — they never see token counts).
+  CallResult = Struct.new(:data, :input_tokens, :output_tokens, keyword_init: true)
+end

data/app/services/llm/client.rb

@@ -0,0 +1,77 @@
+# Common interface every adapter implements:
+#   #call_tool(system:, tool:, max_tokens:, content_blocks: nil, messages: nil) -> Llm::CallResult
+#
+# Exactly one of content_blocks:/messages: is given. content_blocks is a
+# neutral shape each adapter translates to a single implicit user turn:
+# { kind: :text, text: "..." } / { kind: :image, media_type: "...", data: "<base64>" }.
+# messages is a full multi-turn conversation: [{ role: "user"|"assistant", content: "..." }, ...],
+# for callers that need history, not a single turn.
+# tool is { name:, description:, input_schema: } — plain JSON schema, provider-agnostic.
+#
+# Llm::Client itself has the same #call_tool(...) signature, but returns just
+# the tool-call Hash (string keys) — it unwraps Llm::CallResult#data after
+# logging Llm::UsageRecord from #input_tokens/#output_tokens, so callers never
+# deal with the adapter-level result type.
+#
+# Llm::Client tries the owner's Llm::Credential rows in rank order, falling
+# back to the next on ANY error from the current one. If the call includes an
+# image (content_blocks with a :image block), only vision-capable credentials
+# are attempted — everything else has no such constraint.
+#
+# `owner` is whatever model included Llm::Owned (Account, Team, User, ...).
+module Llm
+  class Client
+    class NoCredentialsError < StandardError; end
+    class AllProvidersFailedError < StandardError; end
+
+    def self.for(owner)
+      new(owner)
+    end
+
+    def initialize(owner)
+      @owner = owner
+    end
+
+    def call_tool(system:, tool:, max_tokens:, content_blocks: nil, messages: nil)
+      needs_vision = content_blocks&.any? { |block| block[:kind] == :image } || false
+
+      credentials = @owner.llm_credentials.ranked.to_a
+      credentials = credentials.select { |c| Llm::ProviderRegistry.vision_capable?(c.provider, c.model) } if needs_vision
+      raise NoCredentialsError, "No configured AI API credential can handle this request" if credentials.empty?
+
+      last_error = nil
+      credentials.each do |credential|
+        result = adapter_for(credential).call_tool(system: system, tool: tool, max_tokens: max_tokens, content_blocks: content_blocks, messages: messages)
+        record_usage(credential, result)
+        return result.data
+      rescue StandardError => e
+        last_error = e
+        Rails.logger.warn("Llm::Client: #{credential.provider}/#{credential.model} failed (#{e.class}: #{e.message}), trying next")
+      end
+
+      raise AllProvidersFailedError, "All #{credentials.size} configured AI API credential(s) failed. Last error: #{last_error.class}: #{last_error.message}"
+    end
+
+    private
+
+    def adapter_for(credential)
+      case credential.provider
+      when "anthropic" then AnthropicAdapter.new(credential)
+      else OpenAiCompatibleAdapter.new(credential)
+      end
+    end
+
+    def record_usage(credential, result)
+      @owner.llm_usage_records.create!(
+        provider: credential.provider,
+        model: credential.model,
+        input_tokens: result.input_tokens,
+        output_tokens: result.output_tokens,
+        cost_usd: ProviderRegistry.cost_for(credential.provider, credential.model, input_tokens: result.input_tokens, output_tokens: result.output_tokens)
+      )
+    rescue StandardError => e
+      # Never let usage logging break a successful call.
+      Rails.logger.error("Llm::Client: failed to record usage for #{credential.provider}/#{credential.model}: #{e.message}")
+    end
+  end
+end

data/app/services/llm/open_ai_compatible_adapter.rb

@@ -0,0 +1,45 @@
+# Shared by any provider that speaks the OpenAI-style chat-completions shape
+# (function-calling via tools/tool_choice) — Mistral, OpenRouter, DeepSeek,
+# OpenAI itself, etc.
+module Llm
+  class OpenAiCompatibleAdapter
+    def initialize(credential)
+      @credential = credential
+      @base_uri = ProviderRegistry.base_uri_for(credential.provider)
+    end
+
+    def call_tool(system:, tool:, max_tokens:, content_blocks: nil, messages: nil)
+      wire_messages = messages || [ { role: "user", content: content_blocks.map { |block| to_wire_block(block) } } ]
+      response = HTTParty.post(
+        "#{@base_uri}/chat/completions",
+        headers: { "Authorization" => "Bearer #{@credential.api_key}", "Content-Type" => "application/json" },
+        body: {
+          model: @credential.model,
+          max_tokens: max_tokens,
+          messages: [ { role: "system", content: system } ] + wire_messages,
+          tools: [ { type: "function", function: { name: tool[:name], description: tool[:description], parameters: tool[:input_schema] } } ],
+          tool_choice: { type: "function", function: { name: tool[:name] } }
+        }.to_json
+      )
+      raise "Llm::OpenAiCompatibleAdapter: API error #{response.code} — #{response.body}" unless response.success?
+
+      arguments = response.parsed_response.dig("choices", 0, "message", "tool_calls", 0, "function", "arguments")
+      raise "Llm::OpenAiCompatibleAdapter: no #{tool[:name]} tool call in response" unless arguments
+
+      CallResult.new(
+        data: JSON.parse(arguments),
+        input_tokens: response.parsed_response.dig("usage", "prompt_tokens") || 0,
+        output_tokens: response.parsed_response.dig("usage", "completion_tokens") || 0
+      )
+    end
+
+    private
+
+    def to_wire_block(block)
+      case block[:kind]
+      when :text then { type: "text", text: block[:text] }
+      when :image then { type: "image_url", image_url: { url: "data:#{block[:media_type]};base64,#{block[:data]}" } }
+      end
+    end
+  end
+end

data/app/services/llm/provider_registry.rb

@@ -0,0 +1,118 @@
+# Curated, hand-maintained list of supported providers and models.
+# Deliberately not free-text: provider model catalogs (especially
+# OpenRouter's) move fast, so this needs occasional manual updates in
+# exchange for never risking a typo'd/unsupported model string.
+#
+# Each model carries a `vision:` flag. Callers that need to see an image
+# (e.g. analyzing a screenshot) can restrict Llm::Client to vision-capable
+# ranked credentials only — everything else has no such constraint and can
+# use any ranked credential, vision or not.
+#
+# Each model also carries `input_price_per_million`/`output_price_per_million`
+# (USD per 1M tokens, nil if genuinely variable e.g. openrouter/auto). These
+# are seeded from public pricing pages at the time this was written and WILL
+# drift — re-check against each provider's current pricing before trusting
+# them for real budgeting decisions.
+module Llm
+  module ProviderRegistry
+    PROVIDERS = {
+      "anthropic" => {
+        label: "Anthropic",
+        base_uri: nil, # handled by the Anthropic gem directly, not the generic OpenAI-compatible adapter
+        models: {
+          "claude-sonnet-5" => { vision: true, input_price_per_million: 3.00, output_price_per_million: 15.00 },
+          "claude-opus-4-8" => { vision: true, input_price_per_million: 15.00, output_price_per_million: 75.00 },
+          "claude-haiku-4-5-20251001" => { vision: true, input_price_per_million: 1.00, output_price_per_million: 5.00 }
+        }
+      },
+      "openai" => {
+        label: "OpenAI",
+        base_uri: "https://api.openai.com/v1",
+        models: {
+          "gpt-4o" => { vision: true, input_price_per_million: 2.50, output_price_per_million: 10.00 },
+          "gpt-4o-mini" => { vision: true, input_price_per_million: 0.15, output_price_per_million: 0.60 },
+          "gpt-4.1" => { vision: true, input_price_per_million: 2.00, output_price_per_million: 8.00 },
+          "o3-mini" => { vision: false, input_price_per_million: 1.10, output_price_per_million: 4.40 }
+        }
+      },
+      "deepseek" => {
+        label: "DeepSeek",
+        base_uri: "https://api.deepseek.com/v1",
+        # No vision-capable model today — never eligible for vision-only
+        # calls, but fine for text-only calls.
+        models: {
+          "deepseek-chat" => { vision: false, input_price_per_million: 0.27, output_price_per_million: 1.10 },
+          "deepseek-reasoner" => { vision: false, input_price_per_million: 0.55, output_price_per_million: 2.19 }
+        }
+      },
+      "mistral" => {
+        label: "Mistral",
+        base_uri: "https://api.mistral.ai/v1",
+        models: {
+          "pixtral-large-latest" => { vision: true, input_price_per_million: 2.00, output_price_per_million: 6.00 },
+          "pixtral-12b-latest" => { vision: true, input_price_per_million: 0.15, output_price_per_million: 0.15 }
+        }
+      },
+      "kimi" => {
+        label: "Kimi (Moonshot AI)",
+        base_uri: "https://api.moonshot.ai/v1",
+        # Pricing here is the standard (cache-miss) input rate — this gem
+        # doesn't model cache-hit discounts for any provider.
+        models: {
+          "kimi-k3" => { vision: true, input_price_per_million: 3.00, output_price_per_million: 15.00 },
+          "kimi-k2.7-code" => { vision: false, input_price_per_million: 0.95, output_price_per_million: 4.00 },
+          "kimi-k2.6" => { vision: true, input_price_per_million: 0.95, output_price_per_million: 4.00 }
+        }
+      },
+      "openrouter" => {
+        label: "OpenRouter",
+        base_uri: "https://openrouter.ai/api/v1",
+        models: {
+          "anthropic/claude-3.5-sonnet" => { vision: true, input_price_per_million: 3.00, output_price_per_million: 15.00 },
+          "google/gemini-2.0-flash-001" => { vision: true, input_price_per_million: 0.10, output_price_per_million: 0.40 },
+          "openai/gpt-4o" => { vision: true, input_price_per_million: 2.50, output_price_per_million: 10.00 },
+          # Lets OpenRouter's own router pick the best underlying model per
+          # request instead of pinning one — convenient, but it's a
+          # meta-router rather than a specific model, so treat it as
+          # non-vision-guaranteed (which model it picks varies) and
+          # unpriceable (cost depends on whatever it picks).
+          "openrouter/auto" => { vision: false, input_price_per_million: nil, output_price_per_million: nil }
+        }
+      }
+    }.freeze
+
+    def self.valid?(provider, model)
+      PROVIDERS.key?(provider) && PROVIDERS[provider][:models].key?(model)
+    end
+
+    def self.vision_capable?(provider, model)
+      valid?(provider, model) && PROVIDERS[provider][:models][model][:vision]
+    end
+
+    def self.base_uri_for(provider)
+      PROVIDERS.fetch(provider).fetch(:base_uri)
+    end
+
+    def self.pricing_for(provider, model)
+      return nil unless valid?(provider, model)
+
+      cfg = PROVIDERS[provider][:models][model]
+      return nil if cfg[:input_price_per_million].nil? || cfg[:output_price_per_million].nil?
+
+      { input_price_per_million: cfg[:input_price_per_million], output_price_per_million: cfg[:output_price_per_million] }
+    end
+
+    # Returns nil (rather than 0) when a model has no known/fixed pricing —
+    # callers must not silently treat "unknown" as "free".
+    def self.cost_for(provider, model, input_tokens:, output_tokens:)
+      pricing = pricing_for(provider, model)
+      return nil unless pricing
+
+      (input_tokens * pricing[:input_price_per_million] + output_tokens * pricing[:output_price_per_million]) / 1_000_000.0
+    end
+
+    def self.grouped_options
+      PROVIDERS.map { |key, cfg| [ cfg[:label], cfg[:models].keys.map { |m| [ m, "#{key}:#{m}" ] } ] }
+    end
+  end
+end

data/lib/generators/ranked_llm/install/install_generator.rb

@@ -0,0 +1,41 @@
+require "rails/generators"
+require "rails/generators/migration"
+
+module RankedLlm
+  module Generators
+    class InstallGenerator < ::Rails::Generators::Base
+      include ::Rails::Generators::Migration
+
+      source_root File.expand_path("templates", __dir__)
+
+      def self.next_migration_number(dirname)
+        ::ActiveRecord::Generators::Base.next_migration_number(dirname)
+      end
+
+      def create_migrations
+        migration_template "create_llm_credentials.rb.tt", "db/migrate/create_llm_credentials.rb"
+        sleep 1 if migrations_need_distinct_timestamps?
+        migration_template "create_llm_usage_records.rb.tt", "db/migrate/create_llm_usage_records.rb"
+      end
+
+      def show_post_install_message
+        say ""
+        say "ranked_llm installed. Next steps:", :green
+        say "  1. Run: bin/rails db:migrate"
+        say "  2. Add `include Llm::Owned` to whatever model is your tenant/owner concept (Account, Team, User, ...)."
+        say "  3. Configure Active Record Encryption if this app doesn't already (bin/rails db:encryption:init) —"
+        say "     Llm::Credential#api_key is encrypted at rest using it."
+        say "  4. Call it: Llm::Client.for(current_account).call_tool(system:, tool:, max_tokens:, messages:)"
+        say ""
+        say "  Optional settings UI scaffold: bin/rails generate ranked_llm:views"
+        say ""
+      end
+
+      private
+
+      def migrations_need_distinct_timestamps?
+        true
+      end
+    end
+  end
+end

data/lib/generators/ranked_llm/install/templates/create_llm_credentials.rb.tt

@@ -0,0 +1,15 @@
+class CreateLlmCredentials < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    create_table :llm_credentials do |t|
+      t.references :owner, polymorphic: true, null: false
+      t.string :provider, null: false
+      t.string :model, null: false
+      t.text :api_key, null: false
+      t.integer :position, null: false
+
+      t.timestamps
+    end
+
+    add_index :llm_credentials, [ :owner_type, :owner_id, :position ], unique: true, name: "index_llm_credentials_on_owner_and_position"
+  end
+end

data/lib/generators/ranked_llm/install/templates/create_llm_usage_records.rb.tt

@@ -0,0 +1,18 @@
+class CreateLlmUsageRecords < ActiveRecord::Migration[<%= ActiveRecord::Migration.current_version %>]
+  def change
+    create_table :llm_usage_records do |t|
+      t.references :owner, polymorphic: true, null: false
+      t.string :provider, null: false
+      t.string :model, null: false
+      t.integer :input_tokens, null: false
+      t.integer :output_tokens, null: false
+      # Nullable: some models (e.g. openrouter/auto) have no fixed price.
+      t.decimal :cost_usd, precision: 12, scale: 6
+
+      t.timestamps
+    end
+
+    add_index :llm_usage_records, [ :owner_type, :owner_id, :created_at ], name: "index_llm_usage_records_on_owner_and_created_at"
+    add_index :llm_usage_records, [ :owner_type, :owner_id, :provider, :model ], name: "index_llm_usage_records_on_owner_and_provider_and_model"
+  end
+end

data/lib/generators/ranked_llm/views/templates/ai_apis_controller.rb.tt

@@ -0,0 +1,39 @@
+module Settings
+  class AiApisController < ApplicationController
+    before_action :require_account! # TODO: replace with this app's real auth boundary
+    before_action :set_credential, only: [ :destroy, :move ]
+
+    def index
+      @credentials = current_account.llm_credentials.ranked
+      usage_this_month = current_account.llm_usage_records.this_month.group(:provider, :model)
+      @monthly_cost_by_provider_model = usage_this_month.sum(:cost_usd)
+      @monthly_tokens_by_provider_model = usage_this_month.sum("input_tokens + output_tokens")
+    end
+
+    def create
+      provider, model = params[:llm_credential][:provider_and_model].to_s.split(":", 2)
+      credential = current_account.llm_credentials.new(provider: provider, model: model, api_key: params[:llm_credential][:api_key])
+      if credential.save
+        redirect_to settings_ai_apis_path, notice: "AI API added."
+      else
+        redirect_to settings_ai_apis_path, alert: credential.errors.full_messages.to_sentence
+      end
+    end
+
+    def destroy
+      @credential.destroy
+      redirect_to settings_ai_apis_path, notice: "AI API removed."
+    end
+
+    def move
+      @credential.update!(position: params[:position].to_i.clamp(1, 100))
+      head :ok
+    end
+
+    private
+
+    def set_credential
+      @credential = current_account.llm_credentials.find(params[:id])
+    end
+  end
+end

data/lib/generators/ranked_llm/views/templates/index.html.erb.tt

@@ -0,0 +1,66 @@
+<div class="max-w-2xl mx-auto mt-8">
+  <h1 class="text-2xl font-bold mb-2">AI APIs</h1>
+  <p class="text-sm text-slate-500 mb-6">
+    Every AI feature in this app uses these credentials in rank order, falling back to the next one if a call
+    fails. Drag to reorder.
+  </p>
+
+  <div class="bg-white border border-slate-200 rounded-lg p-5 mb-6">
+    <%%= form_with url: settings_ai_apis_path, method: :post, class: "space-y-4" do |f| %>
+      <div>
+        <%%= label_tag "llm_credential_provider_and_model", "Provider", class: "block text-sm font-medium mb-1" %>
+        <%%= select_tag "llm_credential[provider_and_model]", grouped_options_for_select(Llm::ProviderRegistry.grouped_options),
+              class: "w-full rounded-md border border-slate-300 px-3 py-2 text-sm bg-white" %>
+      </div>
+      <div>
+        <%%= label_tag "llm_credential_api_key", "API key", class: "block text-sm font-medium mb-1" %>
+        <%%= password_field_tag "llm_credential[api_key]", nil, placeholder: "paste key…", autocomplete: "off",
+              class: "w-full rounded-md border border-slate-300 px-3 py-2 text-sm focus:outline-none focus:ring-2 focus:ring-amber-500" %>
+      </div>
+      <%%= f.submit "Add", class: "rounded-md bg-amber-500 hover:bg-amber-600 text-white text-sm font-medium px-4 py-2 cursor-pointer" %>
+    <%% end %>
+  </div>
+
+  <div class="bg-white border border-slate-200 rounded-lg divide-y divide-slate-100" data-controller="ranked-list">
+    <%% if @credentials.empty? %>
+      <p class="px-5 py-6 text-sm text-slate-500 text-center">No AI APIs configured yet — every AI feature will be unavailable until you add one.</p>
+    <%% end %>
+    <%% @credentials.each do |credential| %>
+      <div class="flex items-center justify-between px-5 py-3 cursor-move" data-ranked-list-target="item" data-move-url="<%%= move_settings_ai_api_path(credential) %>">
+        <div class="flex items-center gap-3 min-w-0">
+          <span class="text-slate-300" aria-hidden="true">⠿</span>
+          <div class="min-w-0">
+            <p class="font-medium text-sm">
+              <%%= Llm::ProviderRegistry::PROVIDERS.dig(credential.provider, :label) || credential.provider %>
+              <span class="font-normal text-slate-500">· <%%= credential.model %></span>
+              <%% if Llm::ProviderRegistry.vision_capable?(credential.provider, credential.model) %>
+                <span class="rounded bg-slate-100 text-slate-500 text-xs px-1.5 py-0.5 align-middle">vision</span>
+              <%% end %>
+            </p>
+            <%% pricing = Llm::ProviderRegistry.pricing_for(credential.provider, credential.model) %>
+            <%% tokens_this_month = @monthly_tokens_by_provider_model[[ credential.provider, credential.model ]] %>
+            <%% cost_this_month = @monthly_cost_by_provider_model[[ credential.provider, credential.model ]] %>
+            <p class="text-xs text-slate-500">
+              Rank #<%%= credential.position %>
+              ·
+              <%% if pricing %>
+                <%%= number_to_currency(pricing[:input_price_per_million], precision: 2) %> / <%%= number_to_currency(pricing[:output_price_per_million], precision: 2) %> per 1M tokens (in/out)
+              <%% else %>
+                pricing varies
+              <%% end %>
+              <%% if tokens_this_month.present? && tokens_this_month.positive? %>
+                · <%%= number_with_delimiter(tokens_this_month) %> tokens this month<%% if cost_this_month.present? %> (~<%%= number_to_currency(cost_this_month, precision: 4) %>)<%% end %>
+              <%% end %>
+            </p>
+          </div>
+        </div>
+        <%%= button_to "Remove", settings_ai_api_path(credential), method: :delete,
+              data: { turbo_confirm: "Remove this AI API? Anything using it will fall through to the next rank." },
+              class: "text-xs text-rose-600 hover:underline cursor-pointer shrink-0" %>
+      </div>
+    <%% end %>
+  </div>
+  <p class="text-xs text-slate-400 mt-3">
+    Pricing is seeded from each provider's public rates and may drift — verify against their current pricing page before budgeting off it.
+  </p>
+</div>

data/lib/generators/ranked_llm/views/templates/ranked_list_controller.js

@@ -0,0 +1,37 @@
+import { Controller } from "@hotwired/stimulus"
+import Sortable from "sortablejs"
+
+// Single-list drag-and-drop reordering: persists position when an item is
+// dropped. Always one list — no cross-column "group" concept.
+export default class extends Controller {
+  static targets = ["item"]
+
+  connect() {
+    this.sortable = new Sortable(this.element, {
+      animation: 150,
+      ghostClass: "opacity-40",
+      onEnd: event => this.persist(event)
+    })
+  }
+
+  disconnect() {
+    this.sortable?.destroy()
+  }
+
+  async persist(event) {
+    const url = event.item.dataset.moveUrl
+    const position = event.newIndex + 1
+
+    const response = await fetch(url, {
+      method: "PATCH",
+      headers: {
+        "Content-Type": "application/json",
+        "Accept": "application/json",
+        "X-CSRF-Token": document.querySelector('meta[name="csrf-token"]')?.content
+      },
+      body: JSON.stringify({ position })
+    })
+
+    if (!response.ok) window.location.reload()
+  }
+}

data/lib/generators/ranked_llm/views/views_generator.rb

@@ -0,0 +1,46 @@
+require "rails/generators"
+
+module RankedLlm
+  module Generators
+    # Scaffolds a starting-point settings controller/view/JS for managing
+    # ranked credentials. Deliberately not auto-mounted: every host app has
+    # its own auth boundary (current_account/current_team/...) and UI
+    # conventions, so this is meant to be copied and adapted, not used as-is.
+    class ViewsGenerator < ::Rails::Generators::Base
+      source_root File.expand_path("templates", __dir__)
+
+      def copy_controller
+        template "ai_apis_controller.rb.tt", "app/controllers/settings/ai_apis_controller.rb"
+      end
+
+      def copy_view
+        template "index.html.erb.tt", "app/views/settings/ai_apis/index.html.erb"
+      end
+
+      def copy_stimulus_controller
+        copy_file "ranked_list_controller.js", "app/javascript/controllers/ranked_list_controller.js"
+      end
+
+      def show_post_install_message
+        say ""
+        say "Copied a starting-point settings controller/view/JS for ranked_llm.", :green
+        say "Adjust the following before using it:", :yellow
+        say "  - `current_account` in app/controllers/settings/ai_apis_controller.rb — rename to whatever"
+        say "    your app's auth boundary actually calls the owner (current_team, current_user, ...)."
+        say "  - Restyle app/views/settings/ai_apis/index.html.erb to match this app's existing UI conventions —"
+        say "    these Tailwind classes are just a starting point."
+        say "  - Add a route, e.g.:"
+        say "      namespace :settings do"
+        say "        resources :ai_apis, only: [ :index, :create, :destroy ] do"
+        say "          patch :move, on: :member"
+        say "        end"
+        say "      end"
+        say "  - Register the Stimulus controller if this app doesn't auto-register app/javascript/controllers"
+        say "    (importmap-rails/Stimulus's default setup already does this)."
+        say "  - This view drags with SortableJS via the `ranked-list` Stimulus controller — make sure"
+        say "    the `sortablejs` package (or your import-mapped equivalent) is available."
+        say ""
+      end
+    end
+  end
+end

data/lib/ranked_llm/engine.rb

@@ -0,0 +1,8 @@
+module RankedLlm
+  # Registers the gem's app/models and app/services directories the normal
+  # Rails engine way. Deliberately not `isolate_namespace`d — call sites in
+  # host apps address these classes as plain `Llm::...`, not
+  # `RankedLlm::Llm::...`.
+  class Engine < ::Rails::Engine
+  end
+end

data/lib/ranked_llm/version.rb

@@ -0,0 +1,3 @@
+module RankedLlm
+  VERSION = "0.2.0"
+end

data/lib/ranked_llm.rb

@@ -0,0 +1,8 @@
+require "positioning" # Llm::Credential uses `positioned` at class-definition time — must be loaded before app/models autoloads it.
+require "httparty"
+require "anthropic"
+require "ranked_llm/version"
+require "ranked_llm/engine"
+
+module RankedLlm
+end

metadata

@@ -0,0 +1,164 @@
+--- !ruby/object:Gem::Specification
+name: ranked_llm
+version: !ruby/object:Gem::Version
+  version: 0.2.0
+platform: ruby
+authors:
+- Jeremy
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: rails
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '7.1'
+- !ruby/object:Gem::Dependency
+  name: positioning
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.4'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '0.4'
+- !ruby/object:Gem::Dependency
+  name: httparty
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: anthropic
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: combustion
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.5'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: A ranked list of AI API credentials (OpenAI, Anthropic, DeepSeek, Mistral,
+  OpenRouter, ...) tried in order, falling back to the next on any failure, with per-model
+  pricing and per-call spend logging. Extracted from Progentick.
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- app/models/llm.rb
+- app/models/llm/credential.rb
+- app/models/llm/owned.rb
+- app/models/llm/usage_record.rb
+- app/services/llm/anthropic_adapter.rb
+- app/services/llm/call_result.rb
+- app/services/llm/client.rb
+- app/services/llm/open_ai_compatible_adapter.rb
+- app/services/llm/provider_registry.rb
+- lib/generators/ranked_llm/install/install_generator.rb
+- lib/generators/ranked_llm/install/templates/create_llm_credentials.rb.tt
+- lib/generators/ranked_llm/install/templates/create_llm_usage_records.rb.tt
+- lib/generators/ranked_llm/views/templates/ai_apis_controller.rb.tt
+- lib/generators/ranked_llm/views/templates/index.html.erb.tt
+- lib/generators/ranked_llm/views/templates/ranked_list_controller.js
+- lib/generators/ranked_llm/views/views_generator.rb
+- lib/ranked_llm.rb
+- lib/ranked_llm/engine.rb
+- lib/ranked_llm/version.rb
+homepage: https://github.com/ohheyjeremy/ranked_llm
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/ohheyjeremy/ranked_llm
+  source_code_uri: https://github.com/ohheyjeremy/ranked_llm
+  changelog_uri: https://github.com/ohheyjeremy/ranked_llm/blob/main/CHANGELOG.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.2'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.12
+specification_version: 4
+summary: Ranked, multi-provider AI API credentials with automatic fallback and cost
+  tracking for Rails apps.
+test_files: []