rails-ai-gateway 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 67143b76482c8cdcfc4e4badd4801e1d46c9ecc5bc309e2ddd137f14df7a0e67
4
+ data.tar.gz: 1ac58e22f791aead20feabf049dc8f89250eff16170b860c9f4da690b096af96
5
+ SHA512:
6
+ metadata.gz: 6cdeafa5eaf720bd486ff259620eb6868aa0829f96b9c461626fbd6bb696227534330672480df8b4496fc8182ec56563bd422a476b6c0b4dabb1cfc4e16ddf7d
7
+ data.tar.gz: 550e16aece7e78223753f79cdbe2a2f6c3308767b3e259cce7b917687dd32055daadeae9e9df9134ce205741267208339126b3b2c531d6a2809ea22562a172de
data/CHANGELOG.md ADDED
@@ -0,0 +1,8 @@
1
+ # Changelog
2
+
3
+ ## 0.1.0 - 2026-09-20
4
+
5
+ - Add mountable Rails engine with OpenAI-compatible chat, embedding, and model endpoints.
6
+ - Add ActiveRecord providers, model routing, gateway keys, request logs, and admin UI.
7
+ - Add encrypted provider credentials, ordered fallback, bounded responses, SSE streaming, and SSRF protections.
8
+ - Add initializer configuration for authorization, timeouts, limits, attempts, and network policy.
data/CONTRIBUTING.md ADDED
@@ -0,0 +1,64 @@
1
+ # Contributing to Rails AI Gateway
2
+
3
+ Bug reports, fixes, tests, documentation, and focused feature proposals are welcome.
4
+
5
+ ## Report Bugs
6
+
7
+ Open GitHub issue with:
8
+
9
+ - Rails, Ruby, database adapter, and gem versions
10
+ - Minimal reproduction steps
11
+ - Expected and actual behavior
12
+ - Relevant exception and backtrace with secrets removed
13
+ - Browser name and version for Web UI bugs
14
+ - Upstream provider and endpoint type without API credentials
15
+
16
+ Never include provider API keys, gateway keys, database passwords, prompts, responses, or
17
+ other private application data.
18
+
19
+ Report vulnerabilities privately through
20
+ [GitHub security advisories](https://github.com/azmi2409/rails-ai-gateway/security/advisories/new).
21
+
22
+ ## Development Setup
23
+
24
+ ```bash
25
+ git clone https://github.com/azmi2409/rails-ai-gateway.git
26
+ cd rails-ai-gateway
27
+ bundle install
28
+ bundle exec ruby test/check.rb
29
+ ```
30
+
31
+ Test uses temporary SQLite database and local fake upstream. No provider account or network
32
+ request is required.
33
+
34
+ To test PostgreSQL, create empty disposable database and pass its URL:
35
+
36
+ ```bash
37
+ createdb rails_ai_gateway_test
38
+ DATABASE_URL=postgresql:///rails_ai_gateway_test bundle exec ruby test/check.rb
39
+ dropdb rails_ai_gateway_test
40
+ ```
41
+
42
+ ## Changes
43
+
44
+ - Keep changes focused and backward compatible after public release.
45
+ - Add or update integration checks for behavior changes.
46
+ - Preserve Ruby 3.3 and Rails 8 compatibility.
47
+ - Support SQLite and PostgreSQL unless change explicitly targets one adapter.
48
+ - Avoid runtime dependencies when Ruby, Rails, or current dependencies cover need.
49
+ - Keep Web UI dependency-free, responsive, keyboard-accessible, and screen-reader friendly.
50
+ - Never weaken gateway-key authentication, credential encryption, request limits, SSRF
51
+ protections, CSRF protection, or default-deny admin authorization.
52
+ - Never persist prompts, responses, raw gateway keys, or unencrypted provider credentials.
53
+ - Never retry ambiguous upstream failures or a stream after response bytes are sent.
54
+
55
+ ## Pull Requests
56
+
57
+ 1. Create branch from `main`.
58
+ 2. Make focused change with test coverage.
59
+ 3. Run `bundle exec ruby test/check.rb`.
60
+ 4. Run PostgreSQL integration when changing migrations, models, or queries.
61
+ 5. Run `gem build rails_ai_gateway.gemspec`.
62
+ 6. Open pull request describing problem, solution, security impact, and verification.
63
+
64
+ By contributing, you agree your contribution is licensed under project MIT license.
data/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Rails AI Gateway contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,151 @@
1
+ # Rails AI Gateway
2
+
3
+ [![Gem Version](https://badge.fury.io/rb/rails-ai-gateway.svg)](https://rubygems.org/gems/rails-ai-gateway)
4
+ [![License: MIT](https://img.shields.io/badge/License-MIT-blue.svg)](LICENSE)
5
+
6
+ Put one OpenAI-compatible endpoint in front of your AI providers without running another
7
+ service. Rails AI Gateway mounts inside your Rails app, stores configuration and request
8
+ metadata with ActiveRecord, and includes a Web UI for providers, model routes, and keys.
9
+
10
+ ## What You Get
11
+
12
+ - OpenAI-compatible chat completions, embeddings, and model-list endpoints
13
+ - Streaming chat completions over server-sent events
14
+ - Public model aliases with ordered provider fallbacks
15
+ - Encrypted provider API keys using ActiveRecord encryption
16
+ - Scoped gateway keys stored as SHA-256 digests
17
+ - Request status, latency, attempt, and reported token-usage logs
18
+ - Configurable timeouts, body limits, fallback attempts, and network policy
19
+ - SSRF protection with DNS validation and address pinning
20
+ - Server-rendered, responsive, accessible admin UI
21
+ - SQLite and PostgreSQL support through host Rails database
22
+ - No prompts or responses persisted
23
+
24
+ ## Get Started
25
+
26
+ Add gem:
27
+
28
+ ```ruby
29
+ gem "rails-ai-gateway"
30
+ ```
31
+
32
+ Set it up:
33
+
34
+ ```bash
35
+ bundle install
36
+ bin/rails generate rails_ai_gateway:install
37
+ bin/rails db:migrate
38
+ ```
39
+
40
+ Generator creates `config/initializers/rails_ai_gateway.rb`, copies migrations, and mounts
41
+ engine at `/ai`. Configure admin authorization, restart Rails, then visit:
42
+
43
+ <http://localhost:3000/ai/admin>
44
+
45
+ Host app must configure
46
+ [ActiveRecord encryption](https://guides.rubyonrails.org/active_record_encryption.html).
47
+ Provider API keys cannot be saved without it.
48
+
49
+ ## Configure Gateway
50
+
51
+ Web UI guides initial setup:
52
+
53
+ 1. Add provider with full API base URL, such as `https://api.openai.com/v1`.
54
+ 2. Add public model route, such as `fast-chat` mapped to `gpt-4o-mini`.
55
+ 3. Create gateway key and save token when shown. Raw token cannot be displayed again.
56
+
57
+ Routes with same public model name form fallback chain. Lower priority runs first. Gateway
58
+ retries connection failures before upstream accepts request and HTTP `429`, `500`, `502`,
59
+ `503`, or `504`. Ambiguous timeouts and started streams are never retried.
60
+
61
+ ## Initializer
62
+
63
+ Runtime and security policy live in `config/initializers/rails_ai_gateway.rb`:
64
+
65
+ ```ruby
66
+ RailsAiGateway.configure do |config|
67
+ config.admin_controller = "ApplicationController"
68
+ config.admin_authorization = ->(controller) {
69
+ controller.current_user&.admin? == true
70
+ }
71
+
72
+ config.open_timeout = 5
73
+ config.read_timeout = 60
74
+ config.write_timeout = 30
75
+ config.request_timeout = 120
76
+ config.max_request_bytes = 2 * 1024 * 1024
77
+ config.max_response_bytes = 16 * 1024 * 1024
78
+ config.max_attempts = 3
79
+
80
+ config.allow_private_networks = false
81
+ config.allow_http = false
82
+ end
83
+ ```
84
+
85
+ Admin requests are denied until `admin_authorization` returns exactly `true`. Provider
86
+ definitions, model routes, gateway keys, and logs stay database-backed and editable through
87
+ Web UI.
88
+
89
+ Need another mount path? Change host route:
90
+
91
+ ```ruby
92
+ mount RailsAiGateway::Engine, at: "/gateway"
93
+ ```
94
+
95
+ ## Make Requests
96
+
97
+ ```bash
98
+ curl http://localhost:3000/ai/v1/chat/completions \
99
+ -H "Authorization: Bearer rag_REPLACE_ME" \
100
+ -H "Content-Type: application/json" \
101
+ -d '{"model":"fast-chat","messages":[{"role":"user","content":"Hello"}]}'
102
+ ```
103
+
104
+ Available endpoints:
105
+
106
+ - `GET /ai/v1/models`
107
+ - `POST /ai/v1/chat/completions`
108
+ - `POST /ai/v1/embeddings`
109
+
110
+ OpenAI clients can use `http://localhost:3000/ai/v1` as base URL.
111
+
112
+ ## Development
113
+
114
+ ```bash
115
+ git clone https://github.com/azmi2409/rails-ai-gateway.git
116
+ cd rails-ai-gateway
117
+ bundle install
118
+ bundle exec ruby test/check.rb
119
+ gem build rails_ai_gateway.gemspec
120
+ ```
121
+
122
+ SQLite integration runs by default. Run same suite against PostgreSQL with an empty test
123
+ database:
124
+
125
+ ```bash
126
+ DATABASE_URL=postgresql:///rails_ai_gateway_test bundle exec ruby test/check.rb
127
+ ```
128
+
129
+ Found bug or have focused improvement? Read [CONTRIBUTING.md](CONTRIBUTING.md), then open
130
+ issue or pull request. Release notes live in [CHANGELOG.md](CHANGELOG.md).
131
+
132
+ ## Security
133
+
134
+ Never expose admin UI without host authentication and authorization. Keep
135
+ `allow_private_networks` and `allow_http` disabled for public providers. Enable both only
136
+ for trusted internal endpoints, such as local Ollama. Never commit provider or database
137
+ credentials.
138
+
139
+ Report security issues privately through
140
+ [GitHub security advisories](https://github.com/azmi2409/rails-ai-gateway/security/advisories/new),
141
+ not public issues.
142
+
143
+ ## Current Scope
144
+
145
+ Current release supports OpenAI-compatible chat completions, embeddings, and model-list
146
+ endpoints. Native Anthropic, Gemini, Bedrock, budgets, semantic caching, and multi-tenancy
147
+ are not included.
148
+
149
+ ## License
150
+
151
+ [MIT](LICENSE).
@@ -0,0 +1,41 @@
1
+ :root { color-scheme: light; font-family: system-ui, sans-serif; color: #172538; background: #f5f7fa; line-height: 1.5; }
2
+ * { box-sizing: border-box; }
3
+ body { margin: 0; }
4
+ header { background: #14283f; color: #fff; display: flex; justify-content: space-between; gap: 1rem; flex-wrap: wrap; padding: 1.25rem max(1rem, calc((100vw - 1200px) / 2)); }
5
+ header a { color: #fff; text-decoration: none; }
6
+ .brand { font-weight: 750; font-size: 1.15rem; }
7
+ nav { display: flex; flex-wrap: wrap; gap: 1.25rem; }
8
+ main, footer { max-width: 1200px; margin: auto; padding: 1.5rem; }
9
+ h1 { font-size: 2rem; letter-spacing: -.03em; margin-bottom: .4rem; }
10
+ h2 { margin-top: 0; }
11
+ a { color: #175da1; }
12
+ section { margin: 1.5rem 0; padding: 1.5rem; background: #fff; border: 1px solid #d8e0e9; border-radius: .5rem; scroll-margin-top: 1rem; }
13
+ .muted, footer { color: #536477; }
14
+ .stats { display: grid; grid-template-columns: repeat(auto-fit, minmax(200px, 1fr)); gap: 1rem; }
15
+ .stats div { background: #fff; border: 1px solid #d8e0e9; padding: 1.25rem; border-radius: .5rem; }
16
+ .stats span, .stats strong { display: block; }
17
+ .stats strong { font-size: 2rem; margin-top: .4rem; }
18
+ .grid { display: grid; grid-template-columns: repeat(auto-fit, minmax(min(100%, 230px), 1fr)); gap: 1rem; align-items: end; margin: 1rem 0; }
19
+ label { display: block; font-weight: 600; font-size: .875rem; margin-bottom: .35rem; }
20
+ input, select, textarea, button { font: inherit; }
21
+ input:not([type=checkbox]), select, textarea { width: 100%; padding: .6rem; border: 1px solid #8595a7; border-radius: .25rem; background: #fff; color: inherit; }
22
+ button, input[type=submit] { background: #195a97; color: #fff; border: 1px solid #195a97; border-radius: .25rem; padding: .6rem .85rem; cursor: pointer; font-weight: 600; }
23
+ button.danger { color: #a12828; background: #fff; border-color: #d5aaaa; font-size: .85rem; }
24
+ input:disabled { opacity: .5; cursor: not-allowed; }
25
+ .check { display: flex; align-items: center; gap: .5rem; min-height: 2.8rem; }
26
+ .check label { margin: 0; }
27
+ details { padding: .85rem 0; border-top: 1px solid #e3e8ef; }
28
+ summary { cursor: pointer; font-weight: 600; overflow-wrap: anywhere; }
29
+ .badge { font-weight: 400; color: #536477; font-size: .85rem; margin-left: .5rem; }
30
+ .table-wrap { overflow-x: auto; }
31
+ table { width: 100%; border-collapse: collapse; font-size: .9rem; }
32
+ caption { text-align: left; color: #536477; margin-bottom: .75rem; }
33
+ th, td { text-align: left; padding: .8rem .6rem; border-bottom: 1px solid #e3e8ef; vertical-align: top; }
34
+ th { background: #f5f7fa; white-space: nowrap; }
35
+ td details { border: 0; padding: 0; }
36
+ code, pre, textarea { font-family: ui-monospace, monospace; overflow-wrap: anywhere; }
37
+ pre { max-width: 28rem; white-space: pre-wrap; }
38
+ :focus-visible { outline: 3px solid #b46b00; outline-offset: 3px; }
39
+ .skip { position: absolute; left: -10000px; }
40
+ .skip:focus { left: 1rem; top: .5rem; padding: .5rem; background: #fff; z-index: 1; }
41
+ @media (max-width: 600px) { main, footer { padding: 1rem; } section { padding: 1rem; } }
@@ -0,0 +1,90 @@
1
+ module RailsAiGateway
2
+ class AdminController < RailsAiGateway.configuration.admin_controller.constantize
3
+ layout "rails_ai_gateway/admin"
4
+ protect_from_forgery with: :exception
5
+ before_action :authorize_admin
6
+ rescue_from ActiveRecord::RecordInvalid, with: :invalid_record
7
+ rescue_from ActiveRecord::RecordNotFound, with: -> { head :not_found }
8
+ rescue_from ActiveRecord::RecordNotUnique, with: -> { render plain: "Name or priority already exists", status: 422 }
9
+
10
+ def index
11
+ @providers = Provider.order(:name)
12
+ @routes = ModelRoute.includes(:provider).order(:name, :priority)
13
+ @keys = GatewayKey.order(created_at: :desc)
14
+ recent = RequestLog.where(created_at: 24.hours.ago..)
15
+ @requests = recent.count
16
+ @errors = recent.where(status: 400..599).count
17
+ @latency = recent.average(:duration_ms)&.round || 0
18
+ logs = RequestLog.order(id: :desc)
19
+ logs = logs.where(model: params[:model]) if params[:model].is_a?(String) && params[:model].present?
20
+ logs = logs.where(status: params[:status].to_i) if params[:status].to_s.match?(/\A[1-5][0-9]{2}\z/)
21
+ logs = logs.where("id < ?", params[:before].to_i) if params[:before].to_s.match?(/\A[0-9]+\z/)
22
+ @logs = logs.limit(50)
23
+ end
24
+
25
+ def create_provider
26
+ Provider.create!(provider_params)
27
+ redirect_to admin_path, status: :see_other
28
+ end
29
+
30
+ def update_provider
31
+ attributes = provider_params
32
+ attributes.delete(:api_key) if attributes[:api_key].blank?
33
+ attributes[:api_key] = nil if params[:clear_api_key] == "1"
34
+ Provider.find(params[:id]).update!(attributes)
35
+ redirect_to admin_path, status: :see_other
36
+ end
37
+
38
+ def create_model_route
39
+ ModelRoute.create!(route_params)
40
+ redirect_to admin_path, status: :see_other
41
+ end
42
+
43
+ def update_model_route
44
+ ModelRoute.find(params[:id]).update!(route_params)
45
+ redirect_to admin_path, status: :see_other
46
+ end
47
+
48
+ def destroy_model_route
49
+ ModelRoute.find(params[:id]).destroy!
50
+ redirect_to admin_path, status: :see_other
51
+ end
52
+
53
+ def create_gateway_key
54
+ attributes = params.require(:gateway_key).permit(:name, :allowed_models, :expires_at).to_h
55
+ attributes["allowed_models"] = attributes["allowed_models"].to_s.split(/[,\s]+/).reject(&:empty?).uniq
56
+ @key, @token = GatewayKey.issue!(**attributes.symbolize_keys)
57
+ render :key, status: :created
58
+ end
59
+
60
+ def revoke_gateway_key
61
+ GatewayKey.find(params[:id]).update!(revoked_at: Time.current)
62
+ redirect_to admin_path, status: :see_other
63
+ end
64
+
65
+ def style
66
+ send_file Engine.root.join("app/assets/stylesheets/rails_ai_gateway/admin.css"), type: "text/css", disposition: "inline"
67
+ end
68
+
69
+ private
70
+
71
+ def authorize_admin
72
+ response.headers["Cache-Control"] = "no-store"
73
+ response.headers["Referrer-Policy"] = "no-referrer"
74
+ head :forbidden unless RailsAiGateway.configuration.admin_authorization.call(self) == true
75
+ end
76
+
77
+ def provider_params
78
+ params.require(:provider).permit(:name, :base_url, :api_key, :enabled)
79
+ end
80
+
81
+ def route_params
82
+ params.require(:model_route).permit(:name, :provider_id, :upstream_model, :priority)
83
+ end
84
+
85
+ def invalid_record(exception)
86
+ @errors = exception.record.errors.full_messages
87
+ render :errors, status: 422
88
+ end
89
+ end
90
+ end
@@ -0,0 +1,5 @@
1
+ module RailsAiGateway
2
+ class ApplicationRecord < ActiveRecord::Base
3
+ self.abstract_class = true
4
+ end
5
+ end
@@ -0,0 +1,35 @@
1
+ require "digest"
2
+ require "securerandom"
3
+
4
+ module RailsAiGateway
5
+ class GatewayKey < ApplicationRecord
6
+ validates :name, presence: true, length: { maximum: 255 }
7
+ validates :token_digest, :prefix, presence: true
8
+ validates :token_digest, uniqueness: true
9
+ validate :valid_allowed_models
10
+
11
+ def self.issue!(**attributes)
12
+ token = "rag_#{SecureRandom.hex(32)}"
13
+ key = create!(**attributes, token_digest: Digest::SHA256.hexdigest(token), prefix: token.first(12))
14
+ [key, token]
15
+ end
16
+
17
+ def self.authenticate(token)
18
+ return unless token.is_a?(String) && token.match?(/\Arag_[0-9a-f]{64}\z/)
19
+ key = find_by(token_digest: Digest::SHA256.hexdigest(token), revoked_at: nil)
20
+ key if key && (!key.expires_at || key.expires_at.future?)
21
+ end
22
+
23
+ def allows?(model)
24
+ allowed_models.empty? || allowed_models.include?(model)
25
+ end
26
+
27
+ private
28
+
29
+ def valid_allowed_models
30
+ unless allowed_models.is_a?(Array) && allowed_models.all? { |name| name.is_a?(String) && name.present? && name.length <= 255 }
31
+ errors.add(:allowed_models, "must be an array of model names")
32
+ end
33
+ end
34
+ end
35
+ end
@@ -0,0 +1,8 @@
1
+ module RailsAiGateway
2
+ class ModelRoute < ApplicationRecord
3
+ belongs_to :provider
4
+ validates :name, :upstream_model, presence: true, length: { maximum: 255 }
5
+ validates :priority, numericality: { only_integer: true, greater_than_or_equal_to: 0 }, uniqueness: { scope: :name }
6
+ scope :available, -> { joins(:provider).where(rails_ai_gateway_providers: { enabled: true }) }
7
+ end
8
+ end
@@ -0,0 +1,24 @@
1
+ require "uri"
2
+
3
+ module RailsAiGateway
4
+ class Provider < ApplicationRecord
5
+ encrypts :api_key
6
+ has_many :model_routes, dependent: :restrict_with_error
7
+ validates :name, presence: true, uniqueness: true, length: { maximum: 255 }
8
+ validates :base_url, presence: true, length: { maximum: 2048 }
9
+ validates :api_key, format: { without: /[\r\n]/ }, allow_nil: true
10
+ validate :valid_base_url
11
+
12
+ private
13
+
14
+ def valid_base_url
15
+ uri = URI.parse(base_url.to_s)
16
+ schemes = RailsAiGateway.configuration.allow_http ? %w[https http] : %w[https]
17
+ unless schemes.include?(uri.scheme) && uri.host.present? && !uri.userinfo && !uri.query && !uri.fragment
18
+ errors.add(:base_url, "must be an HTTPS URL without credentials, query, or fragment")
19
+ end
20
+ rescue URI::InvalidURIError
21
+ errors.add(:base_url, "is invalid")
22
+ end
23
+ end
24
+ end
@@ -0,0 +1,5 @@
1
+ module RailsAiGateway
2
+ class RequestLog < ApplicationRecord
3
+ belongs_to :gateway_key, optional: true
4
+ end
5
+ end
@@ -0,0 +1,24 @@
1
+ <!doctype html>
2
+ <html lang="en">
3
+ <head>
4
+ <meta charset="utf-8">
5
+ <meta name="viewport" content="width=device-width, initial-scale=1">
6
+ <title>Rails AI Gateway</title>
7
+ <%= csrf_meta_tags %>
8
+ <%= stylesheet_link_tag style_path %>
9
+ </head>
10
+ <body>
11
+ <a class="skip" href="#main">Skip to content</a>
12
+ <header>
13
+ <%= link_to "Rails AI Gateway", admin_path, class: "brand" %>
14
+ <nav aria-label="Gateway sections">
15
+ <%= link_to "Providers", admin_path(anchor: "providers") %>
16
+ <%= link_to "Models", admin_path(anchor: "models") %>
17
+ <%= link_to "Keys", admin_path(anchor: "keys") %>
18
+ <%= link_to "Logs", admin_path(anchor: "logs") %>
19
+ </nav>
20
+ </header>
21
+ <main id="main"><%= yield %></main>
22
+ <footer>Rails AI Gateway <%= RailsAiGateway::VERSION %> · Request metadata only. Prompts and responses are not stored.</footer>
23
+ </body>
24
+ </html>
@@ -0,0 +1,3 @@
1
+ <h1>Could not save changes</h1>
2
+ <ul role="alert"><% @errors.each do |error| %><li><%= error %></li><% end %></ul>
3
+ <%= link_to "Back to gateway", admin_path %>
@@ -0,0 +1,110 @@
1
+ <h1>Gateway overview</h1>
2
+ <p class="muted">One endpoint for your models. Configuration and traffic live in your Rails database.</p>
3
+ <div class="stats" aria-label="Last 24 hours">
4
+ <div><span>Requests · 24 hours</span><strong><%= @requests %></strong></div>
5
+ <div><span>Errors · 24 hours</span><strong><%= @errors %></strong></div>
6
+ <div><span>Average latency</span><strong><%= @latency %> ms</strong></div>
7
+ </div>
8
+
9
+ <section id="providers">
10
+ <h2>Providers</h2>
11
+ <p>Use the complete API base URL, including <code>/v1</code> where required. Credentials are encrypted with ActiveRecord encryption.</p>
12
+ <% @providers.each do |provider| %>
13
+ <details>
14
+ <summary><%= provider.name %> <span class="badge"><%= provider.enabled? ? "Enabled" : "Disabled" %></span></summary>
15
+ <%= form_with scope: :provider, url: provider_path(provider), method: :patch, local: true, class: "grid", namespace: "provider_#{provider.id}" do |form| %>
16
+ <div><%= form.label :name %><%= form.text_field :name, value: provider.name, required: true %></div>
17
+ <div><%= form.label :base_url, "API base URL" %><%= form.url_field :base_url, value: provider.base_url, required: true %></div>
18
+ <div><%= form.label :api_key, "Replace API key (blank keeps existing)" %><%= form.password_field :api_key, autocomplete: "new-password" %></div>
19
+ <div class="check"><%= form.check_box :enabled, checked: provider.enabled? %><%= form.label :enabled %></div>
20
+ <div class="check"><%= check_box_tag :clear_api_key, "1", false, id: "clear_key_#{provider.id}" %><%= label_tag "clear_key_#{provider.id}", "Remove stored API key" %></div>
21
+ <%= form.submit "Save provider" %>
22
+ <% end %>
23
+ </details>
24
+ <% end %>
25
+ <details <%= "open" if @providers.empty? %>>
26
+ <summary>Add provider</summary>
27
+ <%= form_with scope: :provider, url: providers_path, local: true, class: "grid", namespace: "new_provider" do |form| %>
28
+ <div><%= form.label :name %><%= form.text_field :name, required: true, placeholder: "OpenAI" %></div>
29
+ <div><%= form.label :base_url, "API base URL" %><%= form.url_field :base_url, required: true, placeholder: "https://api.openai.com/v1" %></div>
30
+ <div><%= form.label :api_key, "API key (optional for local providers)" %><%= form.password_field :api_key, autocomplete: "new-password" %></div>
31
+ <%= form.submit "Add provider" %>
32
+ <% end %>
33
+ </details>
34
+ </section>
35
+
36
+ <section id="models">
37
+ <h2>Model routing</h2>
38
+ <p>Clients use the public model name. Lower priority runs first; matching names form an ordered fallback chain.</p>
39
+ <% @routes.each do |route| %>
40
+ <details>
41
+ <summary><%= route.name %> <span class="badge"><%= route.provider.name %> · priority <%= route.priority %></span></summary>
42
+ <%= form_with scope: :model_route, url: model_route_path(route), method: :patch, local: true, class: "grid", namespace: "route_#{route.id}" do |form| %>
43
+ <div><%= form.label :name, "Public model name" %><%= form.text_field :name, value: route.name, required: true %></div>
44
+ <div><%= form.label :provider_id %><%= form.collection_select :provider_id, @providers, :id, :name, selected: route.provider_id %></div>
45
+ <div><%= form.label :upstream_model %><%= form.text_field :upstream_model, value: route.upstream_model, required: true %></div>
46
+ <div><%= form.label :priority %><%= form.number_field :priority, value: route.priority, min: 0, required: true %></div>
47
+ <%= form.submit "Save route" %>
48
+ <% end %>
49
+ <%= button_to "Delete route", model_route_path(route), method: :delete, class: "danger" %>
50
+ </details>
51
+ <% end %>
52
+ <details>
53
+ <summary>Add model route</summary>
54
+ <%= form_with scope: :model_route, url: model_routes_path, local: true, class: "grid", namespace: "new_route" do |form| %>
55
+ <div><%= form.label :name, "Public model name" %><%= form.text_field :name, required: true, placeholder: "fast-chat" %></div>
56
+ <div><%= form.label :provider_id %><%= form.collection_select :provider_id, @providers, :id, :name %></div>
57
+ <div><%= form.label :upstream_model %><%= form.text_field :upstream_model, required: true, placeholder: "gpt-4o-mini" %></div>
58
+ <div><%= form.label :priority %><%= form.number_field :priority, value: 0, min: 0, required: true %></div>
59
+ <%= form.submit "Add route", disabled: @providers.empty? %>
60
+ <% end %>
61
+ </details>
62
+ </section>
63
+
64
+ <section id="keys">
65
+ <h2>Gateway keys</h2>
66
+ <div class="table-wrap">
67
+ <table>
68
+ <caption>Client access keys</caption>
69
+ <thead><tr><th scope="col">Name</th><th scope="col">Prefix</th><th scope="col">Models</th><th scope="col">Expires</th><th scope="col">Status</th><th scope="col">Action</th></tr></thead>
70
+ <tbody>
71
+ <% @keys.each do |key| %>
72
+ <tr><td><%= key.name %></td><td><code><%= key.prefix %>…</code></td><td><%= key.allowed_models.empty? ? "All models" : key.allowed_models.join(", ") %></td><td><%= key.expires_at || "Never" %></td><td><%= key.revoked_at ? "Revoked" : key.expires_at&.past? ? "Expired" : "Active" %></td><td><%= button_to "Revoke #{key.name}", gateway_key_path(key), method: :delete, class: "danger" unless key.revoked_at %></td></tr>
73
+ <% end %>
74
+ </tbody>
75
+ </table>
76
+ </div>
77
+ <details>
78
+ <summary>Create gateway key</summary>
79
+ <%= form_with scope: :gateway_key, url: gateway_keys_path, local: true, class: "grid" do |form| %>
80
+ <div><%= form.label :name %><%= form.text_field :name, required: true, placeholder: "Production app" %></div>
81
+ <div><%= form.label :allowed_models, "Allowed models (comma separated; blank allows all)" %><%= form.text_field :allowed_models, placeholder: "fast-chat, embeddings" %></div>
82
+ <div><%= form.label :expires_at, "Expires at (#{Time.zone.name})" %><%= form.datetime_local_field :expires_at %></div>
83
+ <%= form.submit "Create key" %>
84
+ <% end %>
85
+ </details>
86
+ </section>
87
+
88
+ <section id="logs">
89
+ <h2>Request logs</h2>
90
+ <%= form_with url: admin_path(anchor: "logs"), method: :get, local: true, class: "grid" do |form| %>
91
+ <div><%= form.label :model, "Model filter" %><%= form.text_field :model, value: params[:model] %></div>
92
+ <div><%= form.label :status, "HTTP status filter" %><%= form.number_field :status, value: params[:status], min: 100, max: 599 %></div>
93
+ <%= form.submit "Filter logs" %>
94
+ <% end %>
95
+ <div class="table-wrap">
96
+ <table>
97
+ <caption>Latest 50 matching requests</caption>
98
+ <thead><tr><th scope="col">Time</th><th scope="col">Model</th><th scope="col">Status</th><th scope="col">Latency</th><th scope="col">Reported tokens</th><th scope="col">Details</th></tr></thead>
99
+ <tbody>
100
+ <% @logs.each do |log| %>
101
+ <tr><td><%= log.created_at.strftime("%Y-%m-%d %H:%M:%S %Z") %></td><td><%= log.model %></td><td><%= log.status || "Pending" %></td><td><%= log.duration_ms %> ms</td><td><%= log.usage&.fetch("total_tokens", nil) || "Not reported" %></td><td><details><summary>Attempts and request ID</summary><code><%= log.request_id %></code><pre><%= JSON.pretty_generate(log.attempts) %></pre></details></td></tr>
102
+ <% end %>
103
+ </tbody>
104
+ </table>
105
+ </div>
106
+ <p class="muted">Streaming usage is not collected. Failed or interrupted streams are never retried.</p>
107
+ <% if @logs.size == 50 %>
108
+ <%= link_to "Older requests", admin_path(before: @logs.last.id, model: params[:model], status: params[:status], anchor: "logs") %>
109
+ <% end %>
110
+ </section>
@@ -0,0 +1,5 @@
1
+ <h1>Gateway key created</h1>
2
+ <p>Copy this token now. Only its digest is stored; it cannot be displayed again.</p>
3
+ <label for="new-token"><%= @key.name %></label>
4
+ <textarea id="new-token" readonly rows="3" autocomplete="off" spellcheck="false"><%= @token %></textarea>
5
+ <p><%= link_to "Back to gateway", admin_path %></p>
data/config/routes.rb ADDED
@@ -0,0 +1,16 @@
1
+ RailsAiGateway::Engine.routes.draw do
2
+ get "v1/models", to: RailsAiGateway::Proxy.new("models")
3
+ post "v1/chat/completions", to: RailsAiGateway::Proxy.new("chat/completions")
4
+ post "v1/embeddings", to: RailsAiGateway::Proxy.new("embeddings")
5
+
6
+ root to: redirect("admin")
7
+ get "admin", to: "admin#index", as: :admin
8
+ post "admin/providers", to: "admin#create_provider", as: :providers
9
+ patch "admin/providers/:id", to: "admin#update_provider", as: :provider
10
+ post "admin/model_routes", to: "admin#create_model_route", as: :model_routes
11
+ patch "admin/model_routes/:id", to: "admin#update_model_route", as: :model_route
12
+ delete "admin/model_routes/:id", to: "admin#destroy_model_route"
13
+ post "admin/gateway_keys", to: "admin#create_gateway_key", as: :gateway_keys
14
+ delete "admin/gateway_keys/:id", to: "admin#revoke_gateway_key", as: :gateway_key
15
+ get "admin/style", to: "admin#style", as: :style
16
+ end
@@ -0,0 +1,48 @@
1
+ class CreateRailsAiGateway < ActiveRecord::Migration[8.0]
2
+ def change
3
+ create_table :rails_ai_gateway_providers do |t|
4
+ t.string :name, null: false
5
+ t.string :base_url, null: false
6
+ t.text :api_key
7
+ t.boolean :enabled, default: true, null: false
8
+ t.timestamps
9
+ end
10
+ add_index :rails_ai_gateway_providers, :name, unique: true
11
+
12
+ create_table :rails_ai_gateway_model_routes do |t|
13
+ t.references :provider, null: false, foreign_key: { to_table: :rails_ai_gateway_providers }
14
+ t.string :name, null: false
15
+ t.string :upstream_model, null: false
16
+ t.integer :priority, null: false, default: 0
17
+ t.timestamps
18
+ end
19
+ add_index :rails_ai_gateway_model_routes, [:name, :priority], unique: true
20
+ add_check_constraint :rails_ai_gateway_model_routes, "priority >= 0", name: "gateway_route_priority"
21
+
22
+ create_table :rails_ai_gateway_gateway_keys do |t|
23
+ t.string :name, null: false
24
+ t.string :token_digest, null: false
25
+ t.string :prefix, null: false
26
+ t.json :allowed_models, null: false, default: []
27
+ t.datetime :expires_at
28
+ t.datetime :revoked_at
29
+ t.timestamps
30
+ end
31
+ add_index :rails_ai_gateway_gateway_keys, :token_digest, unique: true
32
+
33
+ create_table :rails_ai_gateway_request_logs do |t|
34
+ t.string :request_id, null: false
35
+ t.references :gateway_key, foreign_key: { to_table: :rails_ai_gateway_gateway_keys }
36
+ t.string :model, null: false
37
+ t.string :endpoint, null: false
38
+ t.integer :status
39
+ t.integer :duration_ms
40
+ t.json :attempts, null: false, default: []
41
+ t.json :usage
42
+ t.timestamps
43
+ end
44
+ add_index :rails_ai_gateway_request_logs, :request_id, unique: true
45
+ add_index :rails_ai_gateway_request_logs, :created_at
46
+ add_index :rails_ai_gateway_request_logs, [:model, :created_at]
47
+ end
48
+ end
@@ -0,0 +1,15 @@
1
+ require "rails/generators"
2
+
3
+ module RailsAiGateway
4
+ class InstallGenerator < Rails::Generators::Base
5
+ source_root File.expand_path("templates", __dir__)
6
+ desc "Install Rails AI Gateway configuration, migrations, and mount route"
7
+
8
+ def install
9
+ template "initializer.rb", "config/initializers/rails_ai_gateway.rb"
10
+ rake "rails_ai_gateway:install:migrations"
11
+ route 'mount RailsAiGateway::Engine, at: "/ai"'
12
+ say "Configure admin_authorization and ActiveRecord encryption, then run bin/rails db:migrate."
13
+ end
14
+ end
15
+ end
@@ -0,0 +1,17 @@
1
+ RailsAiGateway.configure do |config|
2
+ # Deny by default. With Devise, for example:
3
+ # config.admin_controller = "ApplicationController"
4
+ # config.admin_authorization = ->(controller) { controller.current_user&.admin? == true }
5
+
6
+ config.open_timeout = 5
7
+ config.read_timeout = 60
8
+ config.write_timeout = 30
9
+ config.request_timeout = 120 # Total deadline across fallback attempts, including streams.
10
+ config.max_request_bytes = 2 * 1024 * 1024
11
+ config.max_response_bytes = 16 * 1024 * 1024 # Buffered, non-streaming responses.
12
+ config.max_attempts = 3
13
+
14
+ # Enable only for trusted internal providers, such as a local Ollama server.
15
+ config.allow_private_networks = false
16
+ config.allow_http = false
17
+ end
@@ -0,0 +1,37 @@
1
+ module RailsAiGateway
2
+ class Configuration
3
+ attr_accessor :admin_authorization, :admin_controller, :open_timeout, :read_timeout,
4
+ :write_timeout, :request_timeout, :max_request_bytes, :max_response_bytes,
5
+ :max_attempts, :allow_private_networks, :allow_http
6
+
7
+ def initialize
8
+ @admin_controller = "ActionController::Base"
9
+ @admin_authorization = ->(_controller) { false }
10
+ @open_timeout = 5
11
+ @read_timeout = 60
12
+ @write_timeout = 30
13
+ @request_timeout = 120
14
+ @max_request_bytes = 2 * 1024 * 1024
15
+ @max_response_bytes = 16 * 1024 * 1024
16
+ @max_attempts = 3
17
+ @allow_private_networks = false
18
+ @allow_http = false
19
+ end
20
+
21
+ def validate!
22
+ %i[open_timeout read_timeout write_timeout request_timeout].each do |name|
23
+ value = public_send(name)
24
+ raise ArgumentError, "#{name} must be positive and finite" unless value.is_a?(Numeric) && value.finite? && value.positive?
25
+ end
26
+ %i[max_request_bytes max_response_bytes max_attempts].each do |name|
27
+ value = public_send(name)
28
+ raise ArgumentError, "#{name} must be a positive integer" unless value.is_a?(Integer) && value.positive?
29
+ end
30
+ %i[allow_private_networks allow_http].each do |name|
31
+ raise ArgumentError, "#{name} must be boolean" unless [true, false].include?(public_send(name))
32
+ end
33
+ raise ArgumentError, "admin_authorization must be callable" unless admin_authorization.respond_to?(:call)
34
+ raise ArgumentError, "admin_controller must be a class name" unless admin_controller.is_a?(String) && !admin_controller.empty?
35
+ end
36
+ end
37
+ end
@@ -0,0 +1,16 @@
1
+ require "rails"
2
+ require "active_record/railtie"
3
+ require "action_controller/railtie"
4
+ require "rails_ai_gateway/proxy"
5
+
6
+ module RailsAiGateway
7
+ class Engine < ::Rails::Engine
8
+ isolate_namespace RailsAiGateway
9
+
10
+ initializer "rails_ai_gateway.filter_parameters" do |app|
11
+ app.config.filter_parameters += %i[api_key token token_digest authorization]
12
+ end
13
+
14
+ config.after_initialize { RailsAiGateway.configuration.validate! }
15
+ end
16
+ end
@@ -0,0 +1,246 @@
1
+ require "net/http"
2
+ require "json"
3
+ require "ipaddr"
4
+ require "socket"
5
+ require "timeout"
6
+ require "securerandom"
7
+
8
+ module RailsAiGateway
9
+ class Proxy
10
+ class Failure < StandardError
11
+ attr_reader :status
12
+
13
+ def initialize(message, status = 502)
14
+ @status = status
15
+ super(message)
16
+ end
17
+ end
18
+
19
+ # Bounded queue applies backpressure. Rack closes the body on disconnect.
20
+ class Stream
21
+ def initialize(queue, thread)
22
+ @queue, @thread = queue, thread
23
+ end
24
+
25
+ def each
26
+ loop do
27
+ chunk = @queue.pop
28
+ break if chunk.nil?
29
+ yield chunk
30
+ end
31
+ ensure
32
+ close
33
+ end
34
+
35
+ def close
36
+ @thread.kill if @thread.alive?
37
+ @thread.join
38
+ end
39
+ end
40
+
41
+ RETRY_STATUSES = [429, 500, 502, 503, 504].freeze
42
+ PRIVATE_RANGES = %w[0.0.0.0/8 10.0.0.0/8 100.64.0.0/10 127.0.0.0/8 169.254.0.0/16
43
+ 172.16.0.0/12 192.0.0.0/24 192.0.2.0/24 192.168.0.0/16 198.18.0.0/15
44
+ 198.51.100.0/24 203.0.113.0/24 224.0.0.0/4 240.0.0.0/4 ::/128 ::1/128
45
+ fc00::/7 fe80::/10 ff00::/8 2001:db8::/32].map { |range| IPAddr.new(range) }.freeze
46
+
47
+ def initialize(endpoint)
48
+ @endpoint = endpoint
49
+ end
50
+
51
+ def call(env)
52
+ request = Rack::Request.new(env)
53
+ token = request.get_header("HTTP_AUTHORIZATION").to_s[/\ABearer (\S+)\z/i, 1]
54
+ key = GatewayKey.authenticate(token)
55
+ return error("Invalid or expired gateway key", 401) unless key
56
+
57
+ if @endpoint == "models"
58
+ names = ModelRoute.available.distinct.pluck(:name).select { |name| key.allows?(name) }.sort
59
+ return json(200, object: "list", data: names.map { |name| { id: name, object: "model", created: 0, owned_by: "rails_ai_gateway" } })
60
+ end
61
+
62
+ payload = parse_request(request)
63
+ return error("Model is not allowed", 403) unless key.allows?(payload["model"])
64
+ routes = ModelRoute.available.where(name: payload["model"]).includes(:provider).order(:priority).limit(config.max_attempts).to_a
65
+ return error("No route for requested model", 404) if routes.empty?
66
+
67
+ log = RequestLog.create!(request_id: SecureRandom.uuid, gateway_key: key, model: payload["model"], endpoint: @endpoint)
68
+ if payload["stream"]
69
+ stream_response(routes, payload, log.id, log.request_id)
70
+ else
71
+ perform(routes, payload, log.id, log.request_id)
72
+ end
73
+ rescue JSON::ParserError
74
+ error("Invalid JSON", 400)
75
+ rescue Failure => exception
76
+ error(exception.message, exception.status)
77
+ end
78
+
79
+ private
80
+
81
+ def config
82
+ RailsAiGateway.configuration
83
+ end
84
+
85
+ def parse_request(request)
86
+ raise Failure.new("Content-Type must be application/json", 415) unless request.media_type == "application/json"
87
+ raise Failure.new("Request body too large", 413) if request.content_length.to_i > config.max_request_bytes
88
+ raw = request.body.read(config.max_request_bytes + 1)
89
+ raise Failure.new("Request body too large", 413) if raw.bytesize > config.max_request_bytes
90
+ payload = JSON.parse(raw)
91
+ unless payload.is_a?(Hash) && payload["model"].is_a?(String) && payload["model"].present? && payload["model"].length <= 255
92
+ raise Failure.new("A model name is required", 400)
93
+ end
94
+ if payload.key?("stream") && ![true, false].include?(payload["stream"])
95
+ raise Failure.new("stream must be boolean", 400)
96
+ end
97
+ if @endpoint == "chat/completions"
98
+ messages = payload["messages"]
99
+ unless messages.is_a?(Array) && messages.any? && messages.all? { |message| message.is_a?(Hash) && %w[developer system user assistant tool function].include?(message["role"]) }
100
+ raise Failure.new("messages must be a nonempty array with valid roles", 400)
101
+ end
102
+ else
103
+ input = payload["input"]
104
+ valid = input.is_a?(String) && !input.empty? || input.is_a?(Array) && !input.empty? && (
105
+ input.all? { |item| item.is_a?(String) && !item.empty? } ||
106
+ input.all? { |item| item.is_a?(Integer) && item >= 0 } ||
107
+ input.all? { |item| item.is_a?(Array) && item.any? && item.all? { |id| id.is_a?(Integer) && id >= 0 } })
108
+ raise Failure.new("input must contain text or token IDs; embeddings cannot stream", 400) unless valid && !payload["stream"]
109
+ end
110
+ payload
111
+ end
112
+
113
+ def stream_response(routes, payload, log_id, request_id)
114
+ ready, chunks = Queue.new, SizedQueue.new(4)
115
+ thread = Thread.new do
116
+ Thread.current.report_on_exception = false
117
+ Rails.application.executor.wrap do
118
+ sent_headers = false
119
+ begin
120
+ result = perform(routes, payload, log_id, request_id) do |headers, response|
121
+ sent_headers = true
122
+ ready << [response.code.to_i, headers]
123
+ response.read_body { |chunk| chunks << chunk }
124
+ end
125
+ ready << result unless sent_headers
126
+ rescue StandardError
127
+ ready << error("Gateway failed", 500) unless sent_headers
128
+ ensure
129
+ chunks.close
130
+ end
131
+ end
132
+ end
133
+ result = ready.pop
134
+ if result.length == 2
135
+ [*result, Stream.new(chunks, thread)]
136
+ else
137
+ thread.join
138
+ result
139
+ end
140
+ rescue Exception
141
+ thread&.kill
142
+ thread&.join
143
+ raise
144
+ end
145
+
146
+ def perform(routes, payload, log_id, request_id)
147
+ started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
148
+ attempts, usage, status, streaming = [], nil, 502, false
149
+ Timeout.timeout(config.request_timeout, Failure, "Upstream deadline exceeded") do
150
+ routes.each_with_index do |route, index|
151
+ attempt = { provider_id: route.provider_id, upstream_model: route.upstream_model }
152
+ attempts << attempt
153
+ begin
154
+ uri, address = destination(route.provider)
155
+ http = Net::HTTP.new(uri.hostname, uri.port, nil) # Do not inherit HTTP_PROXY or leak credentials to it.
156
+ http.ipaddr = address # Pin validated DNS result, preserving hostname for TLS verification.
157
+ http.use_ssl = uri.scheme == "https"
158
+ http.open_timeout, http.read_timeout, http.write_timeout = config.open_timeout, config.read_timeout, config.write_timeout
159
+ http.max_retries = 0
160
+ request = Net::HTTP::Post.new(uri.request_uri)
161
+ request["Content-Type"] = "application/json"
162
+ request["Accept"] = payload["stream"] ? "text/event-stream" : "application/json"
163
+ request["Accept-Encoding"] = "identity"
164
+ request["Authorization"] = "Bearer #{route.provider.api_key}" if route.provider.api_key.present?
165
+ request.body = JSON.generate(payload.merge("model" => route.upstream_model))
166
+ result = nil
167
+ http.start do
168
+ http.request(request) do |response|
169
+ status = attempt[:status] = response.code.to_i
170
+ raise Failure, "Upstream redirects are not supported" if (300..399).cover?(status)
171
+ headers = { "content-type" => "application/json", "cache-control" => "no-store", "x-request-id" => request_id }
172
+ if payload["stream"] && (200..299).cover?(status)
173
+ raise Failure, "Upstream did not return an event stream" unless response["content-type"].to_s.split(";").first == "text/event-stream"
174
+ headers.merge!("content-type" => "text/event-stream", "x-accel-buffering" => "no")
175
+ streaming = true
176
+ yield headers, response
177
+ result = [status, headers, []]
178
+ else
179
+ body = +""
180
+ response.read_body do |chunk|
181
+ raise Failure, "Upstream response too large" if body.bytesize + chunk.bytesize > config.max_response_bytes
182
+ body << chunk
183
+ end
184
+ unless RETRY_STATUSES.include?(status) && index < routes.length - 1
185
+ begin
186
+ parsed = JSON.parse(body)
187
+ rescue JSON::ParserError
188
+ raise Failure, "Upstream did not return JSON"
189
+ end
190
+ raise Failure, "Upstream did not return a JSON object" unless parsed.is_a?(Hash)
191
+ usage = parsed["usage"].slice("prompt_tokens", "completion_tokens", "total_tokens").select { |_, value| value.is_a?(Integer) && value >= 0 } if parsed["usage"].is_a?(Hash)
192
+ result = [status, headers, [body]]
193
+ end
194
+ end
195
+ end
196
+ end
197
+ return result if result
198
+ rescue Net::OpenTimeout, Errno::ECONNREFUSED, Errno::EHOSTUNREACH, SocketError => exception
199
+ attempt[:error] = exception.class.name
200
+ raise Failure, "Upstream connection failed" if streaming || index == routes.length - 1
201
+ end
202
+ end
203
+ end
204
+ rescue Failure, Timeout::Error, IOError, SystemCallError, OpenSSL::SSL::SSLError, Net::HTTPBadResponse => exception
205
+ status = exception.is_a?(Timeout::Error) || exception.message == "Upstream deadline exceeded" ? 504 : 502
206
+ attempts.last[:error] = exception.class.name if attempts.last
207
+ # Never replay an ambiguous failure or append a second provider to a partial stream.
208
+ error(exception.is_a?(Failure) ? exception.message : "Upstream request failed", status, request_id)
209
+ ensure
210
+ duration = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000).round
211
+ begin
212
+ RequestLog.connection_pool.with_connection do
213
+ RequestLog.find(log_id).update!(status: status, duration_ms: duration, attempts: attempts, usage: usage)
214
+ end
215
+ rescue StandardError => exception
216
+ Rails.logger.error("RailsAiGateway request log write failed: #{exception.class}")
217
+ end
218
+ end
219
+
220
+ def destination(provider)
221
+ raise Failure, "Invalid provider URL" unless provider.valid?
222
+ uri = URI.parse("#{provider.base_url.delete_suffix('/')}/#{@endpoint}")
223
+ addresses = Addrinfo.getaddrinfo(uri.hostname, uri.port, nil, :STREAM).map(&:ip_address).uniq
224
+ raise Failure, "Provider address unavailable" if addresses.empty?
225
+ unless config.allow_private_networks
226
+ addresses.each do |address|
227
+ ip = IPAddr.new(address)
228
+ ip = ip.native if ip.ipv4_mapped?
229
+ raise Failure, "Private provider networks are disabled" if PRIVATE_RANGES.any? { |range| range.include?(ip) } || ip.ipv6? && !IPAddr.new("2000::/3").include?(ip)
230
+ end
231
+ end
232
+ [uri, addresses.first]
233
+ end
234
+
235
+ def error(message, status, request_id = nil)
236
+ response = json(status, error: { message: message, type: "gateway_error", code: status })
237
+ response[1]["www-authenticate"] = "Bearer" if status == 401
238
+ response[1]["x-request-id"] = request_id if request_id
239
+ response
240
+ end
241
+
242
+ def json(status, payload)
243
+ [status, { "content-type" => "application/json", "cache-control" => "no-store" }, [JSON.generate(payload)]]
244
+ end
245
+ end
246
+ end
@@ -0,0 +1,3 @@
1
+ module RailsAiGateway
2
+ VERSION = "0.1.0"
3
+ end
@@ -0,0 +1,15 @@
1
+ require "rails_ai_gateway/version"
2
+ require "rails_ai_gateway/configuration"
3
+
4
+ module RailsAiGateway
5
+ def self.configuration
6
+ @configuration ||= Configuration.new
7
+ end
8
+
9
+ def self.configure
10
+ yield configuration
11
+ configuration.validate!
12
+ end
13
+ end
14
+
15
+ require "rails_ai_gateway/engine"
metadata ADDED
@@ -0,0 +1,167 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: rails-ai-gateway
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Rails AI Gateway contributors
8
+ bindir: bin
9
+ cert_chain: []
10
+ date: 1980-01-02 00:00:00.000000000 Z
11
+ dependencies:
12
+ - !ruby/object:Gem::Dependency
13
+ name: railties
14
+ requirement: !ruby/object:Gem::Requirement
15
+ requirements:
16
+ - - ">="
17
+ - !ruby/object:Gem::Version
18
+ version: '8.0'
19
+ - - "<"
20
+ - !ruby/object:Gem::Version
21
+ version: '9'
22
+ type: :runtime
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ requirements:
26
+ - - ">="
27
+ - !ruby/object:Gem::Version
28
+ version: '8.0'
29
+ - - "<"
30
+ - !ruby/object:Gem::Version
31
+ version: '9'
32
+ - !ruby/object:Gem::Dependency
33
+ name: activerecord
34
+ requirement: !ruby/object:Gem::Requirement
35
+ requirements:
36
+ - - ">="
37
+ - !ruby/object:Gem::Version
38
+ version: '8.0'
39
+ - - "<"
40
+ - !ruby/object:Gem::Version
41
+ version: '9'
42
+ type: :runtime
43
+ prerelease: false
44
+ version_requirements: !ruby/object:Gem::Requirement
45
+ requirements:
46
+ - - ">="
47
+ - !ruby/object:Gem::Version
48
+ version: '8.0'
49
+ - - "<"
50
+ - !ruby/object:Gem::Version
51
+ version: '9'
52
+ - !ruby/object:Gem::Dependency
53
+ name: actionpack
54
+ requirement: !ruby/object:Gem::Requirement
55
+ requirements:
56
+ - - ">="
57
+ - !ruby/object:Gem::Version
58
+ version: '8.0'
59
+ - - "<"
60
+ - !ruby/object:Gem::Version
61
+ version: '9'
62
+ type: :runtime
63
+ prerelease: false
64
+ version_requirements: !ruby/object:Gem::Requirement
65
+ requirements:
66
+ - - ">="
67
+ - !ruby/object:Gem::Version
68
+ version: '8.0'
69
+ - - "<"
70
+ - !ruby/object:Gem::Version
71
+ version: '9'
72
+ - !ruby/object:Gem::Dependency
73
+ name: net-http
74
+ requirement: !ruby/object:Gem::Requirement
75
+ requirements:
76
+ - - ">="
77
+ - !ruby/object:Gem::Version
78
+ version: '0.4'
79
+ - - "<"
80
+ - !ruby/object:Gem::Version
81
+ version: '1'
82
+ type: :runtime
83
+ prerelease: false
84
+ version_requirements: !ruby/object:Gem::Requirement
85
+ requirements:
86
+ - - ">="
87
+ - !ruby/object:Gem::Version
88
+ version: '0.4'
89
+ - - "<"
90
+ - !ruby/object:Gem::Version
91
+ version: '1'
92
+ - !ruby/object:Gem::Dependency
93
+ name: json
94
+ requirement: !ruby/object:Gem::Requirement
95
+ requirements:
96
+ - - ">="
97
+ - !ruby/object:Gem::Version
98
+ version: '2.7'
99
+ - - "<"
100
+ - !ruby/object:Gem::Version
101
+ version: '3'
102
+ type: :runtime
103
+ prerelease: false
104
+ version_requirements: !ruby/object:Gem::Requirement
105
+ requirements:
106
+ - - ">="
107
+ - !ruby/object:Gem::Version
108
+ version: '2.7'
109
+ - - "<"
110
+ - !ruby/object:Gem::Version
111
+ version: '3'
112
+ description: OpenAI-compatible AI gateway mounted inside Rails, with ActiveRecord
113
+ routing, encrypted provider keys, request logs, fallbacks, streaming, and an admin
114
+ UI.
115
+ executables: []
116
+ extensions: []
117
+ extra_rdoc_files: []
118
+ files:
119
+ - CHANGELOG.md
120
+ - CONTRIBUTING.md
121
+ - LICENSE
122
+ - README.md
123
+ - app/assets/stylesheets/rails_ai_gateway/admin.css
124
+ - app/controllers/rails_ai_gateway/admin_controller.rb
125
+ - app/models/rails_ai_gateway/application_record.rb
126
+ - app/models/rails_ai_gateway/gateway_key.rb
127
+ - app/models/rails_ai_gateway/model_route.rb
128
+ - app/models/rails_ai_gateway/provider.rb
129
+ - app/models/rails_ai_gateway/request_log.rb
130
+ - app/views/layouts/rails_ai_gateway/admin.html.erb
131
+ - app/views/rails_ai_gateway/admin/errors.html.erb
132
+ - app/views/rails_ai_gateway/admin/index.html.erb
133
+ - app/views/rails_ai_gateway/admin/key.html.erb
134
+ - config/routes.rb
135
+ - db/migrate/20260920000000_create_rails_ai_gateway.rb
136
+ - lib/generators/rails_ai_gateway/install_generator.rb
137
+ - lib/generators/rails_ai_gateway/templates/initializer.rb
138
+ - lib/rails_ai_gateway.rb
139
+ - lib/rails_ai_gateway/configuration.rb
140
+ - lib/rails_ai_gateway/engine.rb
141
+ - lib/rails_ai_gateway/proxy.rb
142
+ - lib/rails_ai_gateway/version.rb
143
+ homepage: https://github.com/azmi2409/rails-ai-gateway
144
+ licenses:
145
+ - MIT
146
+ metadata:
147
+ source_code_uri: https://github.com/azmi2409/rails-ai-gateway
148
+ changelog_uri: https://github.com/azmi2409/rails-ai-gateway/blob/main/CHANGELOG.md
149
+ rubygems_mfa_required: 'true'
150
+ rdoc_options: []
151
+ require_paths:
152
+ - lib
153
+ required_ruby_version: !ruby/object:Gem::Requirement
154
+ requirements:
155
+ - - ">="
156
+ - !ruby/object:Gem::Version
157
+ version: '3.3'
158
+ required_rubygems_version: !ruby/object:Gem::Requirement
159
+ requirements:
160
+ - - ">="
161
+ - !ruby/object:Gem::Version
162
+ version: '0'
163
+ requirements: []
164
+ rubygems_version: 4.0.4
165
+ specification_version: 4
166
+ summary: Mountable Rails AI gateway with ActiveRecord and an admin UI
167
+ test_files: []