rails_ai_kit 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: d497000abdc4e0a8f02f6167570d5d00888979e31ed5e484fc707647ebdface5
+  data.tar.gz: b721aeaeddabe1c8dce81f6aca4ff94188623c6cdb8fd5b22dd5199a7d1fb3bf
+SHA512:
+  metadata.gz: 0f6a1a9ec8f3f9a4d0404f2ef5b7b8dc61ad41bee7a7557d3701a7e958786114430ca4b5c3db4e1581b98bda4b280856eca1a1cd7152987c6d5db54ebc4b1b63
+  data.tar.gz: 43316a2c58575ae72429deb036cd3499b42bd19ec5c7fadf15404b46a864fdcffa3b41a33b24815fef16dd426ace74bac38fe8460e6e722c9639785ee5c4d7ef

data/CHANGELOG.md

@@ -0,0 +1,16 @@
+# Changelog
+
+## [Unreleased]
+
+## [0.1.0] - 2025-03-07
+
+### Added
+
+- Vector-based classification layer on top of pgvector
+- Configuration for embedding provider (OpenAI, Cohere) and API keys
+- `RailsAiKit::Classifier` with `train`, `classify`, `classify_by_embedding`, `batch_classify`, `labels`
+- `vector_classify` macro for ActiveRecord models (auto embed + classify on save)
+- Similarity search: `Model.similar_to("query text", limit: 5)`
+- Generators: `rails_ai_kit:install`, `rails_ai_kit:vector_columns`
+- Label storage in `rails_ai_kit_labels` (classifier_name, label_name, embedding)
+- Embedding providers: OpenAI (text-embedding-3-small), Cohere (embed-english-v3.0)

data/README.md

@@ -0,0 +1,271 @@
+# Rails AI Kit
+
+A Rails gem that adds a **classification layer on top of [pgvector](https://github.com/pgvector/pgvector)**. Instead of building custom ML or calling LLMs every time, use vector similarity to classify data: support tickets, content moderation, ecommerce categories, document routing, and more.
+
+- **No ML training** – train labels with example texts and compare embeddings
+- **No LLM cost** – one-time embedding per piece of content; classification is nearest-neighbor in PostgreSQL
+- **Rails-friendly** – `vector_classify`, `Classifier.train`, `Classifier.classify`, `Article.similar_to("query")`
+- **Multi-provider embeddings** – pass API keys for OpenAI, Cohere, or your own provider
+
+## Architecture
+
+```
+Your Rails App
+      │
+      ▼
+Rails AI Kit (classification + indexing + similarity)
+      │
+      ▼
+pgvector (PostgreSQL)
+      │
+      ▼
+vector similarity search → predicted label
+```
+
+The gem does **not** store your application data. You store data in your own PostgreSQL tables with pgvector. The gem provides:
+
+- Classification logic (label training, classify by similarity)
+- Indexing helpers (migrations for vector columns and label table)
+- Similarity search (`similar_to`)
+- Filtering (standard `where(label: "sports")`)
+
+## Requirements
+
+- Rails 6+
+- PostgreSQL with [pgvector](https://github.com/pgvector/pgvector) extension
+- An embedding API (OpenAI or Cohere) and API key
+
+## Installation
+
+Add to your Gemfile:
+
+```ruby
+gem "rails_ai_kit"
+```
+
+Then:
+
+```bash
+bundle install
+```
+
+### 1. Enable pgvector and create the labels table
+
+The gem needs one internal table to store **label embeddings** (one vector per label per classifier):
+
+```bash
+rails g rails_ai_kit:install
+rails db:migrate
+```
+
+### 2. Configure embedding provider and API keys
+
+In `config/initializers/rails_ai_kit.rb` (create the file):
+
+```ruby
+RailsAiKit.configure do |config|
+  config.embedding_provider = :openai   # or :cohere
+  config.embedding_dimensions = 1536   # 1536 for OpenAI text-embedding-3-small
+
+  # Pass API keys for the provider(s) you use
+  config.api_keys = {
+    openai: ENV["OPENAI_API_KEY"],
+    cohere: ENV["COHERE_API_KEY"]
+  }
+
+  config.default_classifier_name = "default"
+end
+```
+
+Use environment variables or Rails credentials; do not commit raw API keys.
+
+### 3. Add vector columns to your model
+
+For a new or existing table (e.g. `articles` with a `content` column):
+
+```bash
+rails g rails_ai_kit:vector_columns Article content
+rails db:migrate
+```
+
+This adds `embedding` (vector), `label` (string), and `confidence_score` (float).
+
+## Usage
+
+### Declare vector classification on a model
+
+```ruby
+class Article < ApplicationRecord
+  vector_classify :content,
+    labels: ["sports", "politics", "technology"]
+end
+```
+
+On save, the gem will:
+
+1. Generate an embedding for `content`
+2. Store it in `embedding`
+3. Run classification (nearest label vector) and set `label` and `confidence_score`
+
+Example:
+
+```ruby
+article = Article.create!(content: "Apple released a new iPhone")
+article.label            # => "technology"
+article.confidence_score # => 0.91
+```
+
+### Train labels with examples
+
+Before classifying, train each label with example texts so the gem can build a label vector (average of example embeddings):
+
+```ruby
+classifier = RailsAiKit.classifier("Article")  # or use default
+
+classifier.train("sports", examples: [
+  "football match",
+  "cricket tournament",
+  "Olympic gold medal"
+])
+
+classifier.train("politics", examples: [
+  "election results",
+  "parliament debate"
+])
+
+classifier.train("technology", examples: [
+  "new iPhone launch",
+  "AI software update"
+])
+```
+
+You can use the default classifier name or pass a custom one:
+
+```ruby
+RailsAiKit.classifier("Article").train("sports", examples: [...])
+RailsAiKit.classifier.train("sports", examples: [...])  # default classifier
+```
+
+### Classify text without saving
+
+```ruby
+result = RailsAiKit.classifier("Article").classify("India won the cricket match")
+# => { label: "sports", confidence: 0.91, distance: 0.09 }
+```
+
+### Batch classification
+
+```ruby
+records = Article.where(label: nil).limit(100)
+RailsAiKit.classifier("Article").batch_classify(records,
+  text_attribute: :content,
+  label_attribute: :label,
+  confidence_attribute: :confidence_score
+)
+# Optionally save: records.each(&:save!)
+```
+
+### Filtering
+
+Use normal ActiveRecord scopes:
+
+```ruby
+Article.where(label: "sports")
+Article.where("confidence_score >= ?", 0.8)
+```
+
+### Similarity search
+
+Find records similar to a piece of text (embeds the query, then nearest-neighbor search):
+
+```ruby
+Article.similar_to("new iPhone launch", limit: 5)
+```
+
+## Example: Support ticket routing
+
+```ruby
+# app/models/support_ticket.rb
+class SupportTicket < ApplicationRecord
+  vector_classify :message,
+    labels: ["billing", "technical", "account"],
+    classifier_name: "SupportTicket"
+end
+
+# Train once
+classifier = RailsAiKit.classifier("SupportTicket")
+classifier.train("billing", examples: ["My payment failed", "Refund request", "Invoice issue"])
+classifier.train("technical", examples: ["App crashed", "Login not working", "Error message"])
+classifier.train("account", examples: ["Change email", "Close my account", "Password reset"])
+
+# Incoming ticket
+ticket = SupportTicket.create!(message: "My payment failed last night")
+ticket.label  # => "billing" → route to billing queue
+```
+
+## Configuration reference
+
+| Option | Description | Default |
+|--------|-------------|---------|
+| `embedding_provider` | `:openai` or `:cohere` | `:openai` |
+| `embedding_dimensions` | Vector size (must match provider) | `1536` |
+| `api_keys` | Hash of provider => API key | `{}` |
+| `default_classifier_name` | Name when no classifier given | `"default"` |
+
+## Generators
+
+| Generator | Purpose |
+|-----------|---------|
+| `rails g rails_ai_kit:install` | Migration: enable pgvector + create `rails_ai_kit_labels` |
+| `rails g rails_ai_kit:vector_columns ModelName content_column` | Migration: add `embedding`, `label`, `confidence_score` to a table |
+
+## How it works
+
+1. **Label vectors** – Each label is represented by a vector (average of example embeddings). Stored in `rails_ai_kit_labels`.
+2. **Classification** – New content is embedded and compared to all label vectors with cosine distance. The nearest label wins; confidence is `1 - distance`.
+3. **Storage** – Your table holds the content, its embedding, the predicted label, and confidence. The gem only adds one table for label vectors.
+
+## Future ideas
+
+- Hierarchical labels (e.g. technology → mobile, laptops)
+- Confidence threshold (e.g. mark as "unknown" if &lt; 0.7)
+- Hybrid search (vector + keyword)
+- Incremental learning (add examples to improve labels over time)
+
+## Development
+
+```bash
+bundle install
+bundle exec rake install
+```
+
+Run tests (when added) with `bundle exec rspec` or `bundle exec rake test`.
+
+## License
+
+MIT.
+
+## How it’s built
+
+- **Configuration** (`lib/rails_ai_kit/configuration.rb`) – Embeding provider, dimensions, and API keys (e.g. `api_keys[:openai]`).
+- **Embedding providers** (`lib/rails_ai_kit/embedding_providers/`) – Base class plus OpenAI and Cohere. Each implements `embed(text)` and `embed_batch(texts)` using the provider API.
+- **EmbeddingService** – Wraps the configured provider and API key so `RailsAiKit.embedding.embed(text)` works without passing keys every time.
+- **LabelRecord** – ActiveRecord model for `rails_ai_kit_labels` (classifier_name, label_name, embedding). Uses Neighbor’s `has_neighbors :embedding` for similarity.
+- **Classifier** – Trains labels by averaging example embeddings and storing them; classifies by nearest-neighbor (cosine) against those label vectors. Supports `classify(text)`, `classify_by_embedding(vector)`, and `batch_classify(records)`.
+- **VectorClassify** – Concern that adds the `vector_classify` macro: `has_neighbors` on the embedding column, a before_save that embeds the source column and runs `classify_by_embedding`, and a `similar_to(query_text)` scope that embeds the query and runs nearest-neighbor search.
+
+## How it’s built
+
+- **Configuration** (`lib/rails_ai_kit/configuration.rb`) – Embedding provider, dimensions, and API keys (e.g. `api_keys[:openai]`).
+- **Embedding providers** (`lib/rails_ai_kit/embedding_providers/`) – Base class plus OpenAI and Cohere. Each implements `embed(text)` and `embed_batch(texts)` using the provider API.
+- **EmbeddingService** – Wraps the configured provider and API key so `RailsAiKit.embedding.embed(text)` works without passing keys every time.
+- **LabelRecord** – ActiveRecord model for `rails_ai_kit_labels` (classifier_name, label_name, embedding). Uses Neighbor’s `has_neighbors :embedding` for similarity.
+- **Classifier** – Trains labels by averaging example embeddings and storing them; classifies by nearest-neighbor (cosine) against those label vectors. Supports `classify(text)`, `classify_by_embedding(vector)`, and `batch_classify(records)`.
+- **VectorClassify** – Concern that adds the `vector_classify` macro: `has_neighbors` on the embedding column, a before_save that embeds the source column and runs `classify_by_embedding`, and a `similar_to(query_text)` scope that embeds the query and runs nearest-neighbor search.
+- **Generators** – `rails_ai_kit:install` creates the labels table migration; `rails_ai_kit:vector_columns` adds embedding/label/confidence_score to a given table.
+
+## Related
+
+- [pgvector](https://github.com/pgvector/pgvector) – Open-source vector similarity search for Postgres
+- [Neighbor](https://github.com/ankane/neighbor) – Nearest neighbor search for Rails (used by this gem)
+test

data/lib/generators/rails_ai_kit/install/templates/create_rails_ai_kit_labels.rb

@@ -0,0 +1,16 @@
+# frozen_string_literal: true
+
+class CreateRailsAiKitLabels < ActiveRecord::Migration[<%= Rails::VERSION::MAJOR %>.<%= Rails::VERSION::MINOR %>]
+  def change
+    enable_extension "vector" unless extension_enabled?("vector")
+
+    create_table :rails_ai_kit_labels do |t|
+      t.string :classifier_name, null: false
+      t.string :label_name, null: false
+      t.vector :embedding, limit: <%= (defined?(RailsAiKit) && RailsAiKit.configuration.embedding_dimensions) || 1536 %>, null: false
+      t.timestamps
+    end
+
+    add_index :rails_ai_kit_labels, [:classifier_name, :label_name], unique: true
+  end
+end

data/lib/generators/rails_ai_kit/install/templates/rails_ai_kit.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+# Configure Rails AI Kit (vector classification with pgvector).
+# Set your embedding provider and API keys via ENV or Rails credentials.
+RailsAiKit.configure do |config|
+  config.embedding_provider = :openai
+  config.embedding_dimensions = 1536
+
+  config.api_keys = {
+    openai: ENV["OPENAI_API_KEY"],
+    cohere: ENV["COHERE_API_KEY"]
+  }
+
+  config.default_classifier_name = "default"
+end

data/lib/generators/rails_ai_kit/install_generator.rb

@@ -0,0 +1,28 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+
+module RailsAiKit
+  module Generators
+    class InstallGenerator < Rails::Generators::Base
+      source_root File.expand_path("templates", __dir__)
+
+      desc "Creates a migration to enable pgvector and add rails_ai_kit_labels table for label embeddings"
+
+      def create_migration
+        migration_template "create_rails_ai_kit_labels.rb",
+                          "db/migrate/#{migration_timestamp}_create_rails_ai_kit_labels.rb"
+      end
+
+      def create_initializer
+        copy_file "rails_ai_kit.rb", "config/initializers/rails_ai_kit.rb"
+      end
+
+      private
+
+      def migration_timestamp
+        Time.now.utc.strftime("%Y%m%d%H%M%S")
+      end
+    end
+  end
+end

data/lib/generators/rails_ai_kit/vector_columns/templates/add_vector_columns.rb

@@ -0,0 +1,11 @@
+# frozen_string_literal: true
+
+class AddRailsAiKitVectorColumnsTo<%= table_name.camelize %> < ActiveRecord::Migration[<%= Rails::VERSION::MAJOR %>.<%= Rails::VERSION::MINOR %>]
+  def change
+    enable_extension "vector" unless extension_enabled?("vector")
+
+    add_column :<%= table_name %>, :embedding, :vector, limit: <%= dimensions %>
+    add_column :<%= table_name %>, :label, :string
+    add_column :<%= table_name %>, :confidence_score, :float
+  end
+end

data/lib/generators/rails_ai_kit/vector_columns_generator.rb

@@ -0,0 +1,39 @@
+# frozen_string_literal: true
+
+require "rails/generators"
+require "rails/generators/active_record/migration/migration_generator"
+
+module RailsAiKit
+  module Generators
+    class VectorColumnsGenerator < Rails::Generators::NamedBase
+      source_root File.expand_path("templates", __dir__)
+
+      desc "Adds embedding, label, and confidence_score columns to an existing table for vector_classify"
+
+      argument :content_column, type: :string, default: "content",
+               desc: "Name of the text column to classify (e.g. content, body)"
+
+      class_option :embedding_dimensions, type: :numeric, default: 1536,
+                   desc: "Vector dimensions (must match your embedding provider)"
+
+      def create_migration
+        migration_template "add_vector_columns.rb",
+                          "db/migrate/#{migration_timestamp}_add_rails_ai_kit_vector_columns_to_#{table_name}.rb"
+      end
+
+      private
+
+      def table_name
+        name.underscore.pluralize
+      end
+
+      def migration_timestamp
+        Time.now.utc.strftime("%Y%m%d%H%M%S")
+      end
+
+      def dimensions
+        options[:embedding_dimensions] || (defined?(RailsAiKit) && RailsAiKit.configuration.embedding_dimensions) || 1536
+      end
+    end
+  end
+end

data/lib/rails_ai_kit/classifier.rb

@@ -0,0 +1,115 @@
+# frozen_string_literal: true
+
+module RailsAiKit
+  # Vector-based classifier using label embeddings and nearest-neighbor search.
+  # Train with examples per label, then classify text by comparing its embedding to label vectors.
+  class Classifier
+    attr_reader :classifier_name
+
+    def initialize(classifier_name: nil)
+      @classifier_name = classifier_name || RailsAiKit.configuration.default_classifier_name
+    end
+
+    # Train a label with example texts. Embeds each example and stores the (optionally averaged) label vector.
+    # @param label_name [String]
+    # @param examples [Array<String>] Example texts for this label
+    # @return [RailsAiKit::LabelRecord]
+    def train(label_name, examples:)
+      raise ArgumentError, "At least one example required" if examples.to_a.empty?
+
+      embeddings = RailsAiKit.embedding.embed_batch(examples.to_a)
+      # Average the vectors to get a single label embedding (common approach for few-shot)
+      dim = embeddings.first.size
+      avg = Array.new(dim, 0.0)
+      embeddings.each do |vec|
+        vec.each_with_index { |v, i| avg[i] += v }
+      end
+      n = embeddings.size.to_f
+      avg.map! { |x| x / n }
+
+      record = LabelRecord.find_or_initialize_by(classifier_name: classifier_name, label_name: label_name)
+      record.embedding = avg
+      record.save!
+      record
+    end
+
+    # Classify using an existing embedding vector (e.g. after storing). Avoids re-embedding.
+    # @param embedding [Array<Float>]
+    # @return [Hash] { label:, confidence:, distance: }
+    def classify_by_embedding(embedding)
+      return { label: nil, confidence: 0.0, distance: nil } if embedding.blank?
+
+      nearest = LabelRecord.for_classifier(classifier_name)
+                           .nearest_neighbors(:embedding, embedding, distance: "cosine")
+                           .limit(1)
+                           .first
+
+      return { label: nil, confidence: 0.0, distance: nil } if nearest.blank?
+
+      distance = nearest.neighbor_distance
+      confidence = [0, 1 - distance].max
+      { label: nearest.label_name, confidence: confidence, distance: distance }
+    end
+
+    # Classify a single text. Returns predicted label, confidence (1 - normalized distance), and distance.
+    # @param text [String]
+    # @return [Hash] { label:, confidence:, distance: }
+    def classify(text)
+      query_embedding = RailsAiKit.embedding.embed(text)
+      nearest = LabelRecord.for_classifier(classifier_name)
+                           .nearest_neighbors(:embedding, query_embedding, distance: "cosine")
+                           .limit(1)
+                           .first
+
+      return { label: nil, confidence: 0.0, distance: nil } if nearest.blank?
+
+      # Cosine distance in pgvector is 1 - cosine_similarity; we want confidence in [0,1]
+      distance = nearest.neighbor_distance
+      confidence = [0, 1 - distance].max
+
+      { label: nearest.label_name, confidence: confidence, distance: distance }
+    end
+
+    # Classify multiple records in batch. Expects each record to respond to the text getter and optional setters.
+    # @param records [Enumerable]
+    # @param text_attribute [Symbol, String] Method to get text from each record (e.g. :content)
+    # @param label_attribute [Symbol, String] Attribute to set for predicted label (default :label)
+    # @param confidence_attribute [Symbol, String] Attribute to set for confidence (default :confidence_score)
+    # @return [Array<Hash>] Array of { label:, confidence:, distance: } per record
+    def batch_classify(records, text_attribute: :content, label_attribute: :label, confidence_attribute: :confidence_score)
+      recs = records.to_a
+      return [] if recs.empty?
+
+      texts = recs.map { |r| r.public_send(text_attribute) }
+      embeddings = RailsAiKit.embedding.embed_batch(texts)
+
+      results = embeddings.map do |query_embedding|
+        nearest = LabelRecord.for_classifier(classifier_name)
+                             .nearest_neighbors(:embedding, query_embedding, distance: "cosine")
+                             .limit(1)
+                             .first
+
+        if nearest.blank?
+          { label: nil, confidence: 0.0, distance: nil }
+        else
+          dist = nearest.neighbor_distance
+          conf = [0, 1 - dist].max
+          { label: nearest.label_name, confidence: conf, distance: dist }
+        end
+      end
+
+      recs.each_with_index do |record, i|
+        r = results[i]
+        record.public_send(:"#{label_attribute}=", r[:label]) if record.respond_to?(:"#{label_attribute}=")
+        record.public_send(:"#{confidence_attribute}=", r[:confidence]) if record.respond_to?(:"#{confidence_attribute}=")
+      end
+
+      results
+    end
+
+    # List all trained labels for this classifier.
+    def labels
+      LabelRecord.for_classifier(classifier_name).pluck(:label_name)
+    end
+  end
+end

data/lib/rails_ai_kit/configuration.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+module RailsAiKit
+  class Configuration
+    attr_accessor :embedding_provider,
+                  :embedding_dimensions,
+                  :api_keys,
+                  :default_classifier_name
+
+    def initialize
+      @embedding_provider = :openai
+      @embedding_dimensions = 1536
+      @api_keys = {}
+      @default_classifier_name = "default"
+    end
+
+    def api_key(provider = nil)
+      provider ||= embedding_provider
+      api_keys[provider.to_sym] || api_keys[provider.to_s]
+    end
+  end
+
+  def self.configuration
+    @configuration ||= Configuration.new
+  end
+
+  def self.configure
+    yield configuration
+  end
+end

data/lib/rails_ai_kit/embedding_providers/base.rb

@@ -0,0 +1,30 @@
+# frozen_string_literal: true
+
+module RailsAiKit
+  module EmbeddingProviders
+    class Base
+      class NotImplementedError < RailsAiKit::Error; end
+
+      # @param api_key [String] Provider API key
+      # @param dimensions [Integer] Optional embedding dimensions (provider-dependent)
+      def initialize(api_key:, dimensions: nil)
+        @api_key = api_key
+        @dimensions = dimensions
+      end
+
+      # Generate embedding for a single text.
+      # @param text [String]
+      # @return [Array<Float>]
+      def embed(text)
+        raise NotImplementedError, "#{self.class}#embed must be implemented"
+      end
+
+      # Generate embeddings for multiple texts (optional batching).
+      # @param texts [Array<String>]
+      # @return [Array<Array<Float>>]
+      def embed_batch(texts)
+        texts.map { |text| embed(text) }
+      end
+    end
+  end
+end

data/lib/rails_ai_kit/embedding_providers/cohere.rb

@@ -0,0 +1,68 @@
+# frozen_string_literal: true
+
+module RailsAiKit
+  module EmbeddingProviders
+    # Cohere embed API: https://docs.cohere.com/reference/embed
+    class Cohere < Base
+      DEFAULT_MODEL = "embed-english-v3.0"
+      DEFAULT_DIMENSIONS = 1024
+
+      def initialize(api_key:, dimensions: nil, model: nil)
+        super(api_key: api_key, dimensions: dimensions)
+        @model = model || DEFAULT_MODEL
+        @dimensions ||= DEFAULT_DIMENSIONS
+      end
+
+      def embed(text)
+        response = client.post("/v1/embed") do |req|
+          req.body = {
+            texts: [text.to_s],
+            model: @model,
+            input_type: "search_document",
+            embedding_types: ["float"]
+          }.to_json
+          req.headers["Content-Type"] = "application/json"
+          req.headers["Authorization"] = "Bearer #{@api_key}"
+        end
+        parse_embedding_response(response)
+      end
+
+      def embed_batch(texts)
+        return [] if texts.empty?
+        response = client.post("/v1/embed") do |req|
+          req.body = {
+            texts: texts.map(&:to_s),
+            model: @model,
+            input_type: "search_document",
+            embedding_types: ["float"]
+          }.to_json
+          req.headers["Content-Type"] = "application/json"
+          req.headers["Authorization"] = "Bearer #{@api_key}"
+        end
+        parse_batch_embedding_response(response)
+      end
+
+      private
+
+      def client
+        @client ||= Faraday.new(url: "https://api.cohere.ai") do |f|
+          f.request :json
+          f.response :json
+          f.adapter Faraday.default_adapter
+        end
+      end
+
+      def parse_embedding_response(response)
+        raise RailsAiKit::Error, "Cohere API error: #{response.body}" unless response.success?
+        data = response.body
+        data.dig("embeddings", "float")&.first || raise(RailsAiKit::Error, "No embedding in response")
+      end
+
+      def parse_batch_embedding_response(response)
+        raise RailsAiKit::Error, "Cohere API error: #{response.body}" unless response.success?
+        data = response.body
+        data.dig("embeddings", "float") || []
+      end
+    end
+  end
+end

data/lib/rails_ai_kit/embedding_providers/openai.rb

@@ -0,0 +1,69 @@
+# frozen_string_literal: true
+
+module RailsAiKit
+  module EmbeddingProviders
+    class Openai < Base
+      DEFAULT_MODEL = "text-embedding-3-small"
+      DEFAULT_DIMENSIONS = 1536
+
+      def initialize(api_key:, dimensions: nil, model: nil)
+        super(api_key: api_key, dimensions: dimensions)
+        @model = model || DEFAULT_MODEL
+      end
+
+      def embed(text)
+        response = client.post("/v1/embeddings") do |req|
+          req.body = {
+            input: text.to_s,
+            model: @model,
+            dimensions: (@dimensions || default_dimensions)
+          }.compact.to_json
+          req.headers["Content-Type"] = "application/json"
+          req.headers["Authorization"] = "Bearer #{@api_key}"
+        end
+        parse_embedding_response(response)
+      end
+
+      def embed_batch(texts)
+        return [] if texts.empty?
+        response = client.post("/v1/embeddings") do |req|
+          req.body = {
+            input: texts.map(&:to_s),
+            model: @model,
+            dimensions: (@dimensions || default_dimensions)
+          }.compact.to_json
+          req.headers["Content-Type"] = "application/json"
+          req.headers["Authorization"] = "Bearer #{@api_key}"
+        end
+        parse_batch_embedding_response(response)
+      end
+
+      private
+
+      def client
+        @client ||= Faraday.new(url: "https://api.openai.com") do |f|
+          f.request :json
+          f.response :json
+          f.adapter Faraday.default_adapter
+        end
+      end
+
+      def default_dimensions
+        @model.to_s.include?("large") ? 3072 : DEFAULT_DIMENSIONS
+      end
+
+      def parse_embedding_response(response)
+        raise RailsAiKit::Error, "OpenAI API error: #{response.body}" unless response.success?
+        data = response.body
+        data["data"]&.first&.dig("embedding") || raise(RailsAiKit::Error, "No embedding in response")
+      end
+
+      def parse_batch_embedding_response(response)
+        raise RailsAiKit::Error, "OpenAI API error: #{response.body}" unless response.success?
+        data = response.body
+        items = data["data"] || []
+        items.sort_by { |e| e["index"] }.map { |e| e["embedding"] }
+      end
+    end
+  end
+end

data/lib/rails_ai_kit/embedding_service.rb

@@ -0,0 +1,44 @@
+# frozen_string_literal: true
+
+module RailsAiKit
+  # Wraps the configured embedding provider and API keys.
+  # Use RailsAiKit.embedding.embed("hello") or .embed_batch([...])
+  class EmbeddingService
+    PROVIDERS = {
+      openai: EmbeddingProviders::Openai,
+      cohere: EmbeddingProviders::Cohere
+    }.freeze
+
+    def initialize(config: RailsAiKit.configuration)
+      @config = config
+    end
+
+    def embed(text)
+      provider.embed(text)
+    end
+
+    def embed_batch(texts)
+      provider.embed_batch(texts)
+    end
+
+    private
+
+    def provider
+      @provider ||= build_provider
+    end
+
+    def build_provider
+      name = @config.embedding_provider.to_sym
+      klass = PROVIDERS[name] || raise(ArgumentError, "Unknown embedding provider: #{name}")
+      api_key = @config.api_key(name) || raise(RailsAiKit::Error, "Missing API key for provider: #{name}. Set RailsAiKit.configuration.api_keys[:#{name}]")
+      klass.new(
+        api_key: api_key,
+        dimensions: @config.embedding_dimensions
+      )
+    end
+  end
+
+  def self.embedding
+    @embedding_service ||= EmbeddingService.new
+  end
+end

data/lib/rails_ai_kit/label_record.rb

@@ -0,0 +1,19 @@
+# frozen_string_literal: true
+
+module RailsAiKit
+  # Internal model for storing label embeddings per classifier.
+  # Table: rails_ai_kit_labels (via generator).
+  # We use a constant so the user's app can reference the same table.
+  class LabelRecord < ActiveRecord::Base
+    self.table_name = "rails_ai_kit_labels"
+
+    has_neighbors :embedding, dimensions: -> { RailsAiKit.configuration.embedding_dimensions }
+
+    validates :classifier_name, presence: true
+    validates :label_name, presence: true
+    validates :embedding, presence: true
+    validates :label_name, uniqueness: { scope: :classifier_name }
+
+    scope :for_classifier, ->(name) { where(classifier_name: name) }
+  end
+end

data/lib/rails_ai_kit/vector_classify.rb

@@ -0,0 +1,71 @@
+# frozen_string_literal: true
+
+module RailsAiKit
+  # ActiveRecord integration: vector_classify macro and similarity search.
+  # Expects the model to have columns: embedding (vector), label (string), confidence_score (float).
+  module VectorClassify
+    extend ActiveSupport::Concern
+
+    class_methods do
+      # Declare vector-based classification on this model.
+      #
+      # @param source_column [Symbol] Attribute holding the text to embed and classify (e.g. :content)
+      # @param labels [Array<String>] Label names (for documentation / validation; training is via Classifier.train)
+      # @param embedding_column [Symbol] Column storing the vector (default :embedding)
+      # @param label_column [Symbol] Column to store predicted label (default :label)
+      # @param confidence_column [Symbol] Column to store confidence score (default :confidence_score)
+      # @param classifier_name [String, nil] Classifier namespace (default: model name)
+      # @param auto_classify [Boolean] Run classification on save (default true)
+      def vector_classify(
+        source_column,
+        labels: [],
+        embedding_column: :embedding,
+        label_column: :label,
+        confidence_column: :confidence_score,
+        classifier_name: nil,
+        auto_classify: true
+      )
+        include RailsAiKit::VectorClassify::InstanceMethods
+
+        cattr_accessor :rails_ai_kit_source_column, :rails_ai_kit_embedding_column,
+                       :rails_ai_kit_label_column, :rails_ai_kit_confidence_column,
+                       :rails_ai_kit_classifier_name, :rails_ai_kit_labels
+        self.rails_ai_kit_source_column = source_column.to_sym
+        self.rails_ai_kit_embedding_column = embedding_column.to_sym
+        self.rails_ai_kit_label_column = label_column.to_sym
+        self.rails_ai_kit_confidence_column = confidence_column.to_sym
+        self.rails_ai_kit_classifier_name = classifier_name || name
+        self.rails_ai_kit_labels = labels
+
+        has_neighbors embedding_column, dimensions: RailsAiKit.configuration.embedding_dimensions
+
+        if auto_classify
+          before_save :rails_ai_kit_compute_embedding_and_classify
+        end
+
+        # Similarity search by text: embeds the query and returns nearest records.
+        define_singleton_method(:similar_to) do |query_text, limit: 5|
+          vector = RailsAiKit.embedding.embed(query_text)
+          nearest_neighbors(embedding_column, vector, distance: "cosine").limit(limit)
+        end
+      end
+    end
+
+    module InstanceMethods
+      private
+
+      def rails_ai_kit_compute_embedding_and_classify
+        source = send(self.class.rails_ai_kit_source_column)
+        return if source.blank?
+
+        embedding = RailsAiKit.embedding.embed(source)
+        send(:"#{self.class.rails_ai_kit_embedding_column}=", embedding)
+
+        classifier = RailsAiKit::Classifier.new(classifier_name: self.class.rails_ai_kit_classifier_name)
+        result = classifier.classify_by_embedding(embedding)
+        send(:"#{self.class.rails_ai_kit_label_column}=", result[:label])
+        send(:"#{self.class.rails_ai_kit_confidence_column}=", result[:confidence])
+      end
+    end
+  end
+end

data/lib/rails_ai_kit/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module RailsAiKit
+  VERSION = "0.1.0"
+end

data/lib/rails_ai_kit.rb

@@ -0,0 +1,25 @@
+# frozen_string_literal: true
+
+require_relative "rails_ai_kit/version"
+require_relative "rails_ai_kit/configuration"
+require_relative "rails_ai_kit/embedding_providers/base"
+require_relative "rails_ai_kit/embedding_providers/openai"
+require_relative "rails_ai_kit/embedding_providers/cohere"
+require_relative "rails_ai_kit/embedding_service"
+require_relative "rails_ai_kit/label_record"
+require_relative "rails_ai_kit/classifier"
+require_relative "rails_ai_kit/vector_classify"
+
+module RailsAiKit
+  class Error < StandardError; end
+
+  # Top-level classifier using default classifier name. For custom name use Classifier.new(classifier_name: "MyClassifier").
+  def self.classifier(classifier_name = nil)
+    Classifier.new(classifier_name: classifier_name)
+  end
+end
+
+# Optional Rails integration: extend ActiveRecord so vector_classify is available.
+if defined?(ActiveRecord::Base)
+  ActiveRecord::Base.include RailsAiKit::VectorClassify
+end

metadata

@@ -0,0 +1,117 @@
+--- !ruby/object:Gem::Specification
+name: rails_ai_kit
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Rails AI Kit Contributors
+bindir: bin
+cert_chain: []
+date: 1980-01-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+- !ruby/object:Gem::Dependency
+  name: activesupport
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '6.0'
+- !ruby/object:Gem::Dependency
+  name: faraday
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.0'
+- !ruby/object:Gem::Dependency
+  name: neighbor
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0.2'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0.2'
+description: 'Rails AI Kit provides ready-made tools to classify data using vector
+  similarity: label training, auto/batch classification, similarity search, and filtering—without
+  ML training or LLM calls.'
+email:
+- rohit.kushwaha@w3villa.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- README.md
+- lib/generators/rails_ai_kit/install/templates/create_rails_ai_kit_labels.rb
+- lib/generators/rails_ai_kit/install/templates/rails_ai_kit.rb
+- lib/generators/rails_ai_kit/install_generator.rb
+- lib/generators/rails_ai_kit/vector_columns/templates/add_vector_columns.rb
+- lib/generators/rails_ai_kit/vector_columns_generator.rb
+- lib/rails_ai_kit.rb
+- lib/rails_ai_kit/classifier.rb
+- lib/rails_ai_kit/configuration.rb
+- lib/rails_ai_kit/embedding_providers/base.rb
+- lib/rails_ai_kit/embedding_providers/cohere.rb
+- lib/rails_ai_kit/embedding_providers/openai.rb
+- lib/rails_ai_kit/embedding_service.rb
+- lib/rails_ai_kit/label_record.rb
+- lib/rails_ai_kit/vector_classify.rb
+- lib/rails_ai_kit/version.rb
+homepage: https://github.com/your-org/rails_ai_kit
+licenses: []
+metadata:
+  homepage_uri: https://github.com/your-org/rails_ai_kit
+  source_code_uri: https://github.com/your-org/rails_ai_kit
+  changelog_uri: https://github.com/your-org/rails_ai_kit/blob/main/CHANGELOG.md
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: 3.0.0
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 4.0.7
+specification_version: 4
+summary: Vector-based classification layer on top of pgvector for Rails
+test_files: []