neighbor 0.5.2 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: fb8418a71159a849442d146643f732c28d10a5237d94a5b5b7d7466a224a40b5
-  data.tar.gz: 8a7ededb3071fdef4a77bbbdf4596d74ed323327d4d29ea15538e344d25af2f7
+  metadata.gz: 5caf112992a87ac93ef7d05094b9d5beb71470fef6d5dc07da8f4ba74787c381
+  data.tar.gz: 85d47b002533704a0098e41fb2e752405ea8ae50ca3cdb7d5984e0cd93016ab0
 SHA512:
-  metadata.gz: 05b3b8ccd07570f531aef0132a80563aef046b88508e732b2f16fe02f6c0303bd6f6fdb00804a646845086ba7fd133358503a7c7b69b098c02f548434c859575
-  data.tar.gz: 9bcb61d4e1453e30daeab1fff0364c023278405d170b435cdfb262aac0aa0d91622e19471fbf3f3076e4c944d09318dc81cf334fdac7767fa6f72d95c8560907
+  metadata.gz: af55fbcbe28352582379e84b0e831aae677411cf4a45432cd9a1caff7e33857d38362c6bc484a479e9c4e99f634cb094efa4ea10e02e789b4cbd552c72578b6c
+  data.tar.gz: b896e3edda9fc58a7e8aeb8aa71cf409ae5d0f8b30a785642e8380b071a0bff3e6504bc5123711405504ab7f54fb4ac856a2a26f5088bcd8bc4c963b2f4cab10

data/CHANGELOG.md

@@ -1,3 +1,13 @@
+## 1.0.0 (2026-04-04)
+
+- Dropped support for Ruby < 3.3 and Active Record < 7.2
+
+## 0.6.0 (2025-06-12)
+
+- Added support for MariaDB 11.8
+- Dropped experimental support for MariaDB 11.7
+- Dropped support for Ruby < 3.2 and Active Record < 7.1
+
 ## 0.5.2 (2025-01-05)
 
 - Improved support for Postgres arrays

data/LICENSE.txt

@@ -1,6 +1,6 @@
 The MIT License (MIT)
 
-Copyright (c) 2021-2024 Andrew Kane
+Copyright (c) 2021-2026 Andrew Kane
 
 Permission is hereby granted, free of charge, to any person obtaining a copy
 of this software and associated documentation files (the "Software"), to deal

data/README.md

@@ -5,9 +5,11 @@ Nearest neighbor search for Rails
 Supports:
 
 - Postgres (cube and pgvector)
-- SQLite (sqlite-vec) - experimental
-- MariaDB 11.7 - experimental
+- MariaDB 11.8
 - MySQL 9 (searching requires HeatWave) - experimental
+- SQLite (sqlite-vec) - experimental
+
+Also available for [Redis](https://github.com/ankane/neighbor-redis) and [S3 Vectors](https://github.com/ankane/neighbor-s3)
 
 [![Build Status](https://github.com/ankane/neighbor/actions/workflows/build.yml/badge.svg)](https://github.com/ankane/neighbor/actions)
 
@@ -56,7 +58,7 @@ rails generate neighbor:sqlite
 Create a migration
 
 ```ruby
-class AddEmbeddingToItems < ActiveRecord::Migration[8.0]
+class AddEmbeddingToItems < ActiveRecord::Migration[8.1]
   def change
     # cube
     add_column :items, :embedding, :cube
@@ -107,9 +109,9 @@ See the additional docs for:
 
 - [cube](#cube)
 - [pgvector](#pgvector)
-- [sqlite-vec](#sqlite-vec)
 - [MariaDB](#mariadb)
 - [MySQL](#mysql)
+- [sqlite-vec](#sqlite-vec)
 
 Or check out some [examples](#examples)
 
@@ -174,7 +176,7 @@ The `sparsevec` type can have up to 16,000 non-zero elements, and sparse vectors
 Add an approximate index to speed up queries. Create a migration with:
 
 ```ruby
-class AddIndexToItemsEmbedding < ActiveRecord::Migration[8.0]
+class AddIndexToItemsEmbedding < ActiveRecord::Migration[8.1]
   def change
     add_index :items, :embedding, using: :hnsw, opclass: :vector_l2_ops
     # or
@@ -202,7 +204,7 @@ Item.connection.execute("SET ivfflat.probes = 3")
 Use the `halfvec` type to store half-precision vectors
 
 ```ruby
-class AddEmbeddingToItems < ActiveRecord::Migration[8.0]
+class AddEmbeddingToItems < ActiveRecord::Migration[8.1]
   def change
     add_column :items, :embedding, :halfvec, limit: 3 # dimensions
   end
@@ -214,9 +216,9 @@ end
 Index vectors at half precision for smaller indexes
 
 ```ruby
-class AddIndexToItemsEmbedding < ActiveRecord::Migration[8.0]
+class AddIndexToItemsEmbedding < ActiveRecord::Migration[8.1]
   def change
-    add_index :items, "(embedding::halfvec(3)) vector_l2_ops", using: :hnsw
+    add_index :items, "(embedding::halfvec(3)) halfvec_l2_ops", using: :hnsw
   end
 end
 ```
@@ -232,7 +234,7 @@ Item.nearest_neighbors(:embedding, [0.9, 1.3, 1.1], distance: "euclidean", preci
 Use the `bit` type to store binary vectors
 
 ```ruby
-class AddEmbeddingToItems < ActiveRecord::Migration[8.0]
+class AddEmbeddingToItems < ActiveRecord::Migration[8.1]
   def change
     add_column :items, :embedding, :bit, limit: 3 # dimensions
   end
@@ -250,7 +252,7 @@ Item.nearest_neighbors(:embedding, "101", distance: "hamming").first(5)
 Use expression indexing for binary quantization
 
 ```ruby
-class AddIndexToItemsEmbedding < ActiveRecord::Migration[8.0]
+class AddIndexToItemsEmbedding < ActiveRecord::Migration[8.1]
   def change
     add_index :items, "(binary_quantize(embedding)::bit(3)) bit_hamming_ops", using: :hnsw
   end
@@ -262,7 +264,7 @@ end
 Use the `sparsevec` type to store sparse vectors
 
 ```ruby
-class AddEmbeddingToItems < ActiveRecord::Migration[8.0]
+class AddEmbeddingToItems < ActiveRecord::Migration[8.1]
   def change
     add_column :items, :embedding, :sparsevec, limit: 3 # dimensions
   end
@@ -276,7 +278,7 @@ embedding = Neighbor::SparseVector.new({0 => 0.9, 1 => 1.3, 2 => 1.1}, 3)
 Item.nearest_neighbors(:embedding, embedding, distance: "euclidean").first(5)
 ```
 
-## sqlite-vec
+## MariaDB
 
 ### Distance
 
@@ -284,82 +286,64 @@ Supported values are:
 
 - `euclidean`
 - `cosine`
-- `taxicab`
 - `hamming`
 
-### Dimensions
+### Indexing
 
-For sqlite-vec, it’s a good idea to specify the number of dimensions to ensure all records have the same number.
+Vector columns must use `null: false` to add a vector index
 
 ```ruby
-class Item < ApplicationRecord
-  has_neighbors :embedding, dimensions: 3
+class CreateItems < ActiveRecord::Migration[8.1]
+  def change
+    create_table :items do |t|
+      t.vector :embedding, limit: 3, null: false
+      t.index :embedding, type: :vector
+    end
+  end
 end
 ```
 
-### Virtual Tables
+### Binary Vectors
 
-You can also use [virtual tables](https://alexgarcia.xyz/sqlite-vec/features/knn.html)
+Use the `bigint` type to store binary vectors
 
 ```ruby
-class AddEmbeddingToItems < ActiveRecord::Migration[8.0]
+class AddEmbeddingToItems < ActiveRecord::Migration[8.1]
   def change
-    # Rails 8+
-    create_virtual_table :items, :vec0, [
-      "id integer PRIMARY KEY AUTOINCREMENT NOT NULL",
-      "embedding float[3] distance_metric=L2"
-    ]
-
-    # Rails < 8
-    execute <<~SQL
-      CREATE VIRTUAL TABLE items USING vec0(
-        id integer PRIMARY KEY AUTOINCREMENT NOT NULL,
-        embedding float[3] distance_metric=L2
-      )
-    SQL
+    add_column :items, :embedding, :bigint
   end
 end
 ```
 
-Use `distance_metric=cosine` for cosine distance
-
-You can optionally ignore any shadow tables that are created
-
-```ruby
-ActiveRecord::SchemaDumper.ignore_tables += [
-  "items_chunks", "items_rowids", "items_vector_chunks00"
-]
-```
+Note: Binary vectors can have up to 64 dimensions
 
-Get the `k` nearest neighbors
+Get the nearest neighbors by Hamming distance
 
 ```ruby
-Item.where("embedding MATCH ?", [1, 2, 3].to_s).where(k: 5).order(:distance)
+Item.nearest_neighbors(:embedding, 5, distance: "hamming").first(5)
 ```
 
-Filter by primary key
+## MySQL
 
-```ruby
-Item.where(id: [2, 3]).where("embedding MATCH ?", [1, 2, 3].to_s).where(k: 5).order(:distance)
-```
+### Distance
 
-### Int8 Vectors
+Supported values are:
 
-Use the `type` option for int8 vectors
+- `euclidean`
+- `cosine`
+- `hamming`
 
-```ruby
-class Item < ApplicationRecord
-  has_neighbors :embedding, dimensions: 3, type: :int8
-end
-```
+Note: The `DISTANCE()` function is [only available on HeatWave](https://dev.mysql.com/doc/refman/9.0/en/vector-functions.html)
 
 ### Binary Vectors
 
-Use the `type` option for binary vectors
+Use the `binary` type to store binary vectors
 
 ```ruby
-class Item < ApplicationRecord
-  has_neighbors :embedding, dimensions: 8, type: :bit
+class AddEmbeddingToItems < ActiveRecord::Migration[8.1]
+  def change
+    add_column :items, :embedding, :binary
+  end
 end
 ```
 
@@ -369,7 +353,7 @@ Get the nearest neighbors by Hamming distance
 Item.nearest_neighbors(:embedding, "\x05", distance: "hamming").first(5)
 ```
 
-## MariaDB
+## sqlite-vec
 
 ### Distance
 
@@ -377,64 +361,82 @@ Supported values are:
 
 - `euclidean`
 - `cosine`
+- `taxicab`
 - `hamming`
 
-### Indexing
+### Dimensions
 
-Vector columns must use `null: false` to add a vector index
+For sqlite-vec, it’s a good idea to specify the number of dimensions to ensure all records have the same number.
 
 ```ruby
-class CreateItems < ActiveRecord::Migration[8.0]
-  def change
-    create_table :items do |t|
-      t.vector :embedding, limit: 3, null: false
-      t.index :embedding, type: :vector
-    end
-  end
+class Item < ApplicationRecord
+  has_neighbors :embedding, dimensions: 3
 end
 ```
 
-### Binary Vectors
+### Virtual Tables
 
-Use the `bigint` type to store binary vectors
+You can also use [virtual tables](https://alexgarcia.xyz/sqlite-vec/features/knn.html)
 
 ```ruby
-class AddEmbeddingToItems < ActiveRecord::Migration[8.0]
+class AddEmbeddingToItems < ActiveRecord::Migration[8.1]
   def change
-    add_column :items, :embedding, :bigint
+    # Rails 8+
+    create_virtual_table :items, :vec0, [
+      "id integer PRIMARY KEY AUTOINCREMENT NOT NULL",
+      "embedding float[3] distance_metric=L2"
+    ]
+
+    # Rails < 8
+    execute <<~SQL
+      CREATE VIRTUAL TABLE items USING vec0(
+        id integer PRIMARY KEY AUTOINCREMENT NOT NULL,
+        embedding float[3] distance_metric=L2
+      )
+    SQL
   end
 end
 ```
 
-Note: Binary vectors can have up to 64 dimensions
+Use `distance_metric=cosine` for cosine distance
 
-Get the nearest neighbors by Hamming distance
+You can optionally ignore any shadow tables that are created
 
 ```ruby
-Item.nearest_neighbors(:embedding, 5, distance: "hamming").first(5)
+ActiveRecord::SchemaDumper.ignore_tables += [
+  "items_chunks", "items_rowids", "items_vector_chunks00"
+]
 ```
 
-## MySQL
+Get the `k` nearest neighbors
 
-### Distance
+```ruby
+Item.where("embedding MATCH ?", [1, 2, 3].to_s).where(k: 5).order(:distance)
+```
 
-Supported values are:
+Filter by primary key
 
-- `euclidean`
-- `cosine`
-- `hamming`
+```ruby
+Item.where(id: [2, 3]).where("embedding MATCH ?", [1, 2, 3].to_s).where(k: 5).order(:distance)
+```
 
-Note: The `DISTANCE()` function is [only available on HeatWave](https://dev.mysql.com/doc/refman/9.0/en/vector-functions.html)
+### Int8 Vectors
+
+Use the `type` option for int8 vectors
+
+```ruby
+class Item < ApplicationRecord
+  has_neighbors :embedding, dimensions: 3, type: :int8
+end
+```
 
 ### Binary Vectors
 
-Use the `binary` type to store binary vectors
+Use the `type` option for binary vectors
 
 ```ruby
-class AddEmbeddingToItems < ActiveRecord::Migration[8.0]
-  def change
-    add_column :items, :embedding, :binary
-  end
+class Item < ApplicationRecord
+  has_neighbors :embedding, dimensions: 8, type: :bit
 end
 ```
 
@@ -473,7 +475,7 @@ end
 Create a method to call the [embeddings API](https://platform.openai.com/docs/guides/embeddings)
 
 ```ruby
-def fetch_embeddings(input)
+def embed(input)
   url = "https://api.openai.com/v1/embeddings"
   headers = {
     "Authorization" => "Bearer #{ENV.fetch("OPENAI_API_KEY")}",
@@ -497,7 +499,7 @@ input = [
   "The cat is purring",
   "The bear is growling"
 ]
-embeddings = fetch_embeddings(input)
+embeddings = embed(input)
 ```
 
 Store the embeddings
@@ -524,7 +526,7 @@ See the [complete code](examples/openai/example.rb)
 Generate a model
 
 ```sh
-rails generate model Document content:text embedding:bit{1024}
+rails generate model Document content:text embedding:bit{1536}
 rails db:migrate
 ```
 
@@ -539,15 +541,15 @@ end
 Create a method to call the [embed API](https://docs.cohere.com/reference/embed)
 
 ```ruby
-def fetch_embeddings(input, input_type)
-  url = "https://api.cohere.com/v1/embed"
+def embed(input, input_type)
+  url = "https://api.cohere.com/v2/embed"
   headers = {
     "Authorization" => "Bearer #{ENV.fetch("CO_API_KEY")}",
     "Content-Type" => "application/json"
   }
   data = {
     texts: input,
-    model: "embed-english-v3.0",
+    model: "embed-v4.0",
     input_type: input_type,
     embedding_types: ["ubinary"]
   }
@@ -565,7 +567,7 @@ input = [
   "The cat is purring",
   "The bear is growling"
 ]
-embeddings = fetch_embeddings(input, "search_document")
+embeddings = embed(input, "search_document")
 ```
 
 Store the embeddings
@@ -582,7 +584,7 @@ Embed the search query
 
 ```ruby
 query = "forest"
-query_embedding = fetch_embeddings([query], "search_query")[0]
+query_embedding = embed([query], "search_query")[0]
 ```
 
 And search the documents
@@ -876,7 +878,7 @@ bundle exec rake test:postgresql
 bundle exec rake test:sqlite
 
 # MariaDB
-docker run -e MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1 -e MARIADB_DATABASE=neighbor_test -p 3307:3306 mariadb:11.7-rc
+docker run -e MARIADB_ALLOW_EMPTY_ROOT_PASSWORD=1 -e MARIADB_DATABASE=neighbor_test -p 3307:3306 mariadb:11.8
 bundle exec rake test:mariadb
 
 # MySQL

data/lib/neighbor/model.rb

@@ -27,29 +27,13 @@ module Neighbor
           @neighbor_attributes[attribute_name] = {dimensions: dimensions, normalize: normalize, type: type&.to_sym}
         end
 
-        if ActiveRecord::VERSION::STRING.to_f >= 7.2
-          decorate_attributes(attribute_names) do |name, cast_type|
-            Neighbor::Attribute.new(cast_type: cast_type, model: self, type: type, attribute_name: name)
-          end
-        else
-          attribute_names.each do |attribute_name|
-            attribute attribute_name do |cast_type|
-              Neighbor::Attribute.new(cast_type: cast_type, model: self, type: type, attribute_name: attribute_name)
-            end
-          end
+        decorate_attributes(attribute_names) do |name, cast_type|
+          Neighbor::Attribute.new(cast_type: cast_type, model: self, type: type, attribute_name: name)
         end
 
         if normalize
-          if ActiveRecord::VERSION::STRING.to_f >= 7.1
-            attribute_names.each do |attribute_name|
-              normalizes attribute_name, with: ->(v) { Neighbor::Utils.normalize(v, column_info: columns_hash[attribute_name.to_s]) }
-            end
-          else
-            attribute_names.each do |attribute_name|
-              attribute attribute_name do |cast_type|
-                Neighbor::NormalizedAttribute.new(cast_type: cast_type, model: self, attribute_name: attribute_name)
-              end
-            end
+          attribute_names.each do |attribute_name|
+            normalizes attribute_name, with: ->(v) { Neighbor::Utils.normalize(v, column_info: columns_hash[attribute_name.to_s]) }
           end
         end
 

data/lib/neighbor/mysql.rb

@@ -15,9 +15,6 @@ module Neighbor
 
       # prevent unknown OID warning
       ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter.singleton_class.prepend(RegisterTypes)
-      if ActiveRecord::VERSION::STRING.to_f < 7.1
-        ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter.register_vector_type(ActiveRecord::ConnectionAdapters::AbstractMysqlAdapter::TYPE_MAP)
-      end
     end
 
     module RegisterTypes

data/lib/neighbor/sqlite.rb

@@ -18,7 +18,7 @@ module Neighbor
     module InstanceMethods
       def configure_connection
         super
-        db = ActiveRecord::VERSION::STRING.to_f >= 7.1 ? @raw_connection : @connection
+        db = @raw_connection
         db.enable_load_extension(1)
         SqliteVec.load(db)
         db.enable_load_extension(0)

data/lib/neighbor/version.rb

@@ -1,3 +1,3 @@
 module Neighbor
-  VERSION = "0.5.2"
+  VERSION = "1.0.0"
 end

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification
 name: neighbor
 version: !ruby/object:Gem::Version
-  version: 0.5.2
+  version: 1.0.0
 platform: ruby
 authors:
 - Andrew Kane
 bindir: bin
 cert_chain: []
-date: 2025-01-05 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activerecord
@@ -15,14 +15,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '7'
+        version: '7.2'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '7'
+        version: '7.2'
 email: andrew@ankane.org
 executables: []
 extensions: []
@@ -67,14 +67,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: '3.1'
+      version: '3.3'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.6.2
+rubygems_version: 4.0.3
 specification_version: 4
 summary: Nearest neighbor search for Rails
 test_files: []