red-candle 0.0.5 → 1.0.0.pre.1

data/Cargo.toml

@@ -1,2 +1,6 @@
 [workspace]
 members = ["ext/candle"]
+resolver = "2"
+
+[profile.test]
+opt-level = 3

data/README.md

@@ -3,7 +3,7 @@
 [![build](https://github.com/assaydepot/red-candle/actions/workflows/build.yml/badge.svg)](https://github.com/assaydepot/red-candle/actions/workflows/build.yml)
 [![Gem Version](https://badge.fury.io/rb/red-candle.svg)](https://badge.fury.io/rb/red-candle)
 
-🕯️ [candle](https://github.com/huggingface/candle) - Minimalist ML framework - for Ruby
+ [candle](https://github.com/huggingface/candle) - Minimalist ML framework - for Ruby
 
 ## Usage
 
@@ -20,19 +20,119 @@ x = x.reshape([3, 2])
 
 ```ruby
 require 'candle'
-model = Candle::Model.new
+
+# Default model (JinaBERT) on CPU
+model = Candle::EmbeddingModel.new
+embedding = model.embedding("Hi there!")
+
+# Specify device (CPU, Metal, or CUDA)
+device = Candle::Device.cpu     # or Candle::Device.metal, Candle::Device.cuda
+model = Candle::EmbeddingModel.new(
+  model_path: "jinaai/jina-embeddings-v2-base-en",
+  device: device
+)
 embedding = model.embedding("Hi there!")
+
+# Reranker also supports device selection
+reranker = Candle::Reranker.new(
+  model_path: "cross-encoder/ms-marco-MiniLM-L-12-v2",
+  device: device
+)
+results = reranker.rerank("query", ["doc1", "doc2", "doc3"])
+```
+
+## LLM Support
+
+Red-Candle now supports Large Language Models (LLMs) with GPU acceleration!
+
+> ### ⚠️ Huggingface login warning
+> 
+> Many models, including the one below, require you to agree to the terms. You'll need to:
+> 1. Login to [Huggingface](https://huggingface.co)
+> 2. Agree to the terms. For example: [here](https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.1)
+> 3. Authenticate your session. Simplest way is with `huggingface-cli login`. Detail here: [Huggingface CLI](https://huggingface.co/docs/huggingface_hub/en/guides/cli)
+>
+> More details here: [Huggingface Authentication](HUGGINGFACE.md)
+
+```ruby
+require 'candle'
+
+# Choose your device
+device = Candle::Device.cpu     # CPU (default)
+device = Candle::Device.metal   # Apple GPU (Metal)
+device = Candle::Device.cuda    # NVIDIA GPU (CUDA)
+
+# Load a model
+llm = Candle::LLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1", device: device)
+
+# Generate text
+response = llm.generate("What is Ruby?", config: Candle::GenerationConfig.balanced)
+
+# Stream generation
+llm.generate_stream("Tell me a story", config: Candle::GenerationConfig.balanced) do |token|
+  print token
+end
+
+# Chat interface
+messages = [
+  { role: "system", content: "You are a helpful assistant." },
+  { role: "user", content: "Explain Ruby in one sentence." }
+]
+response = llm.chat(messages)
+```
+
+### GPU Acceleration
+
+```ruby
+# CPU works for all models
+device = Candle::Device.cpu
+llm = Candle::LLM.from_pretrained("mistralai/Mistral-7B-Instruct-v0.1", device: device)
+
+# Metal
+device = Candle::Device.metal 
+
+# CUDA support (for NVIDIA GPUs COMING SOON)
+device = Candle::Device.cuda   # Linux/Windows with NVIDIA GPU
+```
+
+## ⚠️ Model Format Requirement: Safetensors Only
+
+Red-Candle **only supports embedding models that provide their weights in the [safetensors](https://github.com/huggingface/safetensors) format** (i.e., the model repo must contain a `model.safetensors` file). If the model repo does not provide the required file, loading will fail with a clear error. Most official BERT and DistilBERT models do **not** provide safetensors; many Sentence Transformers and JinaBERT models do.
+
+**If you encounter an error like:**
+
+```
+RuntimeError: model.safetensors not found after download. Only safetensors models are supported. Please ensure your model repo contains model.safetensors.
+```
+
+this means the selected model is not compatible. Please choose a model repo that provides the required file.
+
+## Supported Embedding Models
+
+Red-Candle supports the following embedding model types from Hugging Face:
+
+1. `Candle::EmbeddingModelType::JINA_BERT` - Jina BERT models (e.g., `jinaai/jina-embeddings-v2-base-en`) (**safetensors required**)
+2. `Candle::EmbeddingModelType::STANDARD_BERT` - Standard BERT models (e.g., `sentence-transformers/all-MiniLM-L6-v2`) (**safetensors required**)
+3. `Candle::EmbeddingModelType::DISTILBERT` - DistilBERT models (e.g., `distilbert-base-uncased-finetuned-sst-2-english`) (**safetensors required**)
+
+> **Note:** Most official BERT and DistilBERT models do _not_ provide safetensors. Please check the model repo before use.
+
+You can get a list of all supported model types and suggested models paths:
+
+```ruby
+Candle::EmbeddingModelType.all  # Returns all supported model types
+Candle::EmbeddingModelType.suggested_model_paths  # Returns hash of suggested models for each type
 ```
 
 ## A note on memory usage
-The `Candle::Model` defaults to the `jinaai/jina-embeddings-v2-base-en` model with the `sentence-transformers/all-MiniLM-L6-v2` tokenizer (both from [HuggingFace](https://huggingface.co)). With this configuration the model takes a little more than 3GB of memory running on my Mac. The memory stays with the instantiated `Candle::Model` class, if you instantiate more than one, you'll use more memory. Likewise, if you let it go out of scope and call the garbage collector, you'll free the memory. For example:
+The default model (`jinaai/jina-embeddings-v2-base-en` with the `sentence-transformers/all-MiniLM-L6-v2` tokenizer, both from [HuggingFace](https://huggingface.co)) takes a little more than 3GB of memory running on a Mac. The memory stays with the instantiated `Candle::EmbeddingModel` class, if you instantiate more than one, you'll use more memory. Likewise, if you let it go out of scope and call the garbage collector, you'll free the memory. For example:
 
 ```ruby
 > require 'candle'
 # Ruby memory = 25.9 MB
-> model = Candle::Model.new
+> model = Candle::EmbeddingModel.new
 # Ruby memory = 3.50 GB
-> model2 = Candle::Model.new
+> model2 = Candle::EmbeddingModel.new
 # Ruby memory = 7.04 GB
 > model2 = nil
 > GC.start
@@ -58,10 +158,128 @@ And the following ruby:
 
 ```ruby
 require 'candle'
-model = Candle::Model.new
+model = Candle::EmbeddingModel.new
 embedding = model.embedding("Hi there!")
 ```
 
+## Document Reranking
+
+Red-Candle includes support for cross-encoder reranking models, which can be used to reorder documents by relevance to a query. This is particularly useful for improving search results or implementing retrieval-augmented generation (RAG) systems.
+
+### Basic Usage
+
+```ruby
+require 'candle'
+
+# Initialize the reranker with a cross-encoder model
+reranker = Candle::Reranker.new(model_path: "cross-encoder/ms-marco-MiniLM-L-12-v2")
+
+# Define your query and candidate documents
+query = "How many people live in London?"
+documents = [
+  "London is known for its financial district",
+  "Around 9 Million people live in London", 
+  "The weather in London is often rainy",
+  "London is the capital of England"
+]
+
+# Rerank documents by relevance to the query (raw logits)
+ranked_results = reranker.rerank(query, documents, pooling_method: "pooler", apply_sigmoid: false)
+
+# Or apply sigmoid activation to get scores between 0 and 1
+sigmoid_results = reranker.rerank(query, documents, pooling_method: "pooler", apply_sigmoid: true)
+
+# The pooler method is the default and is recommended for cross-encoders, as is apply_sigmod, so the above is the same as:
+ranked_results = reranker.rerank(query, documents)
+
+# Results are returned as an array of hashes, sorted by relevance
+e.g.
+ranked_results.each do |result|
+  puts "Score: #{result[:score].round(4)} - Doc ##{result[:doc_id]}: #{result[:text]}"
+end
+# Output:
+# Score: 1.0 - Doc #1: Around 9 Million people live in London
+# Score: 0.0438 - Doc #3: London is the capital of England
+# Score: 0.0085 - Doc #0: London is known for its financial district
+# Score: 0.0005 - Doc #2: The weather in London is often rainy
+```
+
+### Arguments & Activation Functions
+
+By default, `apply_sigmoid` is `true` (scores between 0 and 1). Set it to `false` to get raw logits. You can also select the pooling method:
+
+- `pooling_method: "pooler"` (default)
+- `pooling_method: "cls"`
+- `pooling_method: "mean"`
+
+Example without sigmoid activation:
+
+```ruby
+# Get raw logits
+ranked_results = reranker.rerank(query, documents, apply_sigmoid: false)
+
+ranked_results.each do |result|
+  puts "Score: #{result[:score].round(4)} - Doc ##{result[:doc_id]}: #{result[:text]}"
+end
+# Output:
+# Score: 10.3918 - Doc #1: Around 9 Million people live in London
+# Score: -3.0829 - Doc #3: London is the capital of England
+# Score: -4.7619 - Doc #0: London is known for its financial district
+# Score: -7.5251 - Doc #2: The weather in London is often rainy
+```
+
+### Output Format
+
+The reranker returns an array of hashes, each with the following keys:
+- `:text` – The original document text
+- `:score` – The relevance score (raw logit or sigmoid-activated)
+- `:doc_id` – The original 0-based index of the document in the input array
+
+This format is compatible with the Informers gem, which returns results as hashes with `:doc_id` and `:score` keys. The `doc_id` allows you to map results back to your original data structure.
+
+### Pooling Methods
+
+The reranker supports different pooling strategies for aggregating BERT embeddings:
+
+```ruby
+# Use alternative pooling methods
+# "pooler" (default) - Uses the pooler layer with tanh activation (most accurate for cross-encoders)
+# "cls" - Uses raw [CLS] token embeddings without the pooler layer
+# "mean" - Mean pooling across all tokens (not recommended for cross-encoders)
+
+# With raw logits
+results = reranker.rerank_with_pooling(query, documents, "cls")
+
+# With sigmoid activation
+results = reranker.rerank_sigmoid_with_pooling(query, documents, "cls")
+```
+
+Note: The default "pooler" method is recommended as it matches how cross-encoder models are trained. Other pooling methods may produce different ranking results.
+
+### CUDA Support
+
+For faster inference on NVIDIA GPUs:
+
+```ruby
+# Initialize with CUDA if available (falls back to CPU if not)
+reranker = Candle::Reranker.new(model_path: "cross-encoder/ms-marco-MiniLM-L-12-v2", cuda: true)
+```
+
+### How It Works
+
+Cross-encoder reranking models differ from bi-encoder embedding models:
+
+- **Bi-encoders** (like the embedding models above) encode queries and documents separately into dense vectors
+- **Cross-encoders** process the query and document together, allowing for more nuanced relevance scoring
+
+The reranker uses a BERT-based architecture that:
+1. Concatenates the query and document with special tokens: `[CLS] query [SEP] document [SEP]`
+2. Processes them jointly through BERT layers
+3. Applies a pooler layer (dense + tanh) to the [CLS] token
+4. Uses a classifier layer to produce a single relevance score
+
+This joint processing allows cross-encoders to capture subtle semantic relationships between queries and documents, making them more accurate for reranking tasks, though at the cost of higher computational requirements.
+
 ## Development
 
 FORK IT!

data/ext/candle/Cargo.toml

@@ -2,18 +2,29 @@
 name = "candle"
 version = "0.1.0"
 edition = "2021"
+build = "build.rs"
 
 [lib]
 crate-type = ["cdylib"]
 
 [dependencies]
-candle-core = "0.4.1"
-candle-nn = "0.4.1"
-candle-transformers = "0.4.1"
-tokenizers = { version = "0.15.0", default-features = true, features = ["fancy-regex"], exclude = ["onig"] }
-hf-hub = "0.3.0"
-half = "2"
-magnus = "0.6"
+candle-core = { version = "0.9.1" }
+candle-nn = { version = "0.9.1" }
+candle-transformers = { version = "0.9.1" }
+tokenizers = { version = "0.21.1", default-features = true, features = ["fancy-regex"] }
+hf-hub = "0.4.3"
+half = "2.6.0"
+magnus = "0.7.1"
+safetensors = "0.3"
+serde_json = "1.0"
+serde = { version = "1.0", features = ["derive"] }
+tokio = { version = "1.45", features = ["rt", "macros"] }
+rand = "0.8"
 
-[profile.test]
-opt-level = 3
+[features]
+default = []
+metal = ["candle-core/metal", "candle-nn/metal", "candle-transformers/metal"]
+cuda = ["candle-core/cuda", "candle-nn/cuda", "candle-transformers/cuda"]
+cudnn = ["candle-core/cudnn", "cuda"]
+mkl = ["candle-core/mkl"]
+accelerate = ["candle-core/accelerate"]

data/ext/candle/extconf.rb

@@ -1,4 +1,80 @@
 require "mkmf"
 require "rb_sys/mkmf"
 
-create_rust_makefile("candle/candle")
+# Detect available hardware acceleration
+features = []
+
+# Force CPU-only build if requested
+if ENV['CANDLE_FORCE_CPU']
+  puts "CANDLE_FORCE_CPU is set, building CPU-only version"
+else
+  # Check for CUDA
+  cuda_available = ENV['CUDA_ROOT'] || ENV['CUDA_PATH'] || ENV['CANDLE_CUDA_PATH'] ||
+                   File.exist?('/usr/local/cuda') || File.exist?('/opt/cuda') ||
+                   (RbConfig::CONFIG['host_os'] =~ /mswin|mingw|cygwin/ && 
+                    (File.exist?('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA') ||
+                     File.exist?('C:\CUDA')))
+  
+  cuda_enabled = ENV['CANDLE_ENABLE_CUDA']
+  
+  if cuda_available && cuda_enabled
+    puts "CUDA detected and enabled via CANDLE_ENABLE_CUDA"
+    features << 'cuda'
+    
+    # Check if CUDNN should be enabled
+    if ENV['CANDLE_CUDNN'] || ENV['CUDNN_ROOT']
+      puts "CUDNN support enabled"
+      features << 'cudnn'
+    end
+  elsif cuda_available && !cuda_enabled
+    puts "=" * 80
+    puts "CUDA detected but not enabled."
+    puts "To enable CUDA support (coming soon), set CANDLE_ENABLE_CUDA=1"
+    puts "=" * 80
+  end
+
+  # Check for Metal (macOS only)
+  if RbConfig::CONFIG['host_os'] =~ /darwin/
+    puts "macOS detected, enabling Metal support"
+    features << 'metal'
+    
+    # Also enable Accelerate framework on macOS
+    puts "Enabling Accelerate framework support"
+    features << 'accelerate'
+  end
+
+  # Check for Intel MKL
+  mkl_available = ENV['MKLROOT'] || ENV['MKL_ROOT'] ||
+                  File.exist?('/opt/intel/mkl') || 
+                  File.exist?('/opt/intel/oneapi/mkl/latest')
+  
+  if mkl_available && !features.include?('accelerate')  # Don't use both MKL and Accelerate
+    puts "Intel MKL detected, enabling MKL support"
+    features << 'mkl'
+  end
+end
+
+# Allow manual override of features
+if ENV['CANDLE_FEATURES']
+  manual_features = ENV['CANDLE_FEATURES'].split(',').map(&:strip)
+  puts "Manual features override: #{manual_features.join(', ')}"
+  features = manual_features
+end
+
+# Display selected features
+unless features.empty?
+  puts "Building with features: #{features.join(', ')}"
+else
+  puts "Building CPU-only version (no acceleration features detected)"
+end
+
+# Create the Rust makefile with proper feature configuration
+create_rust_makefile("candle/candle") do |r|
+  # Pass the features to rb_sys
+  r.features = features unless features.empty?
+  
+  # Pass through any additional cargo flags
+  if ENV['CANDLE_CARGO_FLAGS']
+    r.extra_cargo_args = ENV['CANDLE_CARGO_FLAGS'].split(' ')
+  end
+end

data/ext/candle/src/lib.rs

@@ -1,103 +1,142 @@
 use magnus::{function, method, prelude::*, Ruby};
 
-use crate::model::{candle_utils, RbModel, RbDType, RbDevice, RbQTensor, RbResult, RbTensor};
+use crate::ruby::candle_utils;
+use crate::ruby::{DType, Device, QTensor, Result as RbResult, Tensor};
 
-pub mod model;
+pub mod llm;
+pub mod reranker;
+pub mod ruby;
+
+// Configuration detection from build.rs
+#[cfg(all(has_metal, not(force_cpu)))]
+const DEFAULT_DEVICE: &str = "metal";
+
+#[cfg(all(has_cuda, not(has_metal), not(force_cpu)))]
+const DEFAULT_DEVICE: &str = "cuda";
+
+#[cfg(any(force_cpu, not(any(has_metal, has_cuda))))]
+const DEFAULT_DEVICE: &str = "cpu";
+
+// Export build configuration for runtime checks
+pub fn get_build_info() -> magnus::RHash {
+    let ruby = magnus::Ruby::get().unwrap();
+    let hash = ruby.hash_new();
+    
+    let _ = hash.aset("default_device", DEFAULT_DEVICE);
+    let _ = hash.aset("cuda_available", cfg!(feature = "cuda"));
+    let _ = hash.aset("metal_available", cfg!(feature = "metal"));
+    let _ = hash.aset("mkl_available", cfg!(feature = "mkl"));
+    let _ = hash.aset("accelerate_available", cfg!(feature = "accelerate"));
+    let _ = hash.aset("cudnn_available", cfg!(feature = "cudnn"));
+    
+    hash
+}
 
 #[magnus::init]
 fn init(ruby: &Ruby) -> RbResult<()> {
     let rb_candle = ruby.define_module("Candle")?;
+    
+    // Export build info
+    rb_candle.define_singleton_method("build_info", function!(get_build_info, 0))?;
+    
+    ruby::init_embedding_model(rb_candle)?;
+    ruby::init_llm(rb_candle)?;
+    reranker::init(rb_candle)?;
     candle_utils(rb_candle)?;
     let rb_tensor = rb_candle.define_class("Tensor", Ruby::class_object(ruby))?;
-    rb_tensor.define_singleton_method("new", function!(RbTensor::new, 2))?;
-    // rb_tensor.define_singleton_method("cat", function!(RbTensor::cat, 2))?;
-    // rb_tensor.define_singleton_method("stack", function!(RbTensor::stack, 2))?;
-    rb_tensor.define_singleton_method("rand", function!(RbTensor::rand, 1))?;
-    rb_tensor.define_singleton_method("randn", function!(RbTensor::randn, 1))?;
-    rb_tensor.define_singleton_method("ones", function!(RbTensor::ones, 1))?;
-    rb_tensor.define_singleton_method("zeros", function!(RbTensor::zeros, 1))?;
-    rb_tensor.define_method("values", method!(RbTensor::values, 0))?;
-    rb_tensor.define_method("shape", method!(RbTensor::shape, 0))?;
-    rb_tensor.define_method("stride", method!(RbTensor::stride, 0))?;
-    rb_tensor.define_method("dtype", method!(RbTensor::dtype, 0))?;
-    rb_tensor.define_method("device", method!(RbTensor::device, 0))?;
-    rb_tensor.define_method("rank", method!(RbTensor::rank, 0))?;
-    rb_tensor.define_method("elem_count", method!(RbTensor::elem_count, 0))?;
-    rb_tensor.define_method("sin", method!(RbTensor::sin, 0))?;
-    rb_tensor.define_method("cos", method!(RbTensor::cos, 0))?;
-    rb_tensor.define_method("log", method!(RbTensor::log, 0))?;
-    rb_tensor.define_method("sqr", method!(RbTensor::sqr, 0))?;
-    rb_tensor.define_method("sqrt", method!(RbTensor::sqrt, 0))?;
-    rb_tensor.define_method("recip", method!(RbTensor::recip, 0))?;
-    rb_tensor.define_method("exp", method!(RbTensor::exp, 0))?;
-    rb_tensor.define_method("powf", method!(RbTensor::powf, 1))?;
-    rb_tensor.define_method("index_select", method!(RbTensor::index_select, 2))?;
-    rb_tensor.define_method("matmul", method!(RbTensor::matmul, 1))?;
-    rb_tensor.define_method("broadcast_add", method!(RbTensor::broadcast_add, 1))?;
-    rb_tensor.define_method("broadcast_sub", method!(RbTensor::broadcast_sub, 1))?;
-    rb_tensor.define_method("broadcast_mul", method!(RbTensor::broadcast_mul, 1))?;
-    rb_tensor.define_method("broadcast_div", method!(RbTensor::broadcast_div, 1))?;
-    rb_tensor.define_method("where_cond", method!(RbTensor::where_cond, 2))?;
-    rb_tensor.define_method("+", method!(RbTensor::__add__, 1))?;
-    rb_tensor.define_method("*", method!(RbTensor::__mul__, 1))?;
-    rb_tensor.define_method("-", method!(RbTensor::__sub__, 1))?;
-    rb_tensor.define_method("reshape", method!(RbTensor::reshape, 1))?;
-    rb_tensor.define_method("broadcast_as", method!(RbTensor::broadcast_as, 1))?;
-    rb_tensor.define_method("broadcast_left", method!(RbTensor::broadcast_left, 1))?;
-    rb_tensor.define_method("squeeze", method!(RbTensor::squeeze, 1))?;
-    rb_tensor.define_method("unsqueeze", method!(RbTensor::unsqueeze, 1))?;
-    rb_tensor.define_method("get", method!(RbTensor::get, 1))?;
-    rb_tensor.define_method("[]", method!(RbTensor::get, 1))?;
-    rb_tensor.define_method("transpose", method!(RbTensor::transpose, 2))?;
-    rb_tensor.define_method("narrow", method!(RbTensor::narrow, 3))?;
-    rb_tensor.define_method("argmax_keepdim", method!(RbTensor::argmax_keepdim, 1))?;
-    rb_tensor.define_method("argmin_keepdim", method!(RbTensor::argmin_keepdim, 1))?;
-    rb_tensor.define_method("max_keepdim", method!(RbTensor::max_keepdim, 1))?;
-    rb_tensor.define_method("min_keepdim", method!(RbTensor::min_keepdim, 1))?;
-    // rb_tensor.define_method("eq", method!(RbTensor::eq, 1))?;
-    // rb_tensor.define_method("ne", method!(RbTensor::ne, 1))?;
-    // rb_tensor.define_method("lt", method!(RbTensor::lt, 1))?;
-    // rb_tensor.define_method("gt", method!(RbTensor::gt, 1))?;
-    // rb_tensor.define_method("ge", method!(RbTensor::ge, 1))?;
-    // rb_tensor.define_method("le", method!(RbTensor::le, 1))?;
-    rb_tensor.define_method("sum_all", method!(RbTensor::sum_all, 0))?;
-    rb_tensor.define_method("mean_all", method!(RbTensor::mean_all, 0))?;
-    rb_tensor.define_method("flatten_from", method!(RbTensor::flatten_from, 1))?;
-    rb_tensor.define_method("flatten_to", method!(RbTensor::flatten_to, 1))?;
-    rb_tensor.define_method("flatten_all", method!(RbTensor::flatten_all, 0))?;
-    rb_tensor.define_method("t", method!(RbTensor::t, 0))?;
-    rb_tensor.define_method("contiguous", method!(RbTensor::contiguous, 0))?;
-    rb_tensor.define_method("is_contiguous", method!(RbTensor::is_contiguous, 0))?;
+    rb_tensor.define_singleton_method("new", function!(Tensor::new, 3))?;
+    // rb_tensor.define_singleton_method("cat", function!(Tensor::cat, 2))?;
+    // rb_tensor.define_singleton_method("stack", function!(Tensor::stack, 2))?;
+    rb_tensor.define_singleton_method("rand", function!(Tensor::rand, 2))?;
+    rb_tensor.define_singleton_method("randn", function!(Tensor::randn, 2))?;
+    rb_tensor.define_singleton_method("ones", function!(Tensor::ones, 2))?;
+    rb_tensor.define_singleton_method("zeros", function!(Tensor::zeros, 2))?;
+    rb_tensor.define_method("values", method!(Tensor::values, 0))?;
+    rb_tensor.define_method("values_f32", method!(Tensor::values_f32, 0))?;
+    rb_tensor.define_method("item", method!(Tensor::item, 0))?;
+    rb_tensor.define_method("shape", method!(Tensor::shape, 0))?;
+    rb_tensor.define_method("stride", method!(Tensor::stride, 0))?;
+    rb_tensor.define_method("dtype", method!(Tensor::dtype, 0))?;
+    rb_tensor.define_method("device", method!(Tensor::device, 0))?;
+    rb_tensor.define_method("rank", method!(Tensor::rank, 0))?;
+    rb_tensor.define_method("elem_count", method!(Tensor::elem_count, 0))?;
+    rb_tensor.define_method("sin", method!(Tensor::sin, 0))?;
+    rb_tensor.define_method("cos", method!(Tensor::cos, 0))?;
+    rb_tensor.define_method("log", method!(Tensor::log, 0))?;
+    rb_tensor.define_method("sqr", method!(Tensor::sqr, 0))?;
+    rb_tensor.define_method("mean", method!(Tensor::mean, 1))?;
+    rb_tensor.define_method("sum", method!(Tensor::sum, 1))?;
+    rb_tensor.define_method("sqrt", method!(Tensor::sqrt, 0))?;
+    rb_tensor.define_method("/", method!(Tensor::__truediv__, 1))?; // Accepts Tensor, Float, or Integer
+    rb_tensor.define_method("recip", method!(Tensor::recip, 0))?;
+    rb_tensor.define_method("exp", method!(Tensor::exp, 0))?;
+    rb_tensor.define_method("powf", method!(Tensor::powf, 1))?;
+    rb_tensor.define_method("index_select", method!(Tensor::index_select, 2))?;
+    rb_tensor.define_method("matmul", method!(Tensor::matmul, 1))?;
+    rb_tensor.define_method("broadcast_add", method!(Tensor::broadcast_add, 1))?;
+    rb_tensor.define_method("broadcast_sub", method!(Tensor::broadcast_sub, 1))?;
+    rb_tensor.define_method("broadcast_mul", method!(Tensor::broadcast_mul, 1))?;
+    rb_tensor.define_method("broadcast_div", method!(Tensor::broadcast_div, 1))?;
+    rb_tensor.define_method("where_cond", method!(Tensor::where_cond, 2))?;
+    rb_tensor.define_method("+", method!(Tensor::__add__, 1))?;
+    rb_tensor.define_method("*", method!(Tensor::__mul__, 1))?;
+    rb_tensor.define_method("-", method!(Tensor::__sub__, 1))?;
+    rb_tensor.define_method("reshape", method!(Tensor::reshape, 1))?;
+    rb_tensor.define_method("broadcast_as", method!(Tensor::broadcast_as, 1))?;
+    rb_tensor.define_method("broadcast_left", method!(Tensor::broadcast_left, 1))?;
+    rb_tensor.define_method("squeeze", method!(Tensor::squeeze, 1))?;
+    rb_tensor.define_method("unsqueeze", method!(Tensor::unsqueeze, 1))?;
+    rb_tensor.define_method("get", method!(Tensor::get, 1))?;
+    rb_tensor.define_method("[]", method!(Tensor::get, 1))?;
+    rb_tensor.define_method("transpose", method!(Tensor::transpose, 2))?;
+    rb_tensor.define_method("narrow", method!(Tensor::narrow, 3))?;
+    rb_tensor.define_method("argmax_keepdim", method!(Tensor::argmax_keepdim, 1))?;
+    rb_tensor.define_method("argmin_keepdim", method!(Tensor::argmin_keepdim, 1))?;
+    rb_tensor.define_method("max_keepdim", method!(Tensor::max_keepdim, 1))?;
+    rb_tensor.define_method("min_keepdim", method!(Tensor::min_keepdim, 1))?;
+    // rb_tensor.define_method("eq", method!(Tensor::eq, 1))?;
+    // rb_tensor.define_method("ne", method!(Tensor::ne, 1))?;
+    // rb_tensor.define_method("lt", method!(Tensor::lt, 1))?;
+    // rb_tensor.define_method("gt", method!(Tensor::gt, 1))?;
+    // rb_tensor.define_method("ge", method!(Tensor::ge, 1))?;
+    // rb_tensor.define_method("le", method!(Tensor::le, 1))?;
+    rb_tensor.define_method("sum_all", method!(Tensor::sum_all, 0))?;
+    rb_tensor.define_method("mean_all", method!(Tensor::mean_all, 0))?;
+    rb_tensor.define_method("flatten_from", method!(Tensor::flatten_from, 1))?;
+    rb_tensor.define_method("flatten_to", method!(Tensor::flatten_to, 1))?;
+    rb_tensor.define_method("flatten_all", method!(Tensor::flatten_all, 0))?;
+    rb_tensor.define_method("t", method!(Tensor::t, 0))?;
+    rb_tensor.define_method("contiguous", method!(Tensor::contiguous, 0))?;
+    rb_tensor.define_method("is_contiguous", method!(Tensor::is_contiguous, 0))?;
     rb_tensor.define_method(
         "is_fortran_contiguous",
-        method!(RbTensor::is_fortran_contiguous, 0),
+        method!(Tensor::is_fortran_contiguous, 0),
     )?;
-    rb_tensor.define_method("detach", method!(RbTensor::detach, 0))?;
-    rb_tensor.define_method("copy", method!(RbTensor::copy, 0))?;
-    rb_tensor.define_method("to_dtype", method!(RbTensor::to_dtype, 1))?;
-    rb_tensor.define_method("to_device", method!(RbTensor::to_device, 1))?;
-    rb_tensor.define_method("to_s", method!(RbTensor::__str__, 0))?;
-    rb_tensor.define_method("inspect", method!(RbTensor::__repr__, 0))?;
+    rb_tensor.define_method("detach", method!(Tensor::detach, 0))?;
+    rb_tensor.define_method("copy", method!(Tensor::copy, 0))?;
+    rb_tensor.define_method("to_dtype", method!(Tensor::to_dtype, 1))?;
+    rb_tensor.define_method("to_device", method!(Tensor::to_device, 1))?;
+    rb_tensor.define_method("to_s", method!(Tensor::__str__, 0))?;
+    rb_tensor.define_method("inspect", method!(Tensor::__repr__, 0))?;
 
     let rb_dtype = rb_candle.define_class("DType", Ruby::class_object(ruby))?;
-    rb_dtype.define_method("to_s", method!(RbDType::__str__, 0))?;
-    rb_dtype.define_method("inspect", method!(RbDType::__repr__, 0))?;
+    rb_dtype.define_method("to_s", method!(DType::__str__, 0))?;
+    rb_dtype.define_method("inspect", method!(DType::__repr__, 0))?;
 
     let rb_device = rb_candle.define_class("Device", Ruby::class_object(ruby))?;
-    rb_device.define_method("to_s", method!(RbDevice::__str__, 0))?;
-    rb_device.define_method("inspect", method!(RbDevice::__repr__, 0))?;
+    rb_device.define_singleton_method("cpu", function!(Device::cpu, 0))?;
+    rb_device.define_singleton_method("cuda", function!(Device::cuda, 0))?;
+    rb_device.define_singleton_method("metal", function!(Device::metal, 0))?;
+    rb_device.define_singleton_method("available_devices", function!(ruby::device::available_devices, 0))?;
+    rb_device.define_singleton_method("default", function!(ruby::device::default_device, 0))?;
+    rb_device.define_method("to_s", method!(Device::__str__, 0))?;
+    rb_device.define_method("inspect", method!(Device::__repr__, 0))?;
 
     let rb_qtensor = rb_candle.define_class("QTensor", Ruby::class_object(ruby))?;
-    rb_qtensor.define_method("ggml_dtype", method!(RbQTensor::ggml_dtype, 0))?;
-    rb_qtensor.define_method("rank", method!(RbQTensor::rank, 0))?;
-    rb_qtensor.define_method("shape", method!(RbQTensor::shape, 0))?;
-    rb_qtensor.define_method("dequantize", method!(RbQTensor::dequantize, 0))?;
-
-    let rb_model = rb_candle.define_class("Model", Ruby::class_object(ruby))?;
-    rb_model.define_singleton_method("new", function!(RbModel::new, 0))?;
-    rb_model.define_method("embedding", method!(RbModel::embedding, 1))?;
-    rb_model.define_method("to_s", method!(RbModel::__str__, 0))?;
-    rb_model.define_method("inspect", method!(RbModel::__repr__, 0))?;
+    rb_qtensor.define_method("ggml_dtype", method!(QTensor::ggml_dtype, 0))?;
+    rb_qtensor.define_method("rank", method!(QTensor::rank, 0))?;
+    rb_qtensor.define_method("shape", method!(QTensor::shape, 0))?;
+    rb_qtensor.define_method("dequantize", method!(QTensor::dequantize, 0))?;
 
     Ok(())
 }