red-candle 1.0.0 → 1.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 829a937851c782dfd58b8fb724dc7b08d524d26400047e9f5fc7a5bd0de9cb4b
-  data.tar.gz: e8a9420fc310e977968aa396a47e5e5269d107eb8cb7246ca9e2f980a0a28f4d
+  metadata.gz: 7405c9911d6088106dd7a19e96312f12b86e9a80087c3d7745cd3911e263890a
+  data.tar.gz: a88b75152708e72e019aba9acfeabd899f1ae1d1c567562ded6d2c6aa8eae8d0
 SHA512:
-  metadata.gz: 020e23df61d5679612a7892bdbfc7dbcf2d28055df9fc6b9199a8e09933e74d03a0a0fb97d6ddc998f8f8e70e856d63d7eb758638e072cd492734b21681267b7
-  data.tar.gz: 5e10f888c2bd74dfdf01c97ca18f6666440edb000d524784ad0ac0676208d9485b8d708fa72d8fe73672f018c038b3526109153540ce5bbc53a81e4e30deccd1
+  metadata.gz: ce1cc52dc1223968f3398ab0972283a6309a80d306c14193f23336cd36ed55c8fa5eaaaf05d756f76c88e442abe19b0d82d2742a49199930ef6effcffd6d4482
+  data.tar.gz: 8c30f3c0c096f8186b219a9a5d0fe92928621126f545eaabae26ddedc843515b7da8c45890a1f24a7d519c0c68fcd55b0553ea73f977644258ff30c5e5ccd2f1

data/README.md

@@ -1,9 +1,85 @@
-# red-candle
+# `red-candle` Native LLMs for Ruby 🚀
 
 [![build](https://github.com/assaydepot/red-candle/actions/workflows/build.yml/badge.svg)](https://github.com/assaydepot/red-candle/actions/workflows/build.yml)
 [![Gem Version](https://badge.fury.io/rb/red-candle.svg)](https://badge.fury.io/rb/red-candle)
 
- [candle](https://github.com/huggingface/candle) - Minimalist ML framework - for Ruby
+Run state-of-the-art **language models directly from Ruby**. No Python, no APIs, no external services - just Ruby with blazing-fast Rust under the hood. Hardware accelerated with **Metal (Mac)** and **CUDA (NVIDIA).**
+
+## Install & Chat in 30 Seconds
+
+[![red-candle quickstart](https://img.youtube.com/vi/hbyFCyh8esk/0.jpg)](https://www.youtube.com/watch?v=hbyFCyh8esk)
+
+```bash
+# Install the gem
+gem install red-candle
+```
+
+```ruby
+require 'candle'
+
+# Download a model (one-time, ~650MB) - Mistral, Llama3, Gemma all work!
+llm = Candle::LLM.from_pretrained("TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF", 
+                                  gguf_file: "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf")
+
+# Chat with it - no API calls, running locally in your Ruby process!
+messages = [
+  { role: "user", content: "Explain Ruby in one sentence" }
+]
+
+puts llm.chat(messages)
+# => "Ruby is a dynamic, object-oriented programming language known for its 
+#     simplicity, elegance, and productivity, often used for web development 
+#     with frameworks like Rails."
+```
+
+## What Just Happened?
+
+You just ran a 1.1-billion parameter AI model inside Ruby. The model lives in your process memory, runs on your hardware (CPU/GPU), and responds instantly without network latency.
+
+## Stream Responses Like a Pro
+
+```ruby
+# Watch the AI think in real-time
+llm.chat_stream(messages) do |token|
+  print token
+end
+```
+
+## Why This Matters
+
+- **Privacy**: Your data never leaves your machine
+- **Speed**: No network overhead, direct memory access
+- **Control**: Fine-tune generation parameters, access raw tokens
+- **Integration**: It's just Ruby objects - use it anywhere Ruby runs
+
+## Supports
+
+- **Tokenizers**: Access the tokenizer directly
+- **EmbeddingModel**: Generate embeddings for text
+- **Reranker**: Rerank documents based on relevance
+- **NER**: Named Entity Recognition directly from Ruby
+- **LLM**: Chat with Large Language Models (e.g., Llama, Mistral, Gemma)
+
+## Model Storage
+
+Models are automatically downloaded and cached when you first use them. They are stored in:
+- **Location**: `~/.cache/huggingface/hub/`
+- **Size**: Models range from ~100MB (embeddings) to several GB (LLMs)
+- **Reuse**: Models are downloaded once and reused across sessions
+
+To check your cache or manage storage:
+```bash
+# View cache contents
+ls -la ~/.cache/huggingface/hub/
+
+# Check total cache size
+du -sh ~/.cache/huggingface/
+
+# Clear cache if needed (removes all downloaded models)
+rm -rf ~/.cache/huggingface/hub/
+```
+
+----
 
 ## Usage
 
@@ -137,7 +213,7 @@ llm = Candle::LLM.from_pretrained("TinyLlama/TinyLlama-1.1B-Chat-v1.0", device:
 # Metal
 device = Candle::Device.metal 
 
-# CUDA support (for NVIDIA GPUs COMING SOON)
+# CUDA support (for NVIDIA GPUs)
 device = Candle::Device.cuda   # Linux/Windows with NVIDIA GPU
 ```
 
@@ -671,7 +747,7 @@ All NER methods return entities in a consistent format:
 
 ## Common Runtime Errors
 
-### 1. Weight is negative, too large or not a valid number
+### Weight is negative, too large or not a valid number
 
 **Error:**
 ```
@@ -688,13 +764,12 @@ All NER methods return entities in a consistent format:
 - Q3_K_M (3-bit) - Minimum recommended quantization
 
 ```ruby
-# Instead of Q2_K:
 llm = Candle::LLM.from_pretrained("TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF", 
                                   device: device, 
                                   gguf_file: "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf")
 ```
 
-### 2. Cannot find tensor model.embed_tokens.weight
+### Cannot find tensor model.embed_tokens.weight
 
 **Error:**
 ```
@@ -713,7 +788,7 @@ Failed to load quantized model: cannot find tensor model.embed_tokens.weight (Ru
    ```
 3. If the error persists, the GGUF file may use an unsupported architecture or format
 
-### 3. No GGUF file found in repository
+### No GGUF file found in repository
 
 **Error:**
 ```
@@ -730,7 +805,7 @@ llm = Candle::LLM.from_pretrained("TheBloke/Llama-2-7B-Chat-GGUF",
                                   gguf_file: "llama-2-7b-chat.Q4_K_M.gguf")
 ```
 
-### 4. Failed to download tokenizer
+### Failed to download tokenizer
 
 **Error:**
 ```
@@ -741,7 +816,7 @@ Failed to load quantized model: Failed to download tokenizer: request error: HTT
 
 **Solution:** The code now includes fallback tokenizer loading. If you still encounter this error, ensure you're using the latest version of red-candle.
 
-### 5. Missing metadata in GGUF file
+### Missing metadata in GGUF file
 
 **Error:**
 ```
@@ -770,17 +845,24 @@ Failed to load GGUF model: cannot find llama.attention.head_count in metadata (R
 FORK IT!
 
 ```
-git clone https://github.com/your_name/red-candle
+git clone https://github.com/assaydepot/red-candle
 cd red-candle
 bundle
 bundle exec rake compile
 ```
 
-Implemented with [Magnus](https://github.com/matsadler/magnus), with reference to [Polars Ruby](https://github.com/ankane/polars-ruby)
-
 Pull requests are welcome.
 
-### See Also
+## Release
+
+1. Update version number in `lib/candle/version.rb` and commit.
+2. `bundle exec rake build`
+3. `git tag VERSION_NUMBER`
+4. `git push --follow-tags`
+5. `gem push pkg/red-candle-VERSION_NUMBER.gem`
+
+## See Also
 
-- [Numo::NArray](https://github.com/ruby-numo/numo-narray)
-- [Cumo](https://github.com/sonots/cumo)
+- [Candle](https://github.com/huggingface/candle)
+- [Magnus](https://github.com/matsadler/magnus)
+- [Outlines-core](https://github.com/dottxt-ai/outlines-core)

data/ext/candle/build.rs

@@ -16,6 +16,7 @@ fn main() {
     println!("cargo:rerun-if-env-changed=CUDA_PATH");
     println!("cargo:rerun-if-env-changed=CANDLE_FEATURES");
     println!("cargo:rerun-if-env-changed=CANDLE_ENABLE_CUDA");
+    println!("cargo:rerun-if-env-changed=CANDLE_DISABLE_CUDA");
 
     // Check if we should force CPU only
     if env::var("CANDLE_FORCE_CPU").is_ok() {
@@ -26,13 +27,13 @@ fn main() {
 
     // Detect CUDA availability
     let cuda_available = detect_cuda();
-    let cuda_enabled = env::var("CANDLE_ENABLE_CUDA").is_ok();
+    let cuda_disabled = env::var("CANDLE_DISABLE_CUDA").is_ok();
     
-    if cuda_available && cuda_enabled {
+    if cuda_available && !cuda_disabled {
         println!("cargo:rustc-cfg=has_cuda");
-        println!("cargo:warning=CUDA detected and enabled via CANDLE_ENABLE_CUDA");
-    } else if cuda_available && !cuda_enabled {
-        println!("cargo:warning=CUDA detected but not enabled. To enable CUDA support (coming soon), set CANDLE_ENABLE_CUDA=1");
+        println!("cargo:warning=CUDA detected, CUDA acceleration will be available");
+    } else if cuda_available && cuda_disabled {
+        println!("cargo:warning=CUDA detected but disabled via CANDLE_DISABLE_CUDA");
     }
 
     // Detect Metal availability (macOS only)

data/ext/candle/extconf.rb

@@ -15,10 +15,10 @@ else
                     (File.exist?('C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA') ||
                      File.exist?('C:\CUDA')))
   
-  cuda_enabled = ENV['CANDLE_ENABLE_CUDA']
+  cuda_disabled = ENV['CANDLE_DISABLE_CUDA']
   
-  if cuda_available && cuda_enabled
-    puts "CUDA detected and enabled via CANDLE_ENABLE_CUDA"
+  if cuda_available && !cuda_disabled
+    puts "CUDA detected, enabling CUDA support"
     features << 'cuda'
     
     # Check if CUDNN should be enabled
@@ -26,10 +26,9 @@ else
       puts "CUDNN support enabled"
       features << 'cudnn'
     end
-  elsif cuda_available && !cuda_enabled
+  elsif cuda_available && cuda_disabled
     puts "=" * 80
-    puts "CUDA detected but not enabled."
-    puts "To enable CUDA support (coming soon), set CANDLE_ENABLE_CUDA=1"
+    puts "CUDA detected but disabled via CANDLE_DISABLE_CUDA"
     puts "=" * 80
   end
 

data/lib/candle/build_info.rb

@@ -15,8 +15,8 @@ module Candle
         if cuda_potentially_available
           warn "=" * 80
           warn "Red Candle: CUDA detected on system but not enabled in build."
-          warn "To enable CUDA support (experimental), reinstall with:"
-          warn "  CANDLE_ENABLE_CUDA=1 gem install red-candle"
+          warn "This may be due to CANDLE_DISABLE_CUDA being set during installation."
+          warn "To enable CUDA support, reinstall without CANDLE_DISABLE_CUDA set."
           warn "=" * 80
         end
         # :nocov:

data/lib/candle/version.rb

@@ -1,5 +1,5 @@
 # :nocov:
 module Candle
-  VERSION = "1.0.0"
+  VERSION = "1.0.2"
 end
 # :nocov:

data/lib/red-candle.rb

@@ -0,0 +1 @@
+require 'candle'

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: red-candle
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.0.2
 platform: ruby
 authors:
 - Christopher Petersen
@@ -9,7 +9,7 @@ authors:
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2025-07-19 00:00:00.000000000 Z
+date: 2025-07-22 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rb_sys
@@ -197,6 +197,7 @@ files:
 - lib/candle/tensor.rb
 - lib/candle/tokenizer.rb
 - lib/candle/version.rb
+- lib/red-candle.rb
 homepage: https://github.com/assaydepot/red-candle
 licenses:
 - MIT
@@ -216,9 +217,9 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: 3.3.26
 requirements:
-- Rust >= 1.61
+- Rust >= 1.65
 rubygems_version: 3.5.3
 signing_key:
 specification_version: 4
-summary: huggingface/candle for ruby
+summary: huggingface/candle for Ruby
 test_files: []