red-candle 1.0.0 → 1.0.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 829a937851c782dfd58b8fb724dc7b08d524d26400047e9f5fc7a5bd0de9cb4b
-  data.tar.gz: e8a9420fc310e977968aa396a47e5e5269d107eb8cb7246ca9e2f980a0a28f4d
+  metadata.gz: 52c635f005d25a305f99781763a4a3cc03f85fc5b74f0e576e51973ef8306fac
+  data.tar.gz: 1a0ac260a3803f1920ba2d9f71ec361013ae1eb99cf2caed62c0e9aecc583e96
 SHA512:
-  metadata.gz: 020e23df61d5679612a7892bdbfc7dbcf2d28055df9fc6b9199a8e09933e74d03a0a0fb97d6ddc998f8f8e70e856d63d7eb758638e072cd492734b21681267b7
-  data.tar.gz: 5e10f888c2bd74dfdf01c97ca18f6666440edb000d524784ad0ac0676208d9485b8d708fa72d8fe73672f018c038b3526109153540ce5bbc53a81e4e30deccd1
+  metadata.gz: d301e6ed0fe8ac144c0735288c687f5dd74e7967dbe5d357e550ca5d467f6a33017b2bd9e7f46081711b6bf13555caa3e044183cd74cfaa89151e15c8cdb04a4
+  data.tar.gz: d296c35002b6d0ed919176375e5cc5d93c70fae0c0ae9a02d5cf86b8a4a49a67898c7fbc96e16350bba6792b18792126c976de156fb854b9b4f3260fa052cd79

data/README.md

@@ -1,9 +1,66 @@
-# red-candle
+# `red-candle` Native LLMs for Ruby 🚀
 
 [![build](https://github.com/assaydepot/red-candle/actions/workflows/build.yml/badge.svg)](https://github.com/assaydepot/red-candle/actions/workflows/build.yml)
 [![Gem Version](https://badge.fury.io/rb/red-candle.svg)](https://badge.fury.io/rb/red-candle)
 
- [candle](https://github.com/huggingface/candle) - Minimalist ML framework - for Ruby
+Run state-of-the-art **language models directly from Ruby**. No Python, no APIs, no external services - just Ruby with blazing-fast Rust under the hood. Hardware accelerated with **Metal (Mac)** and **CUDA (NVIDIA).**
+
+## Install & Chat in 30 Seconds
+
+[![red-candle quickstart](https://img.youtube.com/vi/hbyFCyh8esk/0.jpg)](https://www.youtube.com/watch?v=hbyFCyh8esk)
+
+```bash
+# Install the gem
+gem install red-candle
+```
+
+```ruby
+require 'candle'
+
+# Download a model (one-time, ~650MB) - Mistral, Llama3, Gemma all work!
+llm = Candle::LLM.from_pretrained("TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF", 
+                                  gguf_file: "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf")
+
+# Chat with it - no API calls, running locally in your Ruby process!
+messages = [
+  { role: "user", content: "Explain Ruby in one sentence" }
+]
+
+puts llm.chat(messages)
+# => "Ruby is a dynamic, object-oriented programming language known for its 
+#     simplicity, elegance, and productivity, often used for web development 
+#     with frameworks like Rails."
+```
+
+## What Just Happened?
+
+You just ran a 1.1-billion parameter AI model inside Ruby. The model lives in your process memory, runs on your hardware (CPU/GPU), and responds instantly without network latency.
+
+## Stream Responses Like a Pro
+
+```ruby
+# Watch the AI think in real-time
+llm.chat_stream(messages) do |token|
+  print token
+end
+```
+
+## Why This Matters
+
+- **Privacy**: Your data never leaves your machine
+- **Speed**: No network overhead, direct memory access
+- **Control**: Fine-tune generation parameters, access raw tokens
+- **Integration**: It's just Ruby objects - use it anywhere Ruby runs
+
+## Supports
+
+- **Tokenizers**: Access the tokenizer directly
+- **EmbeddingModel**: Generate embeddings for text
+- **Reranker**: Rerank documents based on relevance
+- **NER**: Named Entity Recognition directly from Ruby
+- **LLM**: Chat with Large Language Models (e.g., Llama, Mistral, Gemma)
+
+----
 
 ## Usage
 
@@ -671,7 +728,7 @@ All NER methods return entities in a consistent format:
 
 ## Common Runtime Errors
 
-### 1. Weight is negative, too large or not a valid number
+### Weight is negative, too large or not a valid number
 
 **Error:**
 ```
@@ -688,13 +745,12 @@ All NER methods return entities in a consistent format:
 - Q3_K_M (3-bit) - Minimum recommended quantization
 
 ```ruby
-# Instead of Q2_K:
 llm = Candle::LLM.from_pretrained("TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUF", 
                                   device: device, 
                                   gguf_file: "tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf")
 ```
 
-### 2. Cannot find tensor model.embed_tokens.weight
+### Cannot find tensor model.embed_tokens.weight
 
 **Error:**
 ```
@@ -713,7 +769,7 @@ Failed to load quantized model: cannot find tensor model.embed_tokens.weight (Ru
    ```
 3. If the error persists, the GGUF file may use an unsupported architecture or format
 
-### 3. No GGUF file found in repository
+### No GGUF file found in repository
 
 **Error:**
 ```
@@ -730,7 +786,7 @@ llm = Candle::LLM.from_pretrained("TheBloke/Llama-2-7B-Chat-GGUF",
                                   gguf_file: "llama-2-7b-chat.Q4_K_M.gguf")
 ```
 
-### 4. Failed to download tokenizer
+### Failed to download tokenizer
 
 **Error:**
 ```
@@ -741,7 +797,7 @@ Failed to load quantized model: Failed to download tokenizer: request error: HTT
 
 **Solution:** The code now includes fallback tokenizer loading. If you still encounter this error, ensure you're using the latest version of red-candle.
 
-### 5. Missing metadata in GGUF file
+### Missing metadata in GGUF file
 
 **Error:**
 ```
@@ -770,17 +826,24 @@ Failed to load GGUF model: cannot find llama.attention.head_count in metadata (R
 FORK IT!
 
 ```
-git clone https://github.com/your_name/red-candle
+git clone https://github.com/assaydepot/red-candle
 cd red-candle
 bundle
 bundle exec rake compile
 ```
 
-Implemented with [Magnus](https://github.com/matsadler/magnus), with reference to [Polars Ruby](https://github.com/ankane/polars-ruby)
-
 Pull requests are welcome.
 
-### See Also
+## Release
+
+1. Update version number in `lib/candle/version.rb` and commit.
+2. `bundle exec rake build`
+3. `git tag VERSION_NUMBER`
+4. `git push --follow-tags`
+5. `gem push pkg/red-candle-1.0.0.gem`
+
+## See Also
 
-- [Numo::NArray](https://github.com/ruby-numo/numo-narray)
-- [Cumo](https://github.com/sonots/cumo)
+- [Candle](https://github.com/huggingface/candle)
+- [Magnus](https://github.com/matsadler/magnus)
+- [Outlines-core](https://github.com/dottxt-ai/outlines-core)

data/lib/candle/version.rb

@@ -1,5 +1,5 @@
 # :nocov:
 module Candle
-  VERSION = "1.0.0"
+  VERSION = "1.0.1"
 end
 # :nocov:

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: red-candle
 version: !ruby/object:Gem::Version
-  version: 1.0.0
+  version: 1.0.1
 platform: ruby
 authors:
 - Christopher Petersen
@@ -216,9 +216,9 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: 3.3.26
 requirements:
-- Rust >= 1.61
+- Rust >= 1.65
 rubygems_version: 3.5.3
 signing_key:
 specification_version: 4
-summary: huggingface/candle for ruby
+summary: huggingface/candle for Ruby
 test_files: []