fastembed 1.0.0 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 1bc6e2a53d0c7a8c4679f7af6a23af74b903b0e227c401307e3d936d74719291
-  data.tar.gz: 5664c43f8d7f0632719324b42805abe3806274c18113232f82bbddce918647e5
+  metadata.gz: 954cdd87ba985d20a8a5bf0e5676178ebc891032f275fb4cb8cf547c0378a476
+  data.tar.gz: 4c11fbd4894906ab46a99746a2efa16f8a56e494a4fcd64012d642b992914918
 SHA512:
-  metadata.gz: f7f821a8a8ee49fdbbde65eaf394d8b406b1aecfc53fa572ecbbf3ccaf9fd120771fa5e5d444d339aa9ec5b729c63f283e564d05ea1ce1db1427580ced41cc0d
-  data.tar.gz: a99fda1200c29abf2ea7bed10ab5e7eee30fc5238e5d13b5608f7c41014e98e1e880c67ea1da1e56eef10864697ebca11ed7a7bf6463fb437f4dc4fe80267bef
+  metadata.gz: 948c3d705493fe26f3fd4fcb8b14965c0223b7ddd119307502b9af8f8503552b7693846a2a004ea97aca3d3cc5cd39f77017acd82ee2b48be56b879e6eaa1c7b
+  data.tar.gz: 1ceb5a0de19bc743b215332706d20facc49cda03f7e82211ff4306de1fb443d3f508f2b4b7a59dbdc43f3bbb720fe7accb80f342f45e7cc7cd0bf1c65fbb105e

data/.rubocop.yml

@@ -8,6 +8,7 @@ AllCops:
   Exclude:
     - 'bin/*'
     - 'vendor/**/*'
+    - 'benchmark/**/*'
 
 Style/Documentation:
   Enabled: false

data/.yardopts

@@ -0,0 +1,6 @@
+--markup markdown
+--title "Fastembed Ruby API Documentation"
+--output-dir doc
+--readme README.md
+--files CHANGELOG.md,BENCHMARKS.md
+lib/**/*.rb

data/BENCHMARKS.md

@@ -1,6 +1,6 @@
 # Benchmarks
 
-Performance benchmarks on Apple M1 Max, Ruby 3.3.10.
+Performance benchmarks on Apple M1 Max, Ruby 3.3, Python 3.13 (January 2026).
 
 ## Single Document Latency
 
@@ -69,6 +69,98 @@ We tested CoreML execution provider to see if GPU/Neural Engine acceleration hel
 
 **Recommendation:** Stick with the default CPU provider.
 
+## Ruby vs Python FastEmbed
+
+Comprehensive comparison of fastembed-rb against Python FastEmbed (v0.7.4) on Apple M1 Max.
+
+### Text Embeddings (100 documents)
+
+| Model | Ruby (docs/sec) | Python (docs/sec) | Ratio |
+|-------|-----------------|-------------------|-------|
+| BAAI/bge-small-en-v1.5 | 566 | 629 | 0.90x |
+| BAAI/bge-base-en-v1.5 | 176 | 169 | **1.04x** |
+| all-MiniLM-L6-v2 | 922 | 1309 | 0.70x |
+
+Ruby is within 10-30% of Python for text embeddings. Both use the same ONNX Runtime backend.
+
+### Rerankers (100 query-document pairs)
+
+| Model | Ruby (pairs/sec) | Python (pairs/sec) | Ratio |
+|-------|------------------|-------------------|-------|
+| ms-marco-MiniLM-L-6-v2 | 986 | 982 | **1.00x** |
+| ms-marco-MiniLM-L-12-v2 | 398 | 512 | 0.78x |
+| BAAI/bge-reranker-base | 132 | 124 | **1.06x** |
+
+Ruby matches or beats Python on rerankers.
+
+### Sparse Embeddings - SPLADE (100 documents)
+
+| Model | Ruby (docs/sec) | Python (docs/sec) | Ratio |
+|-------|-----------------|-------------------|-------|
+| Splade_PP_en_v1 | 23 | 108 | 0.21x |
+
+Ruby's SPLADE implementation is slower due to post-processing overhead. Python uses optimized numpy operations for the log1p transformation.
+
+### Late Interaction - ColBERT (100 documents)
+
+| Model | Ruby (docs/sec) | Python (docs/sec) | Ratio |
+|-------|-----------------|-------------------|-------|
+| colbert-ir/colbertv2.0 | 191 | 184 | **1.04x** |
+
+Ruby slightly outperforms Python for ColBERT embeddings.
+
+### Image Embeddings (100 images)
+
+| Model | Ruby (imgs/sec) | Python (imgs/sec) | Ratio |
+|-------|-----------------|-------------------|-------|
+| clip-ViT-B-32-vision | 9 | 42 | 0.22x |
+
+Ruby's image embedding is slower due to MiniMagick subprocess overhead for image preprocessing. Python uses Pillow which is more efficient for batch processing.
+
+### Summary
+
+| Category | Ruby vs Python |
+|----------|---------------|
+| Text Embeddings | ~90% of Python speed |
+| Rerankers | **Equal or faster** |
+| ColBERT | **Equal or faster** |
+| Sparse (SPLADE) | ~21% of Python speed |
+| Image | ~22% of Python speed |
+
+**Recommendation:** Ruby is excellent for text embeddings, reranking, and ColBERT. For heavy sparse or image embedding workloads, consider Python.
+
+### Why the Differences?
+
+Both implementations use the same ONNX Runtime for model inference. The differences come from:
+
+1. **Text/Reranker/ColBERT** - Hot path is tokenization (Rust) + inference (C++). Minimal language overhead. Ruby matches Python.
+
+2. **Sparse (SPLADE)** - Requires post-processing with log1p transformation. Python's numpy vectorization is faster than Ruby loops.
+
+3. **Image** - Requires image preprocessing (resize, normalize). Python's Pillow is faster than Ruby's MiniMagick (subprocess-based).
+
+### Memory Usage
+
+| State | Ruby |
+|-------|------|
+| Initial | 33 MB |
+| Model loaded | 277 MB |
+| +1000 embeddings | 359 MB |
+| After GC | 355 MB |
+
+Memory is stable across multiple embedding rounds - no leaks detected.
+
+### Embedding Quality
+
+Both implementations produce identical embeddings (same ONNX models), verified by cosine similarity tests:
+
+```
+'dog' vs 'puppy' = 0.855 (high - PASS)
+'dog' vs 'cat' = 0.688 (medium - PASS)
+'machine learning' vs 'artificial intelligence' = 0.718 (high - PASS)
+'machine learning' vs 'cooking recipes' = 0.426 (low - PASS)
+```
+
 ## Running Your Own Benchmarks
 
 ```ruby
@@ -81,3 +173,34 @@ texts = Array.new(1000) { "Sample text for benchmarking" }
 result = Benchmark.measure { embedding.embed(texts).to_a }
 puts "#{1000 / result.real} docs/sec"
 ```
+
+## Reranker Performance
+
+TextCrossEncoder (cross-encoder) performance:
+
+| Model | 100 pairs | Throughput |
+|-------|-----------|------------|
+| ms-marco-MiniLM-L-6-v2 | 102ms | **986 pairs/sec** |
+| ms-marco-MiniLM-L-12-v2 | 252ms | **398 pairs/sec** |
+| bge-reranker-base | 758ms | **132 pairs/sec** |
+
+Cross-encoders are slower than embedding models because they process query-document pairs together rather than encoding them independently.
+
+### Profiling Scripts
+
+The `benchmark/` directory contains:
+
+- `profile.rb` - Comprehensive embedding performance profiling
+- `reranker_benchmark.rb` - Reranker/cross-encoder performance
+- `memory_profile.rb` - Memory usage analysis
+- `compare_python.py` - Python FastEmbed comparison
+- `compare_all.rb` - Unified Ruby vs Python comparison
+
+Run with:
+```bash
+ruby benchmark/profile.rb
+ruby benchmark/reranker_benchmark.rb
+ruby benchmark/memory_profile.rb
+ruby benchmark/compare_all.rb
+python3 benchmark/compare_python.py
+```

data/CHANGELOG.md

@@ -7,6 +7,20 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- `TextCrossEncoder` class for reranking query-document pairs
+- Support for cross-encoder/reranker models:
+  - cross-encoder/ms-marco-MiniLM-L-6-v2 (default)
+  - cross-encoder/ms-marco-MiniLM-L-12-v2
+  - BAAI/bge-reranker-base
+  - BAAI/bge-reranker-large
+  - jinaai/jina-reranker-v1-turbo-en
+- `rerank` method for scoring query-document pairs
+- `rerank_with_scores` method for sorted results with top_k support
+- CLI tool (`fastembed`) with `embed` and `list-models` commands
+- Comprehensive benchmark suite comparing Ruby vs Python performance
+
 ## [1.0.0] - 2025-01-08
 
 ### Added