lancelot 0.1.1 → 0.3.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9524a69ce1b7a5065839b41b5619585bc4e1643c79ec90cfc69fc82a3a8eed7e
-  data.tar.gz: e2c8fc98de615913297b7b06e909b8d85179cb2018e3a08c9b70f4bb2756d55f
+  metadata.gz: 59a61f845bead9178dc6b7ca831b0cd1f3577969c189d723b085a706527bb9a9
+  data.tar.gz: f6941d534cc770393803c152f1718dad15639086b4ae78e84da93f8f8ab6f43f
 SHA512:
-  metadata.gz: 043fcea4635008454b61999055be80b21c0ddb7c67462e0f90d242f7b81395068ff7ed1cabeabf1f63f7c1823593f2405fca083c3aa9330c7ac7a7b1f09ae4c7
-  data.tar.gz: 4efedcef3af02685c97531162aa6adea5beb3084759033a0b32bdf535663aa12c4246cb9d67649bd836f04bf317e59101326e17c8a8474ace059908d0b36074d
+  metadata.gz: 04f63038fe699b2441c22618daac20ef710df82a5d95c6c8258317a2600b23ea2844ab5c2f1bc01245fb4ff90d531f3ec09f7ddcb28876e758b983a0bcb1fdc2
+  data.tar.gz: 20b8c06914dc993b6868b7264cfcfba0eeb0a0a0c70b0473ac6e3b9e163c8ed3855c2eeae1ad02ab3f19125f80c844a0aebd2c26a91aafc005af0dfebd8d37d6

data/CLAUDE.md

@@ -0,0 +1,176 @@
+# Lancelot - Project Instructions
+
+You are working on lancelot, a Ruby gem that provides native bindings to the Lance columnar data format through Rust and Magnus.
+
+## Project Context
+
+Lancelot is part of a Ruby-native NLP/ML ecosystem:
+- **lance**: Rust crate providing columnar storage with vector and text search
+- **lancelot**: THIS PROJECT - Ruby bindings to Lance via Magnus
+- **red-candle**: Ruby gem providing LLMs, embeddings, and rerankers
+- **candle-agent**: Future agent capabilities for the ecosystem
+
+## Core Architecture
+
+### Ruby-Rust Bridge
+- Uses Magnus 0.7 for Ruby-Rust interop
+- RefCell pattern for interior mutability
+- Embedded Tokio runtime for async Lance operations
+- Clean separation between Ruby API and Rust implementation
+
+### Key Components
+- `lib/lancelot/dataset.rb`: Ruby-idiomatic API layer
+- `ext/lancelot/src/dataset.rs`: Core Lance operations
+- `ext/lancelot/src/schema.rs`: Schema building and type mapping
+- `ext/lancelot/src/conversion.rs`: Ruby-Rust data conversion
+
+## Design Principles
+
+1. **Ruby-First API**: Make it feel native to Ruby developers
+   - Use symbols for type names (`:string`, `:vector`)
+   - Support operator overloading (`<<` for append)
+   - Include Enumerable for iteration
+   - Return Ruby hashes/arrays, not foreign objects
+
+2. **Schema-First Design**: Lance requires schemas at creation
+   - Clear schema definition API
+   - Helpful error messages for type mismatches
+   - Support all Arrow types Lance supports
+
+3. **Performance Without Complexity**: Hide async/columnar details
+   - Embed Tokio runtime, don't expose it
+   - Convert RecordBatches to Ruby transparently
+   - Efficient batch operations when possible
+
+4. **Error Handling**: Rust errors become Ruby exceptions
+   - Use Magnus's error conversion
+   - Provide context in error messages
+   - Never panic in Rust code
+
+## Current Features
+
+### Implemented
+- Dataset creation with schema
+- CRUD operations (add, update, delete, get)
+- Vector search with ANN indices
+- Full-text search with inverted indices
+- Multi-column text search
+- SQL-like filtering
+- Enumerable support
+
+### Not Yet Implemented
+- Schema evolution
+- Hybrid search (vector + text fusion)
+- Streaming operations
+- Transaction support
+- Data versioning/time travel
+
+## Development Guidelines
+
+### Adding New Features
+
+1. **Start with the Ruby API**: Design how it should feel in Ruby first
+2. **Implement in Rust**: Keep the Rust layer focused on Lance operations
+3. **Type Conversion**: Use conversion.rs patterns for new types
+4. **Testing**: Add Ruby tests in spec/ and Rust tests in src/
+
+### Magnus Best Practices
+
+1. **Memory Management**:
+   ```rust
+   #[magnus::wrap(class = "Lancelot::Dataset", free_immediately, size)]
+   ```
+
+2. **Method Definition**:
+   ```rust
+   class.define_method("method_name", method!(LancelotDataset::method_name, arity))?;
+   ```
+
+3. **Error Handling**:
+   ```rust
+   pub fn operation(&self) -> Result<Value, Error> {
+       self.with_dataset(|dataset| {
+           // Lance operation
+       }).map_err(|e| Error::new(exception::runtime_error(), e.to_string()))
+   }
+   ```
+
+4. **Async Operations**:
+   ```rust
+   self.with_runtime(|runtime| {
+       runtime.block_on(async {
+           // Async Lance operation
+       })
+   })
+   ```
+
+### Type Mappings
+
+Ruby → Arrow/Lance:
+- `:string` → Utf8
+- `:integer` → Int64
+- `:float` → Float64
+- `:vector` → FixedSizeList with dimension
+- `:boolean` → Bool
+- `:date` → Date32
+- `:datetime` → Timestamp
+
+### Testing
+
+- Ruby specs use RSpec
+- Test both successful operations and error cases
+- Use temporary directories for test datasets
+- Clean up resources in after blocks
+
+## Common Tasks
+
+### Adding a New Search Method
+1. Define Ruby API in dataset.rb
+2. Add Rust implementation in dataset.rs
+3. Handle type conversion if needed
+4. Add index support if applicable
+5. Write comprehensive tests
+
+### Exposing Lance Features
+1. Check Lance API documentation
+2. Design Ruby-idiomatic wrapper
+3. Consider if it needs async handling
+4. Implement with proper error handling
+5. Document with YARD comments
+
+## Integration Points
+
+### With Red-Candle
+- Lancelot stores embeddings from red-candle
+- Vector dimensions must match model output
+- Consider batch operations for efficiency
+
+### Future: Hybrid Search
+- Will combine vector and text search results
+- RRF (Reciprocal Rank Fusion) planned
+- Consider implementing in Ruby first
+
+## Performance Considerations
+
+1. **Batch Operations**: Always prefer batch over individual ops
+2. **Index Building**: Build indices after bulk loading
+3. **Memory Usage**: Lance uses memory mapping efficiently
+4. **Ruby GC**: Use free_immediately for deterministic cleanup
+
+## Debugging Tips
+
+1. **Rust Panics**: Use `RUST_BACKTRACE=1` for stack traces
+2. **Lance Logs**: Set `RUST_LOG=lance=debug`
+3. **Ruby-Rust Bridge**: Check type conversions first
+4. **Async Issues**: Ensure operations run on the runtime
+
+## Release Process
+
+1. Update version.rb
+2. Run full test suite
+3. Build gem locally and test
+4. Update CHANGELOG.md
+5. Tag release and push
+6. `rake release` to publish
+
+Remember: The goal is to make Lance's power accessible to Ruby developers without them needing to understand columnar formats, Rust, or async programming.

data/README.md

@@ -2,23 +2,59 @@
 
 Ruby bindings for [Lance](https://github.com/lancedb/lance), a modern columnar data format for ML. Lancelot provides a Ruby-native interface to Lance, enabling efficient storage and search of multimodal data including text, vectors, and more.
 
+## Quickstart
+
+```ruby
+require 'lancelot'
+require 'red-candle'
+
+strings = [
+  "apple",
+  "orange",
+  "google"
+]
+
+model = Candle::EmbeddingModel.from_pretrained
+
+dataset = Lancelot::Dataset.open_or_create("words", schema: {
+  text: :string,
+  embedding: { type: "vector", dimension: 768 }
+})
+
+records = strings.collect do |string|
+  embedding = model.embedding(string).first.to_a
+  { text: string, embedding: embedding }
+end
+
+dataset.add_documents(records)
+
+dataset.create_vector_index("embedding")
+dataset.create_text_index("text")
+
+
+query = "fruit"
+query_embedding = model.embedding(query).first.to_a
+dataset.vector_search(query_embedding, column: "embedding", limit: 5).each { |r| puts r[:text] }; nil
+
+dataset.text_search("apple", column: "text", limit: 5).each { |r| puts r[:text] }; nil
+
+query = "tech company"
+query_embedding = model.embedding(query).first.to_a
+dataset.vector_search(query_embedding, column: "embedding", limit: 5).each { |r| puts r[:text] }; nil
+```
+
 ## Features
 
 ### Implemented
 - **Dataset Creation**: Create Lance datasets with schemas
-- **Data Storage**: Add documents to datasets  
+- **Data Storage**: Add documents to datasets
 - **Document Retrieval**: Read documents from datasets with enumerable support
 - **Vector Search**: Create vector indices and perform similarity search
+- **Full-Text Search**: Built-in full-text search with inverted indices
+- **Hybrid Search**: Combine text and vector search with Reciprocal Rank Fusion (RRF)
 - **Schema Support**: Define schemas with string, float32, and vector types
 - **Row Counting**: Get the number of rows in a dataset
 
-### Planned
-
-- **Full-Text Search**: Built-in full-text search capabilities  
-- **Hybrid Search**: Combine text and vector search with RRF and other fusion methods
-- **Multimodal Support**: Store and search across different data types beyond text and vectors
-- **Schema Evolution**: Add new columns to existing datasets without rewriting data
-
 ## Installation
 
 Install the gem and add to the application's Gemfile by executing:
@@ -121,10 +157,109 @@ results = dataset.text_search("programming", columns: ["title", "content", "tags
 
 **Note**: Full-text search requires creating inverted indices first. For simple pattern matching without indices, use SQL-like filtering with `where`.
 
+### Hybrid Search with Reciprocal Rank Fusion (RRF)
+
+Lancelot now supports hybrid search, combining vector and text search results using Reciprocal Rank Fusion:
+
+```ruby
+# Example 1: Using the same query for both vector and text search
+# First, let's assume we have a function that converts text to embeddings
+def text_to_embedding(text)
+  # Your embedding model here (e.g., using red-candle or another embedding service)
+  # Returns a vector representation of the text
+end
+
+# Search using both modalities with the same query
+query = "machine learning frameworks"
+query_embedding = text_to_embedding(query)
+
+results = dataset.hybrid_search(
+  query,                           # Text query
+  vector: query_embedding,         # Vector query (same content, embedded)
+  vector_column: "embedding",      # Vector column to search
+  text_column: "content",          # Text column to search
+  limit: 10
+)
+
+# Results are fused using RRF and include an rrf_score
+results.each do |doc|
+  puts "#{doc[:title]} - RRF Score: #{doc[:rrf_score]}"
+end
+```
+
+```ruby
+# Example 2: Multiple queries across different modalities
+# You can use different queries for vector and text search
+
+# Semantic vector search for conceptually similar content
+concept_embedding = text_to_embedding("deep learning neural networks")
+
+# Keyword text search for specific terms
+keyword_query = "PyTorch TensorFlow"
+
+results = dataset.hybrid_search(
+  keyword_query,                   # Specific keyword search
+  vector: concept_embedding,       # Broader semantic search
+  vector_column: "embedding",
+  text_column: "content",
+  limit: 20
+)
+```
+
+```ruby
+# Example 3: Multi-column text search with vector search
+# Search across multiple text columns while also doing vector similarity
+
+results = dataset.hybrid_search(
+  "ruby programming",
+  vector: text_to_embedding("object-oriented scripting language"),
+  vector_column: "embedding",
+  text_columns: ["title", "content", "tags"],  # Search multiple text columns
+  limit: 15
+)
+```
+
+```ruby
+# Example 4: Advanced RRF with custom k parameter
+# The k parameter (default 60) controls the fusion behavior
+# Lower k values give more weight to top-ranked results
+
+results = dataset.hybrid_search(
+  "distributed systems",
+  vector: text_to_embedding("distributed systems"),
+  vector_column: "embedding",
+  text_column: "content",
+  limit: 10,
+  rrf_k: 30  # More aggressive fusion, emphasizes top results
+)
+```
+
+```ruby
+# Example 5: Using RankFusion module directly for custom fusion
+# Useful when you want to combine results from multiple separate searches
+require 'lancelot/rank_fusion'
+
+# Perform multiple searches with different queries
+vector_results1 = dataset.vector_search(embedding1, column: "embedding", limit: 20)
+vector_results2 = dataset.vector_search(embedding2, column: "embedding", limit: 20)
+text_results1 = dataset.text_search("machine learning", column: "content", limit: 20)
+text_results2 = dataset.text_search("neural networks", column: "title", limit: 20)
+
+# Fuse all results using RRF
+fused_results = Lancelot::RankFusion.reciprocal_rank_fusion(
+  [vector_results1, vector_results2, text_results1, text_results2],
+  k: 60
+)
+
+# Take top 10 fused results
+top_results = fused_results.first(10)
+```
+
+**RRF Algorithm**: Reciprocal Rank Fusion calculates scores as `Σ(1/(k+rank))` across all result lists, where k=60 by default. Documents appearing in multiple result lists with high ranks get higher RRF scores.
+
 **Current Limitations:**
 - Schema must be defined when creating a dataset
 - Schema evolution is not yet implemented (Lance supports it, but our bindings don't expose it yet)
-- Hybrid search (RRF) is not yet implemented
 - Supported field types: string, float32, float64, int32, int64, boolean, and fixed-size vectors
 
 **Note on Lance's Schema Flexibility:**

data/examples/idempotent_create.rb

@@ -0,0 +1,66 @@
+#!/usr/bin/env ruby
+# Demonstrates idempotent dataset creation with open_or_create
+
+require 'bundler/setup'
+require 'lancelot'
+require 'fileutils'
+
+dataset_path = "words"
+
+puts "="*60
+puts "Idempotent Dataset Creation Demo"
+puts "="*60
+
+schema = {
+  text: :string,
+  embedding: { type: "vector", dimension: 768 }
+}
+
+# First call - will CREATE the dataset
+puts "\n1. First call to open_or_create (should create)..."
+dataset = Lancelot::Dataset.open_or_create(dataset_path, schema: schema)
+puts "   Dataset opened/created. Current count: #{dataset.count}"
+
+# Add some data
+dataset.add_documents([
+  { text: "hello", embedding: Array.new(768) { rand } },
+  { text: "world", embedding: Array.new(768) { rand } }
+])
+puts "   Added 2 documents. New count: #{dataset.count}"
+
+# Second call - will OPEN the existing dataset
+puts "\n2. Second call to open_or_create (should open existing)..."
+dataset2 = Lancelot::Dataset.open_or_create(dataset_path, schema: schema)
+puts "   Dataset opened. Current count: #{dataset2.count}"
+puts "   ✓ Data persisted from previous session!"
+
+# Third call - still idempotent
+puts "\n3. Third call - still works..."
+dataset3 = Lancelot::Dataset.open_or_create(dataset_path, schema: schema)
+dataset3.add_documents([
+  { text: "more", embedding: Array.new(768) { rand } }
+])
+puts "   Added 1 more document. New count: #{dataset3.count}"
+
+# Demonstrate the OLD way that would fail
+puts "\n4. Compare with non-idempotent create (would fail)..."
+begin
+  # This will fail because dataset already exists
+  failing_dataset = Lancelot::Dataset.create(dataset_path, schema: schema)
+  puts "   ✗ This shouldn't happen!"
+rescue => e
+  puts "   ✓ Dataset.create correctly failed: #{e.class}"
+  puts "   Message: #{e.message[0..50]}..."
+end
+
+# Clean up
+FileUtils.rm_rf(dataset_path)
+
+puts "\n" + "="*60
+puts "Summary: Use open_or_create for idempotent operations!"
+puts "="*60
+puts "\nInstead of:"
+puts '  dataset = Lancelot::Dataset.create("words", schema: {...})'
+puts "\nUse:"
+puts '  dataset = Lancelot::Dataset.open_or_create("words", schema: {...})'
+puts "\nThis way your code works whether the dataset exists or not!"

data/examples/optional_fields_demo.rb

@@ -0,0 +1,83 @@
+#!/usr/bin/env ruby
+# This example demonstrates optional field support in lancelot
+# After the fix in conversion.rs, documents can have missing fields
+
+require 'bundler/setup'
+require 'lancelot'
+require 'fileutils'
+
+dataset_path = "example_optional_fields"
+FileUtils.rm_rf(dataset_path)
+
+puts "="*60
+puts "Lancelot Optional Fields Demo"
+puts "="*60
+
+# Step 1: Create dataset with initial schema
+puts "\n1. Creating dataset with 3 fields (id, text, score)..."
+schema = {
+  id: :string,
+  text: :string,
+  score: :float32
+}
+dataset = Lancelot::Dataset.create(dataset_path, schema: schema)
+
+# Add initial documents
+initial_docs = [
+  { id: "1", text: "First document", score: 0.9 },
+  { id: "2", text: "Second document", score: 0.8 }
+]
+dataset.add_documents(initial_docs)
+puts "   Added #{dataset.count} documents"
+
+# Step 2: Simulate schema evolution (adding a new field)
+puts "\n2. Simulating schema evolution (adding 'category' field)..."
+
+# Get existing data
+all_docs = dataset.to_a
+
+# Recreate with expanded schema
+FileUtils.rm_rf(dataset_path)
+expanded_schema = {
+  id: :string,
+  text: :string,
+  score: :float32,
+  category: :string  # NEW FIELD
+}
+dataset = Lancelot::Dataset.create(dataset_path, schema: expanded_schema)
+
+# Re-add existing docs with the new field
+docs_with_category = all_docs.map { |doc| doc.merge(category: "original") }
+dataset.add_documents(docs_with_category)
+puts "   Recreated dataset with expanded schema"
+
+# Step 3: Add new documents WITHOUT the new field
+puts "\n3. Adding new documents WITHOUT the 'category' field..."
+new_docs = [
+  { id: "3", text: "Third document", score: 0.7 },  # No category!
+  { id: "4", text: "Fourth document", score: 0.6 }  # No category!
+]
+
+begin
+  dataset.add_documents(new_docs)
+  puts "   ✅ SUCCESS! Added #{new_docs.size} documents with missing fields"
+rescue => e
+  puts "   ❌ FAILED: #{e.message}"
+  puts "   (This would have failed before the fix in conversion.rs)"
+end
+
+# Step 4: Verify the data
+puts "\n4. Verifying all documents..."
+dataset.to_a.each do |doc|
+  category = doc[:category] || "nil"
+  puts "   Doc #{doc[:id]}: category=#{category}"
+end
+
+puts "\nTotal documents: #{dataset.count}"
+
+# Cleanup
+FileUtils.rm_rf(dataset_path)
+
+puts "\n" + "="*60
+puts "Demo complete! Optional fields work correctly."
+puts "="*60

data/ext/lancelot/src/conversion.rs

@@ -39,11 +39,13 @@ pub fn build_record_batch(
         let item = RHash::try_convert(item)?;
         for field in schema.fields() {
             let key = Symbol::new(field.name());
-            let value: Value = item.fetch(key)
-                .or_else(|_| {
+            // Make fields optional - use get instead of fetch
+            let value: Value = item.get(key)
+                .or_else(|| {
                     // Try with string key  
-                    item.fetch(field.name().as_str())
-                })?;
+                    item.get(field.name().as_str())
+                })
+                .unwrap_or_else(|| Ruby::get().unwrap().qnil().as_value());
             
             match field.data_type() {
                 DataType::Utf8 => {

data/lib/lancelot/dataset.rb

@@ -15,6 +15,14 @@ module Lancelot
         dataset
       end
 
+      def open_or_create(path, schema:)
+        if File.exist?(path)
+          open(path)
+        else
+          create(path, schema: schema)
+        end
+      end
+
       private
 
       def normalize_schema(schema)
@@ -106,6 +114,38 @@ module Lancelot
       end
     end
 
+    def hybrid_search(query, vector_column: "vector", text_column: nil, text_columns: nil, 
+                      vector: nil, limit: 10, rrf_k: 60)
+      require 'lancelot/rank_fusion'
+      
+      result_lists = []
+      
+      # Perform vector search if vector is provided
+      if vector
+        unless vector.is_a?(Array)
+          raise ArgumentError, "Vector must be an array of numbers"
+        end
+        
+        vector_results = vector_search(vector, column: vector_column, limit: limit * 2)
+        result_lists << vector_results if vector_results.any?
+      end
+      
+      # Perform text search if query is provided
+      if query && !query.empty?
+        text_results = text_search(query, column: text_column, columns: text_columns, limit: limit * 2)
+        result_lists << text_results if text_results.any?
+      end
+      
+      # Return empty array if no searches were performed
+      return [] if result_lists.empty?
+      
+      # Return single result list if only one search was performed
+      return result_lists.first[0...limit] if result_lists.size == 1
+      
+      # Perform RRF fusion and limit results
+      Lancelot::RankFusion.reciprocal_rank_fusion(result_lists, k: rrf_k)[0...limit]
+    end
+
     def where(filter_expression, limit: nil)
       filter_scan(filter_expression.to_s, limit)
     end

data/lib/lancelot/rank_fusion.rb

@@ -0,0 +1,84 @@
+# frozen_string_literal: true
+
+module Lancelot
+  module RankFusion
+    class << self
+      def reciprocal_rank_fusion(result_lists, k: 60)
+        return [] if result_lists.nil? || result_lists.empty?
+
+        # Validate inputs
+        result_lists = Array(result_lists)
+        validate_result_lists(result_lists)
+
+        return [] if result_lists.all?(&:empty?)
+
+        # Build document to rank mapping for each result list
+        doc_ranks = build_document_ranks(result_lists)
+
+        # Calculate RRF scores
+        rrf_scores = calculate_rrf_scores(doc_ranks, result_lists.size, k)
+
+        # Sort by RRF score descending and return results
+        rrf_scores.sort_by { |_, score| -score }.map do |doc, score|
+          doc.merge(rrf_score: score)
+        end
+      end
+
+      private
+
+      def validate_result_lists(result_lists)
+        result_lists.each_with_index do |list, i|
+          unless list.is_a?(Array)
+            raise ArgumentError, "Result list at index #{i} must be an Array, got #{list.class}"
+          end
+
+          list.each_with_index do |doc, j|
+            unless doc.is_a?(Hash)
+              raise ArgumentError, "Document at position #{j} in result list #{i} must be a Hash, got #{doc.class}"
+            end
+          end
+        end
+      end
+
+      def build_document_ranks(result_lists)
+        doc_ranks = {}
+
+        result_lists.each_with_index do |list, list_idx|
+          list.each_with_index do |doc, rank|
+            # Use the document content as the key (excluding metadata like distance/score)
+            doc_key = normalize_document(doc)
+            doc_ranks[doc_key] ||= {document: doc, ranks: {}}
+            doc_ranks[doc_key][:ranks][list_idx] = rank + 1  # 1-based ranking
+          end
+        end
+
+        doc_ranks
+      end
+
+      def calculate_rrf_scores(doc_ranks, num_lists, k)
+        doc_ranks.map do |doc_key, data|
+          score = 0.0
+
+          num_lists.times do |list_idx|
+            rank = data[:ranks][list_idx]
+            if rank
+              score += 1.0 / (k + rank)
+            else
+              # Document doesn't appear in this list, treat as infinite rank
+              # RRF score contribution is effectively 0
+            end
+          end
+
+          [data[:document], score]
+        end
+      end
+
+      def normalize_document(doc)
+        # Create a normalized version for comparison, excluding metadata fields
+        # that might differ between search types (like _distance, _score, etc.)
+        doc.reject { |k, _| k.to_s.start_with?("_") || k == :rrf_score }
+      end
+    end
+  end
+end
+

data/lib/lancelot/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Lancelot
-  VERSION = "0.1.1"
+  VERSION = "0.3.1"
 end

data/lib/lancelot.rb

@@ -3,6 +3,7 @@
 require_relative "lancelot/version"
 require_relative "lancelot/lancelot"
 require_relative "lancelot/dataset"
+require_relative "lancelot/rank_fusion"
 
 module Lancelot
   class Error < StandardError; end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: lancelot
 version: !ruby/object:Gem::Version
-  version: 0.1.1
+  version: 0.3.1
 platform: ruby
 authors:
 - Chris Petersen
 autorequire:
 bindir: exe
 cert_chain: []
-date: 2025-07-26 00:00:00.000000000 Z
+date: 2025-08-10 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rb_sys
@@ -106,12 +106,15 @@ files:
 - ".rspec"
 - ".standard.yml"
 - CHANGELOG.md
+- CLAUDE.md
 - CODE_OF_CONDUCT.md
 - LICENSE.txt
 - README.md
 - Rakefile
 - examples/basic_usage.rb
 - examples/full_text_search.rb
+- examples/idempotent_create.rb
+- examples/optional_fields_demo.rb
 - examples/red_candle_integration.rb
 - examples/vector_search.rb
 - ext/lancelot/.gitignore
@@ -123,6 +126,7 @@ files:
 - ext/lancelot/src/schema.rs
 - lib/lancelot.rb
 - lib/lancelot/dataset.rb
+- lib/lancelot/rank_fusion.rb
 - lib/lancelot/version.rb
 - sig/lancelot.rbs
 homepage: https://github.com/cpetersen/lancelot