vectra-client 0.2.0 → 0.2.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 23e0684b20e30c7083b21676f50611b82c2926baec2cd2596a02aa9d0f1524b2
-  data.tar.gz: 892a0094e23003e7c4647a931dca2b785c243439bad35d3511391df0b3275587
+  metadata.gz: 4fb1baf885e79af651a89ca695ead6a909a39c495241586ec3a46bdbecfb48ec
+  data.tar.gz: 9e7d3c14ea8108531821790c2d6d9f3d76901f122ab2d78472bf9ac6dbfc32dc
 SHA512:
-  metadata.gz: 356585c134240dc3b36734f85f3a68965ee18f16dfe1437737628d1c8b9c0c9fcbaca11e7092e1328501fd861f1e1c297a8bad38197a1b0033a9af5159ddb075
-  data.tar.gz: 0abf8ee4bf16c01065db7dad529ff3da125a0f4bb45a91be19a94842405fdfc8e8c86a9aed7ae153b68323e528111fbea13ffbb2888b1216b1b045cbf1937888
+  metadata.gz: b27299838743d313b9416e7c1d54f0bf694853df45526e9e6cdc51f3c2744b2254730100704ca980435ed982baf4d9341ce94a57e9eee8d9da3cc18ceb452fd1
+  data.tar.gz: a4a1ba817e2d094dde020471c63ec954f40de90e5389726b9c486da1aecd7140b80e3f4a0f4a773d654ccb75e3e65dd793bfd655ae749ae2c80a001e9a0259a8

data/CHANGELOG.md

@@ -7,6 +7,33 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [0.2.1] - 2025-01-08
+
+### Added
+
+- **Qdrant Provider** - Full Qdrant vector database support:
+  - Vector upsert, query, fetch, update, delete operations
+  - Collection management (create, list, describe, delete)
+  - Multiple similarity metrics: cosine, euclidean, dot product
+  - Namespace support via payload filtering
+  - Advanced metadata filtering with Qdrant operators ($eq, $ne, $gt, $gte, $lt, $lte, $in, $nin)
+  - Automatic point ID hashing for string IDs
+  - Support for both local and cloud Qdrant instances
+  - Optional API key authentication for local deployments
+
+### Improved
+
+- Enhanced error handling with `Faraday::RetriableResponse` support
+- Configuration now allows optional API key for Qdrant and pgvector (local instances)
+- Better retry middleware integration across all providers
+
+### Provider Support
+
+- ✅ Pinecone - Fully implemented
+- ✅ pgvector (PostgreSQL) - Fully implemented
+- ✅ Qdrant - Fully implemented
+- 🚧 Weaviate - Stub implementation (planned for v0.3.0)
+
 ## [0.2.0] - 2025-01-08
 
 ### Added
@@ -49,13 +76,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - Updated gemspec description to include pgvector
 - Added `pg` gem as development dependency
 
-### Provider Support
-
-- ✅ Pinecone - Fully implemented
-- ✅ pgvector (PostgreSQL) - Fully implemented
-- 🚧 Qdrant - Stub implementation (planned for v0.2.0)
-- 🚧 Weaviate - Stub implementation (planned for v0.3.0)
-
 ## [0.1.0] - 2024-XX-XX
 
 ### Added
@@ -89,13 +109,6 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## Planned
 
-### [0.2.0]
-
-- Qdrant provider implementation
-- Enhanced error messages
-- Connection pooling
-- Improved retry strategies
-
 ### [0.3.0]
 
 - Weaviate provider implementation

data/README.md

@@ -1,434 +1,246 @@
-# Vectra
+# Vectra 🚀
 
-[![Gem Version](https://badge.fury.io/rb/vectra.svg)](https://badge.fury.io/rb/vectra)
+[![Gem Version](https://badge.fury.io/rb/vectra-client.svg)](https://rubygems.org/gems/vectra-client)
 [![CI](https://github.com/stokry/vectra/actions/workflows/ci.yml/badge.svg)](https://github.com/stokry/vectra/actions)
 [![codecov](https://codecov.io/gh/stokry/vectra/branch/main/graph/badge.svg)](https://codecov.io/gh/stokry/vectra)
 [![Ruby Style Guide](https://img.shields.io/badge/code_style-rubocop-brightgreen.svg)](https://github.com/rubocop/rubocop)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 [![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.0-4baaaa.svg)](CODE_OF_CONDUCT.md)
 
-**Vectra** is a unified Ruby client for vector databases. Write once, switch providers seamlessly.
+> **A unified Ruby client for vector databases.** Write once, switch vector database providers seamlessly. Perfect for AI/ML applications, semantic search, and RAG (Retrieval Augmented Generation) systems.
 
-## Features
+## 📖 Complete Documentation
 
-- 🔌 **Unified API** - One interface for multiple vector databases
-- 🚀 **Modern Ruby** - Built for Ruby 3.2+ with modern patterns
-- 🔄 **Automatic Retries** - Built-in retry logic with exponential backoff
-- 📊 **Rich Results** - Enumerable query results with filtering capabilities
-- 🛡️ **Type Safety** - Comprehensive validation and meaningful errors
-- 📝 **Well Documented** - Extensive YARD documentation
+**Full documentation, guides, and API reference available at:** [**https://vectra-docs.netlify.app/**](https://vectra-docs.netlify.app/)
 
-## Supported Providers
+This README provides a quick overview. For detailed guides, examples, and API documentation, visit the official documentation site above.
 
-| Provider | Status | Version |
-|----------|--------|---------|
-| [Pinecone](https://pinecone.io) | ✅ Fully Supported | v0.1.0 |
-| [PostgreSQL + pgvector](https://github.com/pgvector/pgvector) | ✅ Fully Supported | v0.1.1 |
-| [Qdrant](https://qdrant.tech) | 🚧 Planned | v0.2.0 |
-| [Weaviate](https://weaviate.io) | 🚧 Planned | v0.3.0 |
+---
 
-## Installation
+## ✨ Key Features
+
+- 🔌 **Provider Agnostic** - Switch between vector database providers with minimal code changes
+- 🚀 **Production Ready** - Built for Ruby 3.2+ with comprehensive test coverage (95%+)
+- 🔄 **Resilient** - Built-in retry logic with exponential backoff and circuit breaker patterns
+- 📊 **Rich Results** - Enumerable query results with advanced filtering and mapping capabilities
+- 🛡️ **Type Safe** - Comprehensive input validation and meaningful error messages
+- 📈 **Observable** - Native instrumentation support for Datadog and New Relic
+- 🏗️ **Rails Ready** - Seamless ActiveRecord integration with migrations support
+- 📚 **Well Documented** - Extensive YARD documentation and comprehensive examples
+
+## 🗄️ Supported Vector Databases
+
+| Provider | Type | Status | Docs |
+|----------|------|--------|------|
+| **Pinecone** | Managed Cloud | ✅ Fully Supported | [Guide](https://vectra-docs.netlify.app/providers/pinecone) |
+| **PostgreSQL + pgvector** | SQL Database | ✅ Fully Supported | [Guide](https://vectra-docs.netlify.app/providers/pgvector) |
+| **Qdrant** | Open Source | ✅ Fully Supported | [Guide](https://vectra-docs.netlify.app/providers/qdrant) |
+| **Weaviate** | Open Source | ✅ Fully Supported | [Guide](https://vectra-docs.netlify.app/providers/weaviate) |
+
+
+
+## 📦 Installation
 
 Add this line to your application's Gemfile:
 
 ```ruby
-gem 'vectra'
+gem 'vectra-client'
 ```
 
-And then execute:
+Then execute:
 
 ```bash
 bundle install
 ```
 
-Or install it yourself:
+Or install directly:
 
 ```bash
-gem install vectra
+gem install vectra-client
 ```
 
-### Provider-Specific Dependencies
+### RubyGems Information
+
+- **Gem Name:** `vectra-client`
+- **Latest Version:** 0.2.1
+- **Repository:** [stokry/vectra](https://github.com/stokry/vectra)
+- **RubyGems Page:** [vectra-client](https://rubygems.org/gems/vectra-client)
+- **License:** MIT
+- **Ruby Requirement:** >= 3.2.0
 
-For **pgvector** support, add the `pg` gem:
+### Provider-Specific Setup
 
+Each vector database may require additional dependencies:
+
+**PostgreSQL + pgvector:**
 ```ruby
 gem 'pg', '~> 1.5'
 ```
 
-## Quick Start
-
-### Configuration
-
+**For Instrumentation:**
 ```ruby
-require 'vectra'
-
-# Global configuration
-Vectra.configure do |config|
-  config.provider = :pinecone
-  config.api_key = ENV['PINECONE_API_KEY']
-  config.environment = 'us-east-1'  # or config.host = 'your-index-host.pinecone.io'
-end
-
-# Create a client
-client = Vectra::Client.new
+gem 'dogstatsd-ruby'      # Datadog
+gem 'newrelic_rpm'        # New Relic
 ```
 
-Or use per-client configuration:
+## 🚀 Quick Start
+
+### 1. Initialize a Client
 
 ```ruby
-# Shortcut for Pinecone
-client = Vectra.pinecone(
+require 'vectra'
+
+# Pinecone
+client = Vectra::Client.new(
+  provider: :pinecone,
   api_key: ENV['PINECONE_API_KEY'],
-  environment: 'us-east-1'
+  environment: 'us-west-4'
 )
 
-# Shortcut for pgvector (PostgreSQL)
-client = Vectra.pgvector(
-  connection_url: 'postgres://user:password@localhost/mydb'
+# PostgreSQL + pgvector
+client = Vectra::Client.new(
+  provider: :pgvector,
+  database: 'my_app_production',
+  host: 'localhost'
 )
 
-# Generic client with options
+# Qdrant
 client = Vectra::Client.new(
-  provider: :pinecone,
-  api_key: ENV['PINECONE_API_KEY'],
-  environment: 'us-east-1',
-  timeout: 60,
-  max_retries: 5
+  provider: :qdrant,
+  host: 'http://localhost:6333'
 )
 ```
 
-### Basic Operations
-
-#### Upsert Vectors
+### 2. Upsert Vectors
 
 ```ruby
 client.upsert(
-  index: 'my-index',
   vectors: [
-    { id: 'vec1', values: [0.1, 0.2, 0.3], metadata: { text: 'Hello world' } },
-    { id: 'vec2', values: [0.4, 0.5, 0.6], metadata: { text: 'Ruby is great' } }
+    { 
+      id: 'doc-1', 
+      values: [0.1, 0.2, 0.3], 
+      metadata: { title: 'Introduction to AI' }
+    },
+    { 
+      id: 'doc-2', 
+      values: [0.2, 0.3, 0.4], 
+      metadata: { title: 'Advanced ML Techniques' }
+    }
   ]
 )
-# => { upserted_count: 2 }
 ```
 
-#### Query Vectors
+### 3. Search for Similar Vectors
 
 ```ruby
 results = client.query(
-  index: 'my-index',
   vector: [0.1, 0.2, 0.3],
-  top_k: 5,
-  include_metadata: true
+  top_k: 5
 )
 
-# Iterate over results
-results.each do |match|
-  puts "ID: #{match.id}, Score: #{match.score}"
-  puts "Metadata: #{match.metadata}"
+results.matches.each do |match|
+  puts "#{match['id']}: #{match['score']}"
+  puts "  Metadata: #{match['metadata']}"
 end
-
-# Access specific results
-results.first          # First match
-results.ids            # All matching IDs
-results.scores         # All scores
-results.max_score      # Highest score
-
-# Filter by score
-high_quality = results.above_score(0.8)
 ```
 
-#### Query with Filters
+### 4. Delete Vectors
 
 ```ruby
-results = client.query(
-  index: 'my-index',
-  vector: [0.1, 0.2, 0.3],
-  top_k: 10,
-  filter: { category: 'programming', language: 'ruby' }
-)
-```
-
-#### Fetch Vectors by ID
-
-```ruby
-vectors = client.fetch(index: 'my-index', ids: ['vec1', 'vec2'])
-
-vectors['vec1'].values    # [0.1, 0.2, 0.3]
-vectors['vec1'].metadata  # { 'text' => 'Hello world' }
+client.delete(ids: ['doc-1', 'doc-2'])
 ```
 
-#### Update Vector Metadata
+## 📖 Full Documentation
 
-```ruby
-client.update(
-  index: 'my-index',
-  id: 'vec1',
-  metadata: { category: 'updated', processed: true }
-)
-```
-
-#### Delete Vectors
-
-```ruby
-# Delete by IDs
-client.delete(index: 'my-index', ids: ['vec1', 'vec2'])
+For complete documentation, examples, and guides, visit:
 
-# Delete by filter
-client.delete(index: 'my-index', filter: { category: 'old' })
+**👉 [https://vectra-docs.netlify.app/](https://vectra-docs.netlify.app/)**
 
-# Delete all (use with caution!)
-client.delete(index: 'my-index', delete_all: true)
-```
+### Documentation Includes:
 
-### Working with Vectors
+- [Installation Guide](https://vectra-docs.netlify.app/guides/installation)
+- [Getting Started](https://vectra-docs.netlify.app/guides/getting-started)
+- [Provider Guides](https://vectra-docs.netlify.app/providers)
+  - [Pinecone](https://vectra-docs.netlify.app/providers/pinecone)
+  - [PostgreSQL + pgvector](https://vectra-docs.netlify.app/providers/pgvector)
+  - [Qdrant](https://vectra-docs.netlify.app/providers/qdrant)
+  - [Weaviate](https://vectra-docs.netlify.app/providers/weaviate)
+- [API Reference](https://vectra-docs.netlify.app/api/overview)
+- [Code Examples](https://vectra-docs.netlify.app/examples)
+- [Rails Integration Guide](https://vectra-docs.netlify.app/providers/pgvector/)
 
-```ruby
-# Create a Vector object
-vector = Vectra::Vector.new(
-  id: 'my-vector',
-  values: [0.1, 0.2, 0.3],
-  metadata: { text: 'Example' }
-)
+---
 
-vector.dimension        # => 3
-vector.metadata?        # => true
-vector.to_h             # Convert to hash
+## 💡 Use Cases
 
-# Calculate similarity
-other = Vectra::Vector.new(id: 'other', values: [0.1, 0.2, 0.3])
-vector.cosine_similarity(other)    # => 1.0 (identical)
-vector.euclidean_distance(other)   # => 0.0
-```
+### Semantic Search
+Build intelligent search that understands meaning, not just keywords. Perfect for product discovery, knowledge base search, and content recommendation.
 
-### Index Management
+### Retrieval Augmented Generation (RAG)
+Combine vector databases with LLMs to enable precise information retrieval for AI applications, chatbots, and knowledge systems.
 
-```ruby
-# List all indexes
-indexes = client.list_indexes
-indexes.each { |idx| puts idx[:name] }
-
-# Describe an index
-info = client.describe_index(index: 'my-index')
-puts info[:dimension]  # => 384
-puts info[:metric]     # => "cosine"
-
-# Get index statistics
-stats = client.stats(index: 'my-index')
-puts stats[:total_vector_count]
-```
+### Duplicate Detection
+Identify and deduplicate similar items across your dataset using vector similarity.
 
-### Namespaces
+### Recommendation Systems
+Power personalized recommendations based on user behavior and content similarity.
 
-Namespaces allow you to partition vectors within an index:
+---
 
-```ruby
-# Upsert to a namespace
-client.upsert(
-  index: 'my-index',
-  namespace: 'production',
-  vectors: [...]
-)
 
-# Query within a namespace
-client.query(
-  index: 'my-index',
-  namespace: 'production',
-  vector: [0.1, 0.2, 0.3],
-  top_k: 5
-)
-```
+## 🛠️ Development
 
-### pgvector (PostgreSQL)
-
-pgvector uses PostgreSQL tables as indexes. Each "index" is a table with a vector column.
-
-#### Setup PostgreSQL with pgvector
+### Set Up Development Environment
 
 ```bash
-# Using Docker
-docker run -d --name pgvector \
-  -e POSTGRES_PASSWORD=password \
-  -p 5432:5432 \
-  pgvector/pgvector:pg16
-```
-
-#### Create an Index (Table)
-
-```ruby
-client = Vectra.pgvector(connection_url: 'postgres://postgres:password@localhost/postgres')
-
-# Create a new index with cosine similarity
-client.provider.create_index(
-  name: 'documents',
-  dimension: 384,
-  metric: 'cosine'  # or 'euclidean', 'inner_product'
-)
-```
-
-#### Supported Metrics
-
-| Metric | Description | pgvector Operator |
-|--------|-------------|-------------------|
-| `cosine` | Cosine similarity (default) | `<=>` |
-| `euclidean` | Euclidean distance | `<->` |
-| `inner_product` | Inner product / dot product | `<#>` |
-
-#### Table Structure
-
-Vectra creates tables with the following structure:
-
-```sql
-CREATE TABLE documents (
-  id TEXT PRIMARY KEY,
-  embedding vector(384),
-  metadata JSONB DEFAULT '{}',
-  namespace TEXT DEFAULT '',
-  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-);
-
--- IVFFlat index for fast similarity search
-CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);
-```
-
-## Configuration Options
-
-| Option | Description | Default |
-|--------|-------------|---------|
-| `provider` | Vector database provider (`:pinecone`, `:pgvector`, `:qdrant`, `:weaviate`) | Required |
-| `api_key` | API key for authentication (password for pgvector) | Required* |
-| `environment` | Environment/region (Pinecone) | - |
-| `host` | Direct host URL or PostgreSQL connection URL | - |
-| `timeout` | Request timeout in seconds | 30 |
-| `open_timeout` | Connection timeout in seconds | 10 |
-| `max_retries` | Maximum retry attempts | 3 |
-| `retry_delay` | Initial retry delay in seconds | 1 |
-| `logger` | Logger instance for debugging | nil |
-
-*For pgvector, `api_key` is used as the PostgreSQL password.
-
-## Error Handling
-
-Vectra provides specific error classes for different failure scenarios:
-
-```ruby
-begin
-  client.query(index: 'my-index', vector: [0.1, 0.2], top_k: 5)
-rescue Vectra::AuthenticationError => e
-  puts "Authentication failed: #{e.message}"
-rescue Vectra::RateLimitError => e
-  puts "Rate limited. Retry after #{e.retry_after} seconds"
-rescue Vectra::NotFoundError => e
-  puts "Resource not found: #{e.message}"
-rescue Vectra::ValidationError => e
-  puts "Invalid request: #{e.message}"
-rescue Vectra::ServerError => e
-  puts "Server error (#{e.status_code}): #{e.message}"
-rescue Vectra::Error => e
-  puts "General error: #{e.message}"
-end
-```
-
-## Logging
-
-Enable debug logging to see request details:
-
-```ruby
-require 'logger'
-
-Vectra.configure do |config|
-  config.provider = :pinecone
-  config.api_key = ENV['PINECONE_API_KEY']
-  config.environment = 'us-east-1'
-  config.logger = Logger.new($stdout)
-end
+git clone https://github.com/stokry/vectra.git
+cd vectra
+bundle install
 ```
 
-## Best Practices
+### Run Tests
 
-### Batch Upserts
+```bash
+# All tests
+bundle exec rspec
 
-For large datasets, batch your upserts:
+# Unit tests only
+bundle exec rspec spec/vectra
 
-```ruby
-vectors = large_dataset.each_slice(100).map do |batch|
-  client.upsert(index: 'my-index', vectors: batch)
-end
+# Integration tests only
+bundle exec rspec spec/integration
 ```
 
-### Connection Reuse
+### Code Quality
 
-Create a single client instance and reuse it:
+```bash
+# Run RuboCop linter
+bundle exec rubocop
 
-```ruby
-# Good: Reuse the client
-client = Vectra::Client.new(...)
-client.query(...)
-client.upsert(...)
-
-# Avoid: Creating new clients for each operation
-Vectra::Client.new(...).query(...)
-Vectra::Client.new(...).upsert(...)
+# Generate documentation
+bundle exec rake docs
 ```
 
-### Error Recovery
+## 🤝 Contributing
 
-Implement retry logic for transient failures:
+We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.
 
-```ruby
-def query_with_retry(client, **params, retries: 3)
-  client.query(**params)
-rescue Vectra::RateLimitError => e
-  if retries > 0
-    sleep(e.retry_after || 1)
-    retry(retries: retries - 1)
-  else
-    raise
-  end
-end
-```
+See [CHANGELOG.md](CHANGELOG.md) for the complete history of changes.
 
-## Development
+## 🔗 Links
 
-After checking out the repo:
+- **Documentation:** [https://vectra-docs.netlify.app/](https://vectra-docs.netlify.app/)
+- **RubyGems:** [vectra-client](https://rubygems.org/gems/vectra-client)
+- **GitHub:** [stokry/vectra](https://github.com/stokry/vectra)
+- **Issues:** [Report a bug](https://github.com/stokry/vectra/issues)
 
-```bash
-# Install dependencies
-bundle install
+## 📄 License
 
-# Run tests
-bundle exec rspec
-
-# Run linter
-bundle exec rubocop
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
 
-# Generate documentation
-bundle exec rake docs
-```
+---
 
-## Roadmap
-
-### v0.1.0
-- ✅ Pinecone provider
-- ✅ Basic CRUD operations
-- ✅ Configuration system
-- ✅ Error handling with retries
-- ✅ Comprehensive tests
-
-### v0.1.1 (Current)
-- ✅ pgvector (PostgreSQL) provider
-- ✅ Multiple similarity metrics (cosine, euclidean, inner product)
-- ✅ Namespace support for pgvector
-- ✅ IVFFlat index creation
-
-### v0.2.0
-- 🚧 Qdrant provider
-- 🚧 Enhanced error handling
-- 🚧 Connection pooling
-
-### v0.3.0
-- 🚧 Weaviate provider
-- 🚧 Batch operations
+**Built with ❤️ by the Vectra community**
 - 🚧 Performance optimizations
 
 ### v1.0.0

data/Rakefile

@@ -34,7 +34,7 @@ task :changelog do
 end
 
 desc "Bump version to VERSION"
-task :bump_version, [:version] do |t, args|
+task :bump_version, [:version] do |_t, args|
   version = args[:version]
   raise "Version required: rake bump_version[1.2.3]" unless version
 

data/docs/Gemfile

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gem "jekyll", "~> 4.4"
+gem "jekyll-feed", "~> 0.12"
+gem "jekyll-seo-tag"
+gem "jekyll-sitemap"
+gem "minima", "~> 2.5"
+gem "webrick", "~> 1.8"

data/docs/_config.yml

@@ -0,0 +1,20 @@
+title: Vectra Documentation
+description: Unified Ruby client for vector databases
+theme: minima
+plugins:
+  - jekyll-feed
+  - jekyll-sitemap
+
+markdown: kramdown
+highlighter: rouge
+
+exclude:
+  - Gemfile
+  - Gemfile.lock
+  - node_modules
+  - vendor/bundle/
+  - vendor/cache/
+  - vendor/gems/
+  - vendor/ruby/
+
+permalink: /:categories/:title/