vectra-client 0.2.1 → 0.3.0

data/README.md

@@ -1,462 +1,121 @@
 # Vectra
 
-[![Gem Version](https://badge.fury.io/rb/vectra.svg)](https://badge.fury.io/rb/vectra)
+[![Gem Version](https://badge.fury.io/rb/vectra-client.svg)](https://rubygems.org/gems/vectra-client)
 [![CI](https://github.com/stokry/vectra/actions/workflows/ci.yml/badge.svg)](https://github.com/stokry/vectra/actions)
 [![codecov](https://codecov.io/gh/stokry/vectra/branch/main/graph/badge.svg)](https://codecov.io/gh/stokry/vectra)
-[![Ruby Style Guide](https://img.shields.io/badge/code_style-rubocop-brightgreen.svg)](https://github.com/rubocop/rubocop)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
-[![Contributor Covenant](https://img.shields.io/badge/Contributor%20Covenant-2.0-4baaaa.svg)](CODE_OF_CONDUCT.md)
 
-**Vectra** is a unified Ruby client for vector databases. Write once, switch providers seamlessly.
+**A unified Ruby client for vector databases.** Write once, switch providers seamlessly.
 
-## Features
-
-- 🔌 **Unified API** - One interface for multiple vector databases
-- 🚀 **Modern Ruby** - Built for Ruby 3.2+ with modern patterns
-- 🔄 **Automatic Retries** - Built-in retry logic with exponential backoff
-- 📊 **Rich Results** - Enumerable query results with filtering capabilities
-- 🛡️ **Type Safety** - Comprehensive validation and meaningful errors
-- 📝 **Well Documented** - Extensive YARD documentation
+📖 **Documentation:** [vectra-docs.netlify.app](https://vectra-docs.netlify.app/)
 
 ## Supported Providers
 
-| Provider | Status | Version |
-|----------|--------|---------|
-| [Pinecone](https://pinecone.io) | ✅ Fully Supported | v0.1.0 |
-| [PostgreSQL + pgvector](https://github.com/pgvector/pgvector) | ✅ Fully Supported | v0.1.1 |
-| [Qdrant](https://qdrant.tech) | ✅ Fully Supported | v0.2.1 |
-| [Weaviate](https://weaviate.io) | 🚧 Planned | v0.3.0 |
+| Provider | Type | Status |
+|----------|------|--------|
+| **Pinecone** | Managed Cloud | ✅ Supported |
+| **Qdrant** | Open Source | ✅ Supported |
+| **Weaviate** | Open Source | ✅ Supported |
+| **pgvector** | PostgreSQL | ✅ Supported |
 
 ## Installation
 
-Add this line to your application's Gemfile:
-
 ```ruby
-gem 'vectra'
+gem 'vectra-client'
 ```
 
-And then execute:
-
 ```bash
 bundle install
 ```
 
-Or install it yourself:
-
-```bash
-gem install vectra
-```
-
-### Provider-Specific Dependencies
-
-For **pgvector** support, add the `pg` gem:
-
-```ruby
-gem 'pg', '~> 1.5'
-```
-
 ## Quick Start
 
-### Configuration
-
 ```ruby
 require 'vectra'
 
-# Global configuration
-Vectra.configure do |config|
-  config.provider = :pinecone
-  config.api_key = ENV['PINECONE_API_KEY']
-  config.environment = 'us-east-1'  # or config.host = 'your-index-host.pinecone.io'
-end
-
-# Create a client
-client = Vectra::Client.new
-```
-
-Or use per-client configuration:
-
-```ruby
-# Shortcut for Pinecone
-client = Vectra.pinecone(
-  api_key: ENV['PINECONE_API_KEY'],
-  environment: 'us-east-1'
-)
-
-# Shortcut for pgvector (PostgreSQL)
-client = Vectra.pgvector(
-  connection_url: 'postgres://user:password@localhost/mydb'
-)
-
-# Shortcut for Qdrant
-client = Vectra.qdrant(
-  host: 'http://localhost:6333',        # Local Qdrant
-  api_key: ENV['QDRANT_API_KEY']        # Optional for local instances
-)
-
-# Generic client with options
+# Initialize client (works with any provider)
 client = Vectra::Client.new(
   provider: :pinecone,
   api_key: ENV['PINECONE_API_KEY'],
-  environment: 'us-east-1',
-  timeout: 60,
-  max_retries: 5
+  environment: 'us-west-4'
 )
-```
 
-### Basic Operations
-
-#### Upsert Vectors
-
-```ruby
+# Upsert vectors
 client.upsert(
-  index: 'my-index',
   vectors: [
-    { id: 'vec1', values: [0.1, 0.2, 0.3], metadata: { text: 'Hello world' } },
-    { id: 'vec2', values: [0.4, 0.5, 0.6], metadata: { text: 'Ruby is great' } }
+    { id: 'doc-1', values: [0.1, 0.2, 0.3], metadata: { title: 'Hello' } },
+    { id: 'doc-2', values: [0.4, 0.5, 0.6], metadata: { title: 'World' } }
   ]
 )
-# => { upserted_count: 2 }
-```
-
-#### Query Vectors
-
-```ruby
-results = client.query(
-  index: 'my-index',
-  vector: [0.1, 0.2, 0.3],
-  top_k: 5,
-  include_metadata: true
-)
-
-# Iterate over results
-results.each do |match|
-  puts "ID: #{match.id}, Score: #{match.score}"
-  puts "Metadata: #{match.metadata}"
-end
-
-# Access specific results
-results.first          # First match
-results.ids            # All matching IDs
-results.scores         # All scores
-results.max_score      # Highest score
 
-# Filter by score
-high_quality = results.above_score(0.8)
-```
-
-#### Query with Filters
+# Search
+results = client.query(vector: [0.1, 0.2, 0.3], top_k: 5)
+results.each { |match| puts "#{match.id}: #{match.score}" }
 
-```ruby
-results = client.query(
-  index: 'my-index',
-  vector: [0.1, 0.2, 0.3],
-  top_k: 10,
-  filter: { category: 'programming', language: 'ruby' }
-)
+# Delete
+client.delete(ids: ['doc-1', 'doc-2'])
 ```
 
-#### Fetch Vectors by ID
+## Provider Examples
 
 ```ruby
-vectors = client.fetch(index: 'my-index', ids: ['vec1', 'vec2'])
+# Pinecone
+client = Vectra.pinecone(api_key: ENV['PINECONE_API_KEY'], environment: 'us-west-4')
 
-vectors['vec1'].values    # [0.1, 0.2, 0.3]
-vectors['vec1'].metadata  # { 'text' => 'Hello world' }
-```
+# Qdrant (local)
+client = Vectra.qdrant(host: 'http://localhost:6333')
 
-#### Update Vector Metadata
+# Qdrant (cloud)
+client = Vectra.qdrant(host: 'https://your-cluster.qdrant.io', api_key: ENV['QDRANT_API_KEY'])
 
-```ruby
-client.update(
-  index: 'my-index',
-  id: 'vec1',
-  metadata: { category: 'updated', processed: true }
-)
-```
+# Weaviate
+client = Vectra.weaviate(host: 'http://localhost:8080', api_key: ENV['WEAVIATE_API_KEY'])
 
-#### Delete Vectors
-
-```ruby
-# Delete by IDs
-client.delete(index: 'my-index', ids: ['vec1', 'vec2'])
-
-# Delete by filter
-client.delete(index: 'my-index', filter: { category: 'old' })
-
-# Delete all (use with caution!)
-client.delete(index: 'my-index', delete_all: true)
-```
-
-### Working with Vectors
-
-```ruby
-# Create a Vector object
-vector = Vectra::Vector.new(
-  id: 'my-vector',
-  values: [0.1, 0.2, 0.3],
-  metadata: { text: 'Example' }
-)
-
-vector.dimension        # => 3
-vector.metadata?        # => true
-vector.to_h             # Convert to hash
-
-# Calculate similarity
-other = Vectra::Vector.new(id: 'other', values: [0.1, 0.2, 0.3])
-vector.cosine_similarity(other)    # => 1.0 (identical)
-vector.euclidean_distance(other)   # => 0.0
-```
-
-### Index Management
-
-```ruby
-# List all indexes
-indexes = client.list_indexes
-indexes.each { |idx| puts idx[:name] }
-
-# Describe an index
-info = client.describe_index(index: 'my-index')
-puts info[:dimension]  # => 384
-puts info[:metric]     # => "cosine"
-
-# Get index statistics
-stats = client.stats(index: 'my-index')
-puts stats[:total_vector_count]
+# pgvector (PostgreSQL)
+client = Vectra.pgvector(connection_url: 'postgres://user:pass@localhost/mydb')
 ```
 
-### Namespaces
-
-Namespaces allow you to partition vectors within an index:
-
-```ruby
-# Upsert to a namespace
-client.upsert(
-  index: 'my-index',
-  namespace: 'production',
-  vectors: [...]
-)
-
-# Query within a namespace
-client.query(
-  index: 'my-index',
-  namespace: 'production',
-  vector: [0.1, 0.2, 0.3],
-  top_k: 5
-)
-```
-
-### pgvector (PostgreSQL)
-
-pgvector uses PostgreSQL tables as indexes. Each "index" is a table with a vector column.
-
-#### Setup PostgreSQL with pgvector
-
-```bash
-# Using Docker
-docker run -d --name pgvector \
-  -e POSTGRES_PASSWORD=password \
-  -p 5432:5432 \
-  pgvector/pgvector:pg16
-```
-
-#### Create an Index (Table)
-
-```ruby
-client = Vectra.pgvector(connection_url: 'postgres://postgres:password@localhost/postgres')
-
-# Create a new index with cosine similarity
-client.provider.create_index(
-  name: 'documents',
-  dimension: 384,
-  metric: 'cosine'  # or 'euclidean', 'inner_product'
-)
-```
-
-#### Supported Metrics
-
-| Metric | Description | pgvector Operator |
-|--------|-------------|-------------------|
-| `cosine` | Cosine similarity (default) | `<=>` |
-| `euclidean` | Euclidean distance | `<->` |
-| `inner_product` | Inner product / dot product | `<#>` |
-
-#### Table Structure
-
-Vectra creates tables with the following structure:
-
-```sql
-CREATE TABLE documents (
-  id TEXT PRIMARY KEY,
-  embedding vector(384),
-  metadata JSONB DEFAULT '{}',
-  namespace TEXT DEFAULT '',
-  created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
-);
-
--- IVFFlat index for fast similarity search
-CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);
-```
-
-## Configuration Options
-
-| Option | Description | Default |
-|--------|-------------|---------|
-| `provider` | Vector database provider (`:pinecone`, `:pgvector`, `:qdrant`, `:weaviate`) | Required |
-| `api_key` | API key for authentication (password for pgvector) | Required* |
-| `environment` | Environment/region (Pinecone) | - |
-| `host` | Direct host URL or PostgreSQL connection URL | - |
-| `timeout` | Request timeout in seconds | 30 |
-| `open_timeout` | Connection timeout in seconds | 10 |
-| `max_retries` | Maximum retry attempts | 3 |
-| `retry_delay` | Initial retry delay in seconds | 1 |
-| `logger` | Logger instance for debugging | nil |
-
-*For pgvector, `api_key` is used as the PostgreSQL password.
-
-## Error Handling
-
-Vectra provides specific error classes for different failure scenarios:
-
-```ruby
-begin
-  client.query(index: 'my-index', vector: [0.1, 0.2], top_k: 5)
-rescue Vectra::AuthenticationError => e
-  puts "Authentication failed: #{e.message}"
-rescue Vectra::RateLimitError => e
-  puts "Rate limited. Retry after #{e.retry_after} seconds"
-rescue Vectra::NotFoundError => e
-  puts "Resource not found: #{e.message}"
-rescue Vectra::ValidationError => e
-  puts "Invalid request: #{e.message}"
-rescue Vectra::ServerError => e
-  puts "Server error (#{e.status_code}): #{e.message}"
-rescue Vectra::Error => e
-  puts "General error: #{e.message}"
-end
-```
+## Features
 
-## Logging
+- **Provider Agnostic** - Switch providers with one line change
+- **Production Ready** - Ruby 3.2+, 95%+ test coverage
+- **Resilient** - Retry logic with exponential backoff
+- **Observable** - Datadog & New Relic instrumentation
+- **Rails Ready** - ActiveRecord integration with `has_vector` DSL
 
-Enable debug logging to see request details:
+## Rails Integration
 
 ```ruby
-require 'logger'
+class Document < ApplicationRecord
+  include Vectra::ActiveRecord
 
-Vectra.configure do |config|
-  config.provider = :pinecone
-  config.api_key = ENV['PINECONE_API_KEY']
-  config.environment = 'us-east-1'
-  config.logger = Logger.new($stdout)
+  has_vector :embedding,
+    provider: :qdrant,
+    index: 'documents',
+    dimension: 1536
 end
-```
-
-## Best Practices
-
-### Batch Upserts
-
-For large datasets, batch your upserts:
-
-```ruby
-vectors = large_dataset.each_slice(100).map do |batch|
-  client.upsert(index: 'my-index', vectors: batch)
-end
-```
-
-### Connection Reuse
 
-Create a single client instance and reuse it:
+# Auto-indexes on save
+doc = Document.create!(title: 'Hello', embedding: [0.1, 0.2, ...])
 
-```ruby
-# Good: Reuse the client
-client = Vectra::Client.new(...)
-client.query(...)
-client.upsert(...)
-
-# Avoid: Creating new clients for each operation
-Vectra::Client.new(...).query(...)
-Vectra::Client.new(...).upsert(...)
-```
-
-### Error Recovery
-
-Implement retry logic for transient failures:
-
-```ruby
-def query_with_retry(client, **params, retries: 3)
-  client.query(**params)
-rescue Vectra::RateLimitError => e
-  if retries > 0
-    sleep(e.retry_after || 1)
-    retry(retries: retries - 1)
-  else
-    raise
-  end
-end
+# Search
+Document.vector_search(embedding: query_vector, limit: 10)
 ```
 
 ## Development
 
-After checking out the repo:
-
 ```bash
-# Install dependencies
+git clone https://github.com/stokry/vectra.git
+cd vectra
 bundle install
-
-# Run tests
 bundle exec rspec
-
-# Run linter
 bundle exec rubocop
-
-# Generate documentation
-bundle exec rake docs
 ```
 
-## Roadmap
-
-### v0.1.0
-- ✅ Pinecone provider
-- ✅ Basic CRUD operations
-- ✅ Configuration system
-- ✅ Error handling with retries
-- ✅ Comprehensive tests
-
-### v0.1.1 (Current)
-- ✅ pgvector (PostgreSQL) provider
-- ✅ Multiple similarity metrics (cosine, euclidean, inner product)
-- ✅ Namespace support for pgvector
-- ✅ IVFFlat index creation
-
-### v0.2.1
-- ✅ Qdrant provider (fully implemented)
-- ✅ Enhanced error handling
-- ✅ Improved retry middleware
-
-### v0.3.0
-- 🚧 Weaviate provider
-- 🚧 Batch operations
-- 🚧 Performance optimizations
-
-### v1.0.0
-- 🚧 Rails integration
-- 🚧 ActiveRecord-like DSL
-- 🚧 Background job support
-- 🚧 Full documentation
-
 ## Contributing
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/stokry/vectra.
-
-1. Fork it
-2. Create your feature branch (`git checkout -b feature/my-new-feature`)
-3. Commit your changes (`git commit -am 'Add some feature'`)
-4. Push to the branch (`git push origin feature/my-new-feature`)
-5. Create a new Pull Request
+Bug reports and pull requests welcome at [github.com/stokry/vectra](https://github.com/stokry/vectra).
 
 ## License
 
-The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
-
-## Acknowledgments
-
-Inspired by the simplicity of Ruby database gems and the need for a unified vector database interface.
+MIT License - see [LICENSE](LICENSE) file.

data/docs/Gemfile

@@ -0,0 +1,9 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gem "jekyll", "~> 4.4"
+gem "jekyll-feed", "~> 0.12"
+gem "jekyll-seo-tag"
+gem "jekyll-sitemap"
+gem "webrick", "~> 1.8"

data/docs/_config.yml

@@ -0,0 +1,37 @@
+title: Vectra Documentation
+description: A unified Ruby client for vector databases. Build AI-powered search, RAG applications, and recommendation systems.
+url: "https://vectra-docs.netlify.app"
+baseurl: ""
+
+plugins:
+  - jekyll-feed
+  - jekyll-sitemap
+  - jekyll-seo-tag
+
+markdown: kramdown
+highlighter: rouge
+
+kramdown:
+  syntax_highlighter: rouge
+  syntax_highlighter_opts:
+    block:
+      line_numbers: false
+
+exclude:
+  - Gemfile
+  - Gemfile.lock
+  - node_modules
+  - vendor/bundle/
+  - vendor/cache/
+  - vendor/gems/
+  - vendor/ruby/
+  - _site/
+
+permalink: /:categories/:title/
+
+defaults:
+  - scope:
+      path: ""
+      type: "pages"
+    values:
+      layout: "page"

data/docs/_layouts/default.html

@@ -0,0 +1,14 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+  <meta charset="utf-8">
+  <meta name="viewport" content="width=device-width, initial-scale=1">
+  <meta name="description" content="{{ page.description | default: site.description }}">
+  <title>{{ page.title }} - {{ site.title }}</title>
+  <link rel="stylesheet" href="{{ site.baseurl }}/assets/style.css">
+  <link rel="icon" type="image/svg+xml" href="data:image/svg+xml,<svg xmlns=%22http://www.w3.org/2000/svg%22 viewBox=%220 0 100 100%22><text y=%22.9em%22 font-size=%2290%22>◆</text></svg>">
+</head>
+<body>
+  {{ content }}
+</body>
+</html>