vectra-client 1.0.8 → 1.1.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '05778206f66d4e2ead830f5b7c5885f6336b46d36af696f3df5a56eb26163c5f'
-  data.tar.gz: 1ae5814cd10df006ecf5568f34e457400b072ecc8a06dbf85a4e4f5b94120fb9
+  metadata.gz: 70730299dab8475b05688f7017dccd1965b87a49589c96be7633a356af8d57f2
+  data.tar.gz: 14999ebde62586578b444ba45d396cb38f1a4dd5dcbf2fd126e042ebb1c6fae5
 SHA512:
-  metadata.gz: d25889a4626c9caa9edf7ccc285c849799de87cf38a8a7a6f56a49edb46da592b28bc0232960e2612fd0d9199e990c21bd5fe6b88f15351d8ff38c73bd7afc84
-  data.tar.gz: d4ef355089814ed4caf8c61e216b9554f23035df12be72a65c778b3230bd2a32f16ecf4c50de3bc33e40dc323f1aee83d7a891ad962bec105585c8bc6b30902b
+  metadata.gz: 69c8fa722ee4abfe3ddf6b19f8bd12f46d09edefee50612021142c3951600666ed854018e8a5c1bc33249895fec42ae18ab1d821beffa989d2700da99c28bcf4
+  data.tar.gz: f27a0df4bbcf618659297b1376ebe977588d32d5dfef76fe7c8f5f38c8780b37e055a0e8d91f6bc3ae8eb7cad88606c760d4ed5882c61756ac1495dedfc339a5

data/README.md

@@ -242,6 +242,59 @@ Real-world patterns for common use cases:
 
 👉 **[Browse all Recipes & Patterns](https://vectra-docs.netlify.app/guides/recipes/)**
 
+## Middleware System
+
+Vectra features a powerful **Rack-style middleware system** that allows you to extend functionality without modifying core code.
+
+### Quick Start
+
+```ruby
+# Global middleware (applies to all clients)
+Vectra::Client.use Vectra::Middleware::Logging
+Vectra::Client.use Vectra::Middleware::Retry, max_attempts: 5
+Vectra::Client.use Vectra::Middleware::CostTracker
+
+# Per-client middleware
+client = Vectra::Client.new(
+  provider: :qdrant,
+  middleware: [
+    Vectra::Middleware::PIIRedaction,
+    Vectra::Middleware::Instrumentation
+  ]
+)
+```
+
+### Built-in Middleware
+
+- **Logging** - Request/response logging with timing
+- **Retry** - Automatic retries with exponential backoff
+- **Instrumentation** - Metrics and monitoring integration
+- **PIIRedaction** - Automatic PII (email, phone, SSN) redaction
+- **CostTracker** - Track API usage costs per operation
+
+### Custom Middleware
+
+```ruby
+class MyAuditMiddleware < Vectra::Middleware::Base
+  def before(request)
+    # Called before the operation
+    AuditLog.create!(action: request.operation, user: Current.user)
+  end
+
+  def after(request, response)
+    # Called after successful operation
+    puts "Duration: #{response.metadata[:duration_ms]}ms"
+  end
+
+  def on_error(request, error)
+    # Called when an error occurs
+    ErrorTracker.notify(error, context: { operation: request.operation })
+  end
+end
+
+Vectra::Client.use MyAuditMiddleware
+```
+
 ## Production Patterns
 
 Vectra includes 7 production-ready patterns out of the box:

data/docs/_layouts/home.html

@@ -55,7 +55,7 @@
   <!-- Hero Section -->
   <section class="tma-hero">
     <div class="tma-hero__container">
-      <span class="tma-hero__badge">v1.0.1 — Hybrid Search & Rails Generator</span>
+      <span class="tma-hero__badge">v1.1.0 — Hybrid Search, Rails Generator & Middleware</span>
       <h1 class="tma-hero__title">
         Vector Databases,<br>
         <span class="tma-hero__title-gradient">Unified for Ruby.</span>

data/docs/api/overview.md

@@ -216,3 +216,9 @@ end
 ```
 
 See [Complete API Methods Reference]({{ site.baseurl }}/api/methods/) for detailed method documentation.
+
+## Next Steps
+
+- [Providers Guide]({{ site.baseurl }}/providers/)
+- [Rails Integration Guide]({{ site.baseurl }}/guides/rails-integration/)
+- [Middleware System]({{ site.baseurl }}/guides/middleware/)

data/docs/guides/middleware.md

@@ -0,0 +1,324 @@
+---
+layout: page
+title: Middleware System
+permalink: /guides/middleware/
+---
+
+# Middleware System
+
+Vectra includes a **Rack-style middleware stack** that lets you extend the client
+without forking the gem or patching providers.
+
+You can:
+
+- Add **logging, metrics, retries, PII redaction, cost tracking**.
+- Inject **custom behaviour** before/after every operation.
+- Enable features **globally** or **per client**, same kao Faraday/Sidekiq.
+
+---
+
+## Core Concepts
+
+### Request / Response
+
+- **`Vectra::Middleware::Request`**
+  - `operation` – npr. `:upsert`, `:query`, `:fetch`, `:delete`, `:stats`, `:hybrid_search`
+  - `index` – ime indeksa (može biti `nil` ako koristiš default)
+  - `namespace` – namespace (može biti `nil`)
+  - `params` – originalni keyword parametri koje je `Vectra::Client` proslijedio
+  - `provider` – ime providera (`:pinecone`, `:qdrant`, `:pgvector`, `:memory`, …)
+  - helperi: `write_operation?`, `read_operation?`
+
+- **`Vectra::Middleware::Response`**
+  - `result` – što god provider vrati (npr. hash, `QueryResult`, itd.)
+  - `error` – iznimka ako je došlo do greške
+  - `metadata` – slobodan hash za dodatne informacije (trajanje, cost, retry_count…)
+  - helperi: `success?`, `failure?`, `raise_if_error!`, `value!`
+
+### Base i Stack
+
+- **`Vectra::Middleware::Base`**
+  - Hookovi koje možeš override-ati:
+    - `before(request)` – prije poziva providera / sljedećeg middlewarea
+    - `after(request, response)` – nakon uspješnog poziva
+    - `on_error(request, error)` – kad dođe do iznimke (error se zatim re-raise-a)
+
+- **`Vectra::Middleware::Stack`**
+  - Gradi chain oko konkretnog providera:
+    ```ruby
+    stack = Vectra::Middleware::Stack.new(provider, [Logging.new, Retry.new])
+    result = stack.call(:upsert, index: "docs", vectors: [...], provider: :qdrant)
+    ```
+  - `Stack` interno:
+    - kreira `Request`,
+    - kroz sve middlewares propagira isti `Request`,
+    - na kraju zove `provider.public_send(request.operation, **provider_params)`,
+    - vraća `Response` (s `result` ili `error`).
+
+---
+
+## Enabling Middleware
+
+### Global Middleware
+
+Primjenjuje se na **sve** `Vectra::Client` instance.
+
+```ruby
+require "vectra"
+
+# Global logging + retry + cost tracking
+Vectra::Client.use Vectra::Middleware::Logging
+Vectra::Client.use Vectra::Middleware::Retry, max_attempts: 5
+Vectra::Client.use Vectra::Middleware::CostTracker
+```
+
+Sve sljedeće `Vectra::Client.new(...)` instance će koristiti ovaj globalni stack.
+
+### Per‑Client Middleware
+
+Dodatni ili prilagođeni middleware samo za jedan klijent:
+
+```ruby
+client = Vectra::Client.new(
+  provider: :qdrant,
+  index: "products",
+  middleware: [
+    Vectra::Middleware::PIIRedaction,
+    Vectra::Middleware::Instrumentation
+  ]
+)
+```
+
+- Per‑client middleware se izvodi **nakon** globalnog, u istom chainu.
+- Redoslijed u arrayu definira redoslijed ekzekucije (zadnji je najunutarnji, tik do providera).
+
+---
+
+## Koje operacije prolaze kroz stack?
+
+Sve standardne operacije `Vectra::Client`a koriste middleware stack:
+
+- `upsert`
+- `query`
+- `fetch`
+- `update`
+- `delete`
+- `list_indexes`
+- `describe_index`
+- `stats`
+- `hybrid_search`
+
+To znači da middleware može:
+
+- logirati / instrumentirati **sve pozive** prema provideru,
+- raditi **PII redakciju** na `upsert` zahtjevima,
+- brojati i retry-ati i **read** i **write** operacije,
+- računati trošak po operaciji (npr. za billing / budžete).
+
+---
+
+## Built‑in Middleware
+
+### Logging (`Vectra::Middleware::Logging`)
+
+**Što radi:**
+- logira početak i kraj svake operacije (`operation`, `index`, `namespace`),
+- mjeri trajanje i sprema ga u `response.metadata[:duration_ms]`.
+
+**Konfiguracija:**
+
+```ruby
+# Globalno
+Vectra::Client.use Vectra::Middleware::Logging
+
+# S custom loggerom
+logger = Logger.new($stdout)
+Vectra::Client.use Vectra::Middleware::Logging, logger: logger
+```
+
+**Tipična upotreba:** debugiranje, audit logovi, korelacija s HTTP logovima.
+
+---
+
+### Retry (`Vectra::Middleware::Retry`)
+
+**Što radi:**
+- automatski retry-a transient greške:
+  - `Vectra::RateLimitError`
+  - `Vectra::ConnectionError`
+  - `Vectra::TimeoutError`
+  - `Vectra::ServerError`
+- koristi exponential ili linear backoff,
+- upisuje broj retry-a u `response.metadata[:retry_count]`.
+
+**Konfiguracija:**
+
+```ruby
+# 3 pokušaja, exponential backoff (default)
+Vectra::Client.use Vectra::Middleware::Retry
+
+# 5 pokušaja, linearni backoff
+Vectra::Client.use Vectra::Middleware::Retry,
+  max_attempts: 5,
+  backoff: :linear
+
+# Fiksni delay 1.0s
+Vectra::Client.use Vectra::Middleware::Retry,
+  max_attempts: 3,
+  backoff: 1.0
+```
+
+**Tipična upotreba:** zaštita od povremenih mrežnih problema i rate‑limit grešaka.
+
+---
+
+### Instrumentation (`Vectra::Middleware::Instrumentation`)
+
+**Što radi:**
+- emitira događaje preko postojećeg `Vectra::Instrumentation` sustava,
+- prikuplja trajanje, status, error class, dodatni `metadata`.
+
+**Primjer:**
+
+```ruby
+Vectra::Client.use Vectra::Middleware::Instrumentation
+
+Vectra.on_operation do |event|
+  # event[:operation], event[:provider], event[:duration_ms], event[:success], ...
+  StatsD.timing("vectra.#{event[:operation]}", event[:duration_ms])
+end
+```
+
+**Tipična upotreba:** integracija s Prometheus/Grafana, Datadog, New Relic…
+
+---
+
+### PII Redaction (`Vectra::Middleware::PIIRedaction`)
+
+**Što radi:**
+- prije `upsert` operacija prolazi kroz `vectors[:metadata]`,
+- prepoznaje PII pattern-e (email, phone, SSN, credit card) i zamjenjuje ih placeholderom
+  npr. `[REDACTED_EMAIL]`, `[REDACTED_PHONE]`, itd.
+
+**Primjer:**
+
+```ruby
+Vectra::Client.use Vectra::Middleware::PIIRedaction
+
+client.upsert(
+  index: "sensitive",
+  vectors: [
+    {
+      id: "user-1",
+      values: [0.1, 0.2, 0.3],
+      metadata: {
+        email: "user@example.com",
+        phone: "555-1234",
+        note:  "Contact at user@example.com"
+      }
+    }
+  ]
+)
+```
+
+Nakon `upsert`‑a, provider će vidjeti već **redaktirani** metadata.
+
+**Custom patterni:**
+
+```ruby
+patterns = {
+  credit_card: /\b\d{4}[-\s]?\d{4}[-\s]?\d{4}[-\s]?\d{4}\b/,
+  api_key: /sk-[a-zA-Z0-9]{32}/
+}
+
+Vectra::Client.use Vectra::Middleware::PIIRedaction, patterns: patterns
+```
+
+**Tipična upotreba:** GDPR, SOC2, PCI‑DSS okruženja gdje je zabranjen PII u vektor bazi.
+
+---
+
+### CostTracker (`Vectra::Middleware::CostTracker`)
+
+**Što radi:**
+- procjenjuje trošak po operaciji na temelju providera i tipa operacije (`read` / `write`),
+- upisuje trošak u `response.metadata[:cost_usd]`,
+- opcionalno zove `on_cost` callback za real‑time praćenje.
+
+**Primjer:**
+
+```ruby
+Vectra::Client.use Vectra::Middleware::CostTracker,
+  on_cost: ->(event) {
+    puts "💰 Cost: $#{event[:cost_usd].round(6)} for #{event[:operation]} (#{event[:provider]})"
+  }
+```
+
+**Custom pricing:**
+
+```ruby
+pricing = {
+  pinecone: { read: 0.0001, write: 0.0002 },
+  qdrant:   { read: 0.00005, write: 0.0001 }
+}
+
+Vectra::Client.use Vectra::Middleware::CostTracker, pricing: pricing
+```
+
+**Tipična upotreba:** unutarnji billing, budget guardrails, cost dashboards.
+
+---
+
+## Custom Middleware
+
+Najjednostavniji način je naslijediti `Vectra::Middleware::Base` i override-ati hookove:
+
+```ruby
+class MyAuditMiddleware < Vectra::Middleware::Base
+  def before(request)
+    AuditLog.create!(
+      operation: request.operation,
+      index:     request.index,
+      namespace: request.namespace,
+      provider:  request.provider
+    )
+  end
+
+  def after(_request, response)
+    puts "Duration: #{response.metadata[:duration_ms]}ms"
+  end
+
+  def on_error(request, error)
+    ErrorTracker.notify(error, context: { operation: request.operation })
+  end
+end
+
+Vectra::Client.use MyAuditMiddleware
+```
+
+**Savjeti:**
+
+- Ne mijenjaj strukturu `request.params` na način koji provider ne očekuje.
+- Svoje pomoćne podatke stavljaj u `response.metadata` ili `request.metadata`.
+- Ako hvataš greške u `on_error`, **nemoj ih gutati** – middleware stack će ih ponovo baciti.
+
+---
+
+## Primjer: `examples/middleware_demo.rb`
+
+U repozitoriju imaš kompletan demo:
+
+- konfigurira globalni stack (`Logging`, `Retry`, `CostTracker`),
+- pokazuje per‑client PII redaction,
+- definira custom `TimingMiddleware`,
+- demonstrira kako izgleda output u konzoli.
+
+Pokretanje:
+
+```bash
+bundle exec ruby examples/middleware_demo.rb
+```
+
+Ovaj demo je dobar “živi” primjer kako kombinirati više middleware-a u praksi.
+
+

data/examples/middleware_demo.rb

@@ -0,0 +1,103 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Middleware System Demo
+#
+# This script demonstrates the new middleware system in Vectra.
+# Run with: ruby examples/middleware_demo.rb
+
+require_relative "../lib/vectra"
+
+# Configure global middleware
+puts "🎯 Configuring global middleware..."
+Vectra::Client.use Vectra::Middleware::Logging
+Vectra::Client.use Vectra::Middleware::Retry, max_attempts: 3
+Vectra::Client.use Vectra::Middleware::CostTracker, on_cost: ->(event) {
+  puts "💰 Cost: $#{event[:cost_usd].round(6)} for #{event[:operation]}"
+}
+
+# Create client
+puts "\n📦 Creating client with Memory provider..."
+client = Vectra::Client.new(
+  provider: :memory,
+  index: "demo"
+)
+
+# Example 1: Upsert with middleware
+puts "\n🔄 Example 1: Upsert with middleware stack"
+puts "=" * 50
+client.upsert(
+  index: "demo",
+  vectors: [
+    { id: "doc-1", values: [0.1, 0.2, 0.3], metadata: { title: "Ruby" } },
+    { id: "doc-2", values: [0.4, 0.5, 0.6], metadata: { title: "Python" } }
+  ]
+)
+
+# Example 2: Query with middleware
+puts "\n🔍 Example 2: Query with middleware stack"
+puts "=" * 50
+results = client.query(
+  index: "demo",
+  vector: [0.1, 0.2, 0.3],
+  top_k: 2
+)
+puts "Found #{results.size} results"
+
+# Example 3: Per-client middleware
+puts "\n🎨 Example 3: Per-client middleware (PII Redaction)"
+puts "=" * 50
+pii_client = Vectra::Client.new(
+  provider: :memory,
+  index: "sensitive",
+  middleware: [Vectra::Middleware::PIIRedaction]
+)
+
+pii_client.upsert(
+  index: "sensitive",
+  vectors: [
+    {
+      id: "user-1",
+      values: [0.1, 0.2, 0.3],
+      metadata: {
+        email: "user@example.com",
+        phone: "555-1234",
+        note: "Contact at user@example.com"
+      }
+    }
+  ]
+)
+
+# Fetch to see redacted data
+fetched = pii_client.fetch(index: "sensitive", ids: ["user-1"])
+puts "Original email: user@example.com"
+puts "Redacted: #{fetched["user-1"].metadata[:email]}"
+puts "Redacted note: #{fetched["user-1"].metadata[:note]}"
+
+# Example 4: Custom middleware
+puts "\n🛠️  Example 4: Custom middleware"
+puts "=" * 50
+
+class TimingMiddleware < Vectra::Middleware::Base
+  def before(request)
+    puts "⏱️  Starting #{request.operation}..."
+  end
+
+  def after(request, response)
+    duration = response.metadata[:duration_ms] || 0
+    puts "✅ Completed in #{duration.round(2)}ms"
+  end
+end
+
+custom_client = Vectra::Client.new(
+  provider: :memory,
+  index: "custom",
+  middleware: [TimingMiddleware]
+)
+
+custom_client.upsert(
+  index: "custom",
+  vectors: [{ id: "test", values: [1, 2, 3] }]
+)
+
+puts "\n✨ Demo complete!"