rubyllm-semantic_router 0.1.0 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 5b01b92cdb4fa6f65278ec83440543f95642cf85026ea142cb2564c326c0512c
-  data.tar.gz: c1670b2f6379c451e711950f8d50982ec01671a1070ade3805f4fab5b557108a
+  metadata.gz: 4aecffcf2d9ae00e08f10c2f606ca51ef8d3a7fd0fc7616b6e98c0826d85649c
+  data.tar.gz: 2b50357814fa44df4aad83c63de2418e0ac78581de10eb3ccd975a91b4e76c86
 SHA512:
-  metadata.gz: f06a9a3b105253c45af254f3fdb98346ed28af13a73c36abd0f6eb57709de9b47cdec4a8dd24d214abac004da547124b4468e98b8e6b9d574c0e81583b978489
-  data.tar.gz: 9e092a880b2c627850e50ce2f376fba33bd88ae7de824ab1bf50e32d88f416f74128bd9920a456bbbf394f26f6eba0ac74d2bdd8a43019d7bdd7060e41ad43a4
+  metadata.gz: d9dcf6225321e5dde48d54fec9df1c8ca64e111ec948ed35ede94434f1a27bbcaec39b73c22c2d8b078926ae1f1171d1538f0d83133aa5e0febef5db032784d1
+  data.tar.gz: 0d3620664177877adf2518ffd22766d6a831e14afde7501a7864c53cd219dc6b2b4a525a7d5939c3a7ff9a961eb0fe29cfd5f0db52d3b88619ec8b0a97dc4cd9

data/.gitignore

@@ -19,3 +19,7 @@ vendor/
 
 # RSpec
 .rspec_status
+*.gem
+
+# Test Rails app
+/test_app/

data/ARCHITECTURE.md

@@ -0,0 +1,329 @@
+# Architecture
+
+This document describes the architecture and design decisions of RubyLLM Semantic Router.
+
+## Overview
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│                         Router                              │
+│  - Manages agents and conversation state                    │
+│  - Delegates routing to strategy                            │
+│  - Handles agent switching and chat                         │
+│  - Provides ask, ask_batch, match, debug_routing APIs       │
+└─────────────────────────────────────────────────────────────┘
+           │                    │                    │
+           │                    │                    │
+           ▼                    ▼                    ▼
+┌─────────────────┐  ┌─────────────────┐  ┌─────────────────┐
+│ EmbeddingCache  │  │     Logger      │  │  Retry Logic    │
+│ (optional TTL)  │  │   (optional)    │  │ (exp. backoff)  │
+└─────────────────┘  └─────────────────┘  └─────────────────┘
+                              │
+                              ▼
+┌─────────────────────────────────────────────────────────────┐
+│                    Strategies::Semantic                     │
+│  - Generates embeddings via RubyLLM                         │
+│  - Finds nearest neighbors (kNN)                            │
+│  - Returns RoutingDecision                                  │
+└─────────────────────────────────────────────────────────────┘
+                              │
+                              ▼
+┌─────────────────────────────────────────────────────────────┐
+│                     Example Storage                         │
+│  - In-memory arrays                                         │
+│  - ActiveRecord with neighbor gem                           │
+│  - Custom vector databases via find_examples                │
+└─────────────────────────────────────────────────────────────┘
+```
+
+## Core Components
+
+### Router (`lib/rubyllm/semantic_router/router.rb`)
+
+The main entry point that orchestrates routing and agent management.
+
+**Responsibilities:**
+- Accept and normalize agent configurations
+- Store and manage routing examples
+- Delegate routing decisions to the strategy
+- Maintain conversation state and agent switching
+- Provide the `ask`, `ask_batch`, `match`, and `debug_routing` APIs
+- Manage embedding cache (if configured)
+- Handle retry logic for transient failures
+- Emit debug logs (if logger configured)
+
+**Key Design Decisions:**
+- Accepts RubyLLM chat objects directly for ergonomic API
+- Extracts configuration from chat objects (instructions, tools, model)
+- Maintains single chat instance, switching agent config on route changes
+- Preserves full conversation history across agent switches
+
+### Strategy (`lib/rubyllm/semantic_router/strategies/`)
+
+Pluggable routing strategies following the Strategy pattern.
+
+**Base Class:** `Strategies::Base`
+- Defines the `#route` interface
+- Provides shared `apply_fallback` logic
+
+**Semantic Strategy:** `Strategies::Semantic`
+- Generates embeddings using RubyLLM.embed
+- Performs kNN search against examples
+- Supports multiple storage backends via duck typing
+- Returns `RoutingDecision` with agent, confidence, and reason
+
+### RoutingDecision (`lib/rubyllm/semantic_router/routing_decision.rb`)
+
+Value object representing a routing decision.
+
+**Attributes:**
+- `agent` - Target agent name (symbol)
+- `confidence` - Match confidence (0.0-1.0)
+- `matched_example` - The example text that matched
+- `reason` - Why this decision was made (`:semantic_match`, `:fallback`, etc.)
+- `inject_instruction` - Optional instruction for clarification flow
+
+### Configuration (`lib/rubyllm/semantic_router/configuration.rb`)
+
+Global configuration with validation. All setters validate input and raise `ConfigurationError` for invalid values.
+
+**Routing Options:**
+- `default_embedding_model` - Model for generating embeddings
+- `default_similarity_threshold` - Minimum confidence for routing (0.0-1.0)
+- `default_k_neighbors` - Number of neighbors for kNN
+- `default_fallback` - Behavior when no match found
+- `default_max_words` - Message truncation limit
+
+**Reliability Options:**
+- `logger` - Logger instance for debug output (default: nil)
+- `cache_ttl` - Embedding cache TTL in seconds (default: nil = no caching)
+- `max_retries` - Maximum retry attempts for embedding failures (default: 3)
+- `retry_base_delay` - Base delay for exponential backoff in seconds (default: 0.5)
+
+### Utils (`lib/rubyllm/semantic_router/utils.rb`)
+
+Shared utility functions.
+
+- `cosine_distance(a, b)` - Calculate cosine distance between vectors
+- `cosine_similarity(a, b)` - Calculate cosine similarity
+- `truncate_to_max_words(text, max_words)` - Truncate text by word count
+
+### EmbeddingCache (`lib/rubyllm/semantic_router/embedding_cache.rb`)
+
+Thread-safe in-memory cache for embeddings with TTL support.
+
+**Purpose:** Reduce API calls and latency when the same text is embedded multiple times (e.g., adding the same example after clearing, or routing identical messages).
+
+**Features:**
+- TTL-based expiration
+- Thread-safe with Mutex
+- Simple get/set/fetch interface
+- Automatic cleanup of expired entries
+
+```ruby
+# Internal structure
+CacheEntry = Struct.new(:embedding, :expires_at)
+
+# Usage (internal to Router)
+@embedding_cache.fetch(text) { generate_embedding_api_call(text) }
+```
+
+**Note:** The cache is per-router instance. Each router maintains its own cache.
+
+### Errors (`lib/rubyllm/semantic_router/errors.rb`)
+
+Custom exception hierarchy for clear error handling.
+
+```
+Error (base)
+├── AgentNotFoundError
+├── NoDefaultAgentError
+├── NoAgentsError
+├── NoRoutingExamplesError
+├── EmbeddingError
+├── InvalidFallbackError
+├── InvalidAgentError
+└── ConfigurationError
+```
+
+## Storage Backends
+
+The router supports multiple storage backends through duck typing:
+
+### In-Memory
+
+Default storage using simple arrays of `InMemoryExample` structs.
+
+```ruby
+# Internal structure
+InMemoryExample = Struct.new(:agent_name, :example_text, :embedding)
+```
+
+### ActiveRecord with neighbor gem
+
+Expects model with `has_neighbors :embedding` and `agent_name`, `example_text` columns.
+
+```ruby
+# Detection
+examples.respond_to?(:nearest_neighbors)  # Use neighbor gem's kNN
+```
+
+### Custom Vector Database
+
+Accepts a `find_examples` callable for custom search:
+
+```ruby
+find_examples: ->(embedding, limit:) {
+  # Return array of hashes or objects with:
+  # - agent_name (required)
+  # - example_text (optional)
+  # - distance OR score (optional, defaults to 0)
+}
+```
+
+## Routing Flow
+
+1. **Message received** via `router.ask(message)`
+2. **Log** routing attempt (if logger configured)
+3. **Check cache** for existing embedding (if cache enabled)
+4. **Generate embedding** for the message using configured model
+   - On failure: retry with exponential backoff (up to `max_retries`)
+   - Cache result (if cache enabled)
+5. **Find nearest neighbors** from examples (k = `k_neighbors`)
+6. **Calculate confidence** from cosine distance of best match
+7. **Apply threshold** - if below `similarity_threshold`, use fallback
+8. **Log** routing decision (if logger configured)
+9. **Return decision** with target agent and metadata
+10. **Switch agent** if target differs from current
+11. **Send message** to current agent's chat and return response
+
+## Batch Routing Flow
+
+For `router.ask_batch(messages)`:
+
+1. **Generate embeddings** for all messages in single API call
+2. **Route each message** using its pre-computed embedding
+3. **Return array** of `RoutingDecision` objects
+4. **Note:** Does not send messages or switch agents - only returns decisions
+
+## Reliability Features
+
+### Logging
+
+When a logger is configured, the router emits debug information:
+
+```
+[SemanticRouter] Router initialized with agents: product, support
+[SemanticRouter] Routing message: Show me laptops...
+[SemanticRouter] Cache hit for embedding
+[SemanticRouter] Routed to :product (confidence: 0.892, reason: semantic_match)
+[SemanticRouter] Switching from :support to :product
+```
+
+Log levels used:
+- `debug` - Detailed operation info (routing attempts, cache hits, agent switches)
+- `info` - Routing decisions
+- `warn` - Retry attempts
+- `error` - Final failures after retries exhausted
+
+### Retry with Exponential Backoff
+
+Embedding API calls are retried on failure:
+
+```
+Attempt 1: fail → wait 0.5s
+Attempt 2: fail → wait 1.0s
+Attempt 3: fail → wait 2.0s
+Attempt 4: fail → raise EmbeddingError
+```
+
+Formula: `delay = retry_base_delay * (2 ** attempt_number)`
+
+### Embedding Cache
+
+Reduces API calls by caching embeddings:
+
+```ruby
+# First call - generates embedding, stores in cache
+router.add_example("Show products", agent: :product)
+
+# Later - same text uses cached embedding
+router.clear_examples!
+router.add_example("Show products", agent: :product)  # Cache hit
+```
+
+Cache is keyed by the truncated text (after `max_words` applied).
+
+## Design Principles
+
+### 1. Duck Typing for Flexibility
+
+The router uses duck typing to support multiple storage backends without requiring specific interfaces:
+
+```ruby
+# ActiveRecord detection
+if examples.respond_to?(:nearest_neighbors)
+  # Use neighbor gem
+elsif examples.respond_to?(:where)
+  # Use ActiveRecord scoping
+else
+  # Use array operations
+end
+```
+
+### 2. Sensible Defaults
+
+All configuration has reasonable defaults that work out of the box:
+- Embedding model: `text-embedding-3-small` (fast, cheap)
+- Threshold: `0.7` (balanced precision/recall)
+- Fallback: `:default_agent` (predictable behavior)
+
+### 3. Validation at Boundaries
+
+Input validation occurs at configuration and router initialization, failing fast with clear error messages rather than producing undefined behavior during routing.
+
+### 4. Strategy Pattern for Routing
+
+Using the strategy pattern allows for future alternative routing implementations (e.g., keyword-based, LLM-based) without changing the Router class.
+
+### 5. Value Objects for Decisions
+
+`RoutingDecision` is immutable with clear semantics, making it easy to inspect, log, and test routing behavior.
+
+## Extension Points
+
+### Custom Strategies
+
+Create new routing strategies by extending `Strategies::Base`:
+
+```ruby
+class MyStrategy < RubyLLM::SemanticRouter::Strategies::Base
+  def route(message, agents:, examples:, current_agent:, config:, find_examples: nil)
+    # Custom routing logic
+    RoutingDecision.new(agent: :some_agent, confidence: 0.9, reason: :custom)
+  end
+end
+
+router = Router.new(agents: {...}, strategy: MyStrategy.new, ...)
+```
+
+### Callbacks
+
+Register callbacks for routing events:
+
+```ruby
+router.on(:on_route) do |decision|
+  Rails.logger.info("Routed to #{decision.agent} with confidence #{decision.confidence}")
+end
+```
+
+## Performance Considerations
+
+- **Embedding generation** is the main latency source (~50-200ms per call)
+- **In-memory kNN** is O(n) - fine for hundreds of examples
+- **neighbor gem** uses database indexes for O(log n) performance
+- **Batch imports** (`import_examples`) reduce embedding API calls
+- **Batch routing** (`ask_batch`) generates all embeddings in one API call
+- **Embedding cache** eliminates redundant API calls for repeated text
+- **max_words** truncation reduces embedding costs for long messages

data/CHANGELOG.md

@@ -0,0 +1,98 @@
+# Changelog
+
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.4.0] - 2026-05-29
+
+### Changed
+- Verified compatibility with RubyLLM 1.15.0 (tested with real OpenAI API)
+- Lowered default `similarity_threshold` from 0.7 to 0.3 to match real-world embedding similarity ranges
+
+### Added
+- Integration test suite (`spec/integration/`) for testing against real OpenAI API
+
+## [0.3.0] - 2025-01-24
+
+### Added
+- Shared `Utils` module with `cosine_distance`, `cosine_similarity`, and `truncate_to_max_words` methods
+- Configuration validation for `similarity_threshold` (must be 0.0-1.0), `k_neighbors` (must be positive integer), `max_words` (must be nil or positive integer), and `fallback` (must be valid symbol)
+- `ConfigurationError` exception for invalid configuration values
+- Debug logging support with configurable `logger` option
+- Embedding cache with TTL support via `cache_ttl` option
+- `ask_batch` method for routing multiple messages efficiently
+- Retry logic with exponential backoff via `max_retries` and `retry_base_delay` options
+- Comprehensive test suite for Utils, Configuration, and edge cases
+- CHANGELOG.md, CONTRIBUTING.md, and ARCHITECTURE.md documentation
+
+### Changed
+- Extracted duplicate `cosine_distance` and `truncate_to_max_words` implementations into shared Utils module
+- Improved error messages for configuration validation
+
+### Breaking Changes
+- Configuration now validates values and raises `ConfigurationError` for invalid settings:
+  - `similarity_threshold` must be between 0.0 and 1.0
+  - `k_neighbors` must be a positive integer
+  - `max_words` must be nil or a positive integer
+  - `fallback` must be one of `:default_agent`, `:keep_current`, `:ask_clarification`
+- Previously, invalid values were silently accepted but could cause unexpected behavior. If your code was using invalid configuration values, you will now receive clear error messages indicating what needs to be fixed.
+
+## [0.2.0] - 2025-01-21
+
+### Added
+- `max_words` option to truncate messages before embedding generation
+- Global `default_max_words` configuration option
+- Message truncation applied consistently to both example import and routing
+
+### Changed
+- Updated README to be database-agnostic
+
+## [0.1.3] - 2025-01-20
+
+### Added
+- Custom vector search support via `find_examples` callback
+- Support for both `distance` (lower=better) and `score` (higher=better) in custom search results
+- Documented custom vector database integration (Pinecone, Qdrant, OpenSearch, etc.)
+
+### Fixed
+- Dependency versions and file permissions
+
+## [0.1.2] - 2025-01-19
+
+### Changed
+- Simplified API to accept `RubyLLM.chat` objects directly as agents
+- Agents now extract configuration from chat objects automatically
+
+## [0.1.1] - 2025-01-18
+
+### Added
+- ActiveRecord + pgvector example to README
+- Installation instructions
+
+### Changed
+- Simplified README structure
+
+## [0.1.0] - 2025-01-17
+
+### Added
+- Initial implementation of semantic routing for RubyLLM
+- Core `Router` class for managing multiple agents
+- `Semantic` routing strategy using embeddings and kNN search
+- Support for multiple fallback behaviors: `:default_agent`, `:keep_current`, `:ask_clarification`
+- In-memory example storage with `add_example` and `import_examples`
+- External example sources via `with_examples` (ActiveRecord compatible)
+- Scoped examples for multi-tenant applications
+- Routing callbacks via `on(:on_route)`
+- Debug routing with `match` and `debug_routing` methods
+- Global configuration via `RubyLLM::SemanticRouter.configure`
+- Comprehensive test suite
+
+[0.4.0]: https://github.com/khasinski/ruby_llm-semantic_router/compare/v0.3.0...v0.4.0
+[0.3.0]: https://github.com/khasinski/ruby_llm-semantic_router/compare/v0.2.0...v0.3.0
+[0.2.0]: https://github.com/khasinski/ruby_llm-semantic_router/compare/v0.1.3...v0.2.0
+[0.1.3]: https://github.com/khasinski/ruby_llm-semantic_router/compare/v0.1.2...v0.1.3
+[0.1.2]: https://github.com/khasinski/ruby_llm-semantic_router/compare/v0.1.1...v0.1.2
+[0.1.1]: https://github.com/khasinski/ruby_llm-semantic_router/compare/v0.1.0...v0.1.1
+[0.1.0]: https://github.com/khasinski/ruby_llm-semantic_router/releases/tag/v0.1.0

data/CONTRIBUTING.md

@@ -0,0 +1,103 @@
+# Contributing to RubyLLM Semantic Router
+
+Thank you for your interest in contributing! This document provides guidelines for contributing to the project.
+
+## Getting Started
+
+1. Fork the repository
+2. Clone your fork: `git clone https://github.com/YOUR_USERNAME/rubyllm-semantic_router.git`
+3. Install dependencies: `bundle install`
+4. Run tests: `bundle exec rspec`
+
+## Development Setup
+
+### Requirements
+
+- Ruby 3.1+
+- Bundler 2.0+
+
+### Running Tests
+
+```bash
+# Run all tests
+bundle exec rspec
+
+# Run specific test file
+bundle exec rspec spec/rubyllm/semantic_router/router_spec.rb
+
+# Run with verbose output
+bundle exec rspec --format documentation
+```
+
+## Making Changes
+
+### Code Style
+
+- Use frozen string literals (`# frozen_string_literal: true`)
+- Follow existing code patterns and naming conventions
+- Keep methods focused and under 20 lines when possible
+- Add YARD documentation for public methods
+
+### Testing
+
+- Write tests for all new functionality
+- Maintain or improve existing test coverage
+- Tests should be fast and not require external API calls
+- Use the mock RubyLLM classes in `spec/spec_helper.rb`
+
+### Commit Messages
+
+- Use clear, descriptive commit messages
+- Start with a verb (Add, Fix, Update, Remove, Refactor)
+- Keep the first line under 72 characters
+- Reference issues when applicable: `Fix #123`
+
+Examples:
+```
+Add batch routing support with ask_batch method
+Fix configuration validation for edge cases
+Update README with multi-tenant scoping docs
+```
+
+## Pull Request Process
+
+1. Create a feature branch: `git checkout -b feature/my-feature`
+2. Make your changes with tests
+3. Run the test suite: `bundle exec rspec`
+4. Update CHANGELOG.md with your changes under `[Unreleased]`
+5. Push to your fork and create a Pull Request
+
+### PR Checklist
+
+- [ ] Tests pass locally
+- [ ] New functionality has tests
+- [ ] CHANGELOG.md updated
+- [ ] Documentation updated (if applicable)
+- [ ] Code follows existing style
+
+## Reporting Issues
+
+When reporting issues, please include:
+
+- Ruby version (`ruby -v`)
+- Gem version
+- Minimal reproduction steps
+- Expected vs actual behavior
+- Error messages with backtraces
+
+## Feature Requests
+
+Feature requests are welcome! Please:
+
+- Check existing issues first
+- Describe the use case
+- Explain why existing functionality doesn't solve it
+- Consider if it fits the gem's scope
+
+## Architecture
+
+See [ARCHITECTURE.md](ARCHITECTURE.md) for an overview of the codebase structure and design decisions.
+
+## License
+
+By contributing, you agree that your contributions will be licensed under the MIT License.

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    rubyllm-semantic_router (0.1.0)
+    rubyllm-semantic_router (0.4.0)
       ruby_llm (~> 1.0)
 
 GEM
@@ -10,19 +10,19 @@ GEM
     base64 (0.3.0)
     diff-lcs (1.6.2)
     event_stream_parser (1.0.0)
-    faraday (2.14.0)
+    faraday (2.14.2)
       faraday-net_http (>= 2.0, < 3.5)
       json
       logger
     faraday-multipart (1.2.0)
       multipart-post (~> 2.0)
-    faraday-net_http (3.4.2)
+    faraday-net_http (3.4.3)
       net-http (~> 0.5)
     faraday-retry (2.4.0)
       faraday (~> 2.0)
-    json (2.18.0)
+    json (2.19.7)
     logger (1.7.0)
-    marcel (1.1.0)
+    marcel (1.2.1)
     multipart-post (2.4.1)
     net-http (0.9.1)
       uri (>= 0.11.1)
@@ -40,19 +40,19 @@ GEM
       diff-lcs (>= 1.2.0, < 2.0)
       rspec-support (~> 3.13.0)
     rspec-support (3.13.6)
-    ruby_llm (1.9.1)
+    ruby_llm (1.15.0)
       base64
       event_stream_parser (~> 1)
       faraday (>= 1.10.0)
       faraday-multipart (>= 1)
       faraday-net_http (>= 1)
       faraday-retry (>= 1)
-      marcel (~> 1.0)
-      ruby_llm-schema (~> 0.2.1)
+      marcel (~> 1)
+      ruby_llm-schema (~> 0)
       zeitwerk (~> 2)
-    ruby_llm-schema (0.2.5)
+    ruby_llm-schema (0.4.0)
     uri (1.1.1)
-    zeitwerk (2.7.4)
+    zeitwerk (2.8.2)
 
 PLATFORMS
   arm64-darwin-25