exaonruby 1.1.0 → 1.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 120800cd9da222f8a7cd47f1e608d4208a71e4a5b4d0a035b8ebfd5ffd17c115
-  data.tar.gz: 559bae5acadd43bc85854c2e62fde0852a40d683b2328fd0eea27f99aeef9470
+  metadata.gz: f7fa267a3b26d29aff9a6e5886cd43568ad0820aad17e54c8760141bbe208aa8
+  data.tar.gz: 2a1823d83853927fee270c1070b4ead7801d1d2df2d6e5c8769d644cefb31459
 SHA512:
-  metadata.gz: 788f13f5bc62a00227dd41b3803bb9d2bb88e5dc5f160ac6d684b6b3df487ebd66d050dc9f2da56355b7f4752ba10d1551ace7c8d49a1c1be6863b333c10b070
-  data.tar.gz: 90cdc099f08b5815329ed77551d40004ace0e869138888679a268c73c92dedb96539864f1f5b8480d67fe6b26a9baf489c47e212d60cfc09d9b5939d221023fd
+  metadata.gz: d128dce03cfee863a805039fcb4e9d4b12b93bff213ae0e03be21702040422a284864910df6916183c6aca5b59a91811d4cc3589c348733c31b441eee46db518
+  data.tar.gz: 87fb73c8641ef0a78af2f426d0a9499a934d4149ac6c482acab4a688575b87449a8d4c1feb23d46cf9f0b9c7853c787cf052acda30367c9c6fa8f38946559c47

data/CHANGELOG.md

@@ -5,6 +5,46 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.2.0] - 2025-12-18
+
+### Added
+
+- **Request Logging Middleware** - Pretty HTTP request/response logging
+  - Configurable log levels and body logging
+  - Sensitive data filtering (API keys, passwords)
+  - Request timing with status indicators (✓ ⚠ ✗)
+
+- **Rate Limiter Middleware** - Client-side rate limiting
+  - Token bucket algorithm for smooth rate control
+  - Configurable requests per second and burst size
+  - Thread-safe implementation
+
+- **Response Cache Middleware** - Reduce API costs
+  - `Exa::Cache::MemoryStore` for single-process apps
+  - `Exa::Cache::RedisStore` for distributed caching
+  - Configurable TTL and cacheable endpoints
+
+- **OpenTelemetry Instrumentation** - Distributed tracing
+  - Automatic span creation for all requests
+  - HTTP attributes and status codes
+  - Error recording
+
+- **Parallel Requests Utility** - Batch operations
+  - `Exa::Utils::Parallel.map` for concurrent requests
+  - Configurable concurrency limits
+  - Multiple error handling strategies
+
+- **Rails Integration** - First-class Rails support
+  - Railtie for automatic configuration
+  - `rails g exa:install` generator
+  - ActiveJob adapter for async research
+  - ActionCable integration for streaming
+  - Rake tasks (`rails exa:verify`, `rails exa:config`)
+
+### Changed
+
+- Added `concurrent-ruby` as a core dependency for parallel requests
+
 ## [1.1.0] - 2025-12-18
 
 ### Added

data/README.md

@@ -1,6 +1,8 @@
 # Exa Ruby
 
-A production-ready Ruby gem wrapper for the [Exa.ai](https://exa.ai) API, providing intelligent web search, content extraction, and structured data collection capabilities.
+A Ruby gem wrapper for the [Exa.ai](https://exa.ai) API, providing intelligent web search, content extraction, and structured data collection capabilities.
+
+Want to say thanks? Click the ⭐ at the top of the page.
 
 ## Features
 
@@ -14,11 +16,16 @@ A production-ready Ruby gem wrapper for the [Exa.ai](https://exa.ai) API, provid
 - **Imports**: Upload CSV data into Websets
 - **Webhooks & Events**: Real-time notifications for Websets activity
 - **SSE Streaming**: Real-time token streaming for Answer and Research APIs
-- **Sorbet Types**: Optional T::Struct type definitions for static type checking
+- **Request Logging**: Pretty HTTP logging with timing and status indicators
+- **Rate Limiting**: Client-side token bucket rate limiting
+- **Response Caching**: Memory and Redis caching to reduce API costs
+- **Instrumentation**: OpenTelemetry distributed tracing
+- **Parallel Requests**: Concurrent batch operations
+- **Rails Integration**: Railtie, generator, ActiveJob, ActionCable
+- **Sorbet Types**: Optional T::Struct type definitions
 - **Beautiful CLI**: Colorful command-line interface
 - **n8n/Zapier Integration**: Webhook signature verification utilities
 - **Automatic Retries**: Built-in retry logic for transient failures
-- **Rate Limit Handling**: Proper error handling with retry information
 - **Type Documentation**: Comprehensive YARD documentation
 
 ## Installation
@@ -670,7 +677,10 @@ params = Exa::Types::SearchParams.new(
 - faraday >= 2.0
 - faraday-retry >= 2.0
 - thor >= 1.0
+- concurrent-ruby >= 1.2
 - sorbet-runtime >= 0.5 (optional, for type definitions)
+- opentelemetry-sdk (optional, for instrumentation)
+- redis (optional, for distributed caching)
 
 ## Development
 
@@ -683,4 +693,3 @@ The gem is available as open source under the terms of the [MIT License](https:/
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/exa-labs/exa-ruby.
-

data/exaonruby.gemspec

@@ -8,11 +8,12 @@ Gem::Specification.new do |spec|
   spec.authors = ["tigel-agm"]
   spec.email = []
 
-  spec.summary = "Complete Ruby client for the Exa.ai API with CLI and SSE streaming"
+  spec.summary = "Complete Ruby client for the Exa.ai API with CLI, middleware, and Rails integration"
   spec.description = "A production-ready Ruby gem wrapper for the Exa.ai Search and Websets APIs. " \
                      "Features neural search, LLM-powered answers, async research tasks, " \
                      "Websets management (monitors, imports, webhooks), SSE streaming, " \
-                     "Sorbet type definitions, and a beautiful CLI. " \
+                     "request logging, rate limiting, response caching, OpenTelemetry instrumentation, " \
+                     "parallel requests, Rails integration, Sorbet types, and a beautiful CLI. " \
                      "Includes n8n/Zapier webhook signature verification utilities."
   spec.homepage = "https://github.com/tigel-agm/exaonruby"
   spec.license = "MIT"
@@ -36,7 +37,9 @@ Gem::Specification.new do |spec|
   spec.add_dependency "faraday", ">= 2.0", "< 3.0"
   spec.add_dependency "faraday-retry", ">= 2.0", "< 3.0"
   spec.add_dependency "thor", ">= 1.0", "< 3.0"
+  spec.add_dependency "concurrent-ruby", ">= 1.2", "< 2.0"
 
   # Optional: Sorbet types (install sorbet-runtime for type checking)
-  # gem install sorbet-runtime
+  # Optional: OpenTelemetry (install opentelemetry-sdk for instrumentation)
+  # Optional: Redis (install redis gem for distributed caching)
 end

data/lib/exa/middleware/instrumentation.rb

@@ -0,0 +1,97 @@
+# frozen_string_literal: true
+
+# typed: strict
+
+module Exa
+  module Middleware
+    # OpenTelemetry instrumentation middleware for distributed tracing
+    #
+    # Integrates with OpenTelemetry for comprehensive request tracing.
+    # Requires opentelemetry-sdk gem to be installed.
+    #
+    # @example Enable instrumentation
+    #   require 'opentelemetry/sdk'
+    #   OpenTelemetry::SDK.configure
+    #
+    #   client = Exa::Client.new(api_key: key) do |config|
+    #     config.instrumentation = true
+    #   end
+    class Instrumentation < Faraday::Middleware
+      # @param app [Faraday::Middleware] Next middleware
+      # @param tracer_name [String] Name for the tracer
+      def initialize(app, tracer_name: "exa.api")
+        super(app)
+        @tracer_name = tracer_name
+        @tracer = nil
+      end
+
+      def call(env)
+        return @app.call(env) unless otel_available?
+
+        tracer.in_span(span_name(env), attributes: span_attributes(env), kind: :client) do |span|
+          begin
+            response = @app.call(env)
+            
+            response.on_complete do |response_env|
+              span.set_attribute("http.status_code", response_env[:status])
+              span.status = otel_status(response_env[:status])
+            end
+            
+            response
+          rescue StandardError => e
+            span.record_exception(e)
+            span.status = OpenTelemetry::Trace::Status.error(e.message)
+            raise
+          end
+        end
+      end
+
+      private
+
+      def otel_available?
+        defined?(OpenTelemetry) && OpenTelemetry.tracer_provider
+      rescue StandardError
+        false
+      end
+
+      def tracer
+        @tracer ||= OpenTelemetry.tracer_provider.tracer(@tracer_name)
+      end
+
+      def span_name(env)
+        method = env[:method].to_s.upcase
+        path = env[:url].path.split("/").reject(&:empty?).first || "request"
+        "Exa #{method} /#{path}"
+      end
+
+      def span_attributes(env)
+        {
+          "http.method" => env[:method].to_s.upcase,
+          "http.url" => sanitize_url(env[:url].to_s),
+          "http.target" => env[:url].path,
+          "http.host" => env[:url].host,
+          "http.scheme" => env[:url].scheme,
+          "net.peer.name" => env[:url].host,
+          "net.peer.port" => env[:url].port
+        }
+      end
+
+      def sanitize_url(url)
+        url.gsub(/api_key=[^&]+/, "api_key=[REDACTED]")
+      end
+
+      def otel_status(status_code)
+        case status_code
+        when 200..299
+          OpenTelemetry::Trace::Status.ok
+        when 400..599
+          OpenTelemetry::Trace::Status.error("HTTP #{status_code}")
+        else
+          OpenTelemetry::Trace::Status.unset
+        end
+      end
+    end
+
+    Faraday::Middleware.register_middleware(exa_instrumentation: Instrumentation)
+  end
+end

data/lib/exa/middleware/rate_limiter.rb

@@ -0,0 +1,72 @@
+# frozen_string_literal: true
+
+# typed: strict
+
+module Exa
+  module Middleware
+    # Client-side rate limiter using token bucket algorithm
+    #
+    # Prevents 429 errors by limiting requests before they're sent.
+    # Uses a thread-safe token bucket with configurable rates.
+    #
+    # @example Enable rate limiting
+    #   client = Exa::Client.new(api_key: key) do |config|
+    #     config.rate_limit = 10         # requests per second
+    #     config.rate_limit_burst = 20   # max burst size
+    #   end
+    #
+    # @example Lower limits for safety
+    #   client = Exa::Client.new(api_key: key) do |config|
+    #     config.rate_limit = 5          # conservative 5 req/sec
+    #   end
+    class RateLimiter < Faraday::Middleware
+      # @param app [Faraday::Middleware] Next middleware
+      # @param rate [Float] Tokens per second (requests per second)
+      # @param burst [Integer] Maximum tokens (burst capacity)
+      def initialize(app, rate: 10.0, burst: 20)
+        super(app)
+        @rate = rate.to_f
+        @burst = burst
+        @tokens = burst.to_f
+        @last_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        @mutex = Mutex.new
+      end
+
+      def call(env)
+        wait_for_token
+        @app.call(env)
+      end
+
+      private
+
+      # Wait until a token is available
+      def wait_for_token
+        @mutex.synchronize do
+          refill_tokens
+          
+          if @tokens < 1.0
+            # Calculate wait time needed
+            wait_time = (1.0 - @tokens) / @rate
+            sleep(wait_time) if wait_time > 0
+            refill_tokens
+          end
+
+          @tokens -= 1.0
+        end
+      end
+
+      # Refill tokens based on time elapsed
+      def refill_tokens
+        now = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        elapsed = now - @last_time
+        @last_time = now
+
+        # Add tokens based on elapsed time
+        @tokens = [@tokens + (elapsed * @rate), @burst.to_f].min
+      end
+    end
+
+    # Faraday middleware registration
+    Faraday::Middleware.register_middleware(exa_rate_limiter: RateLimiter)
+  end
+end

data/lib/exa/middleware/request_logger.rb

@@ -0,0 +1,170 @@
+# frozen_string_literal: true
+
+# typed: strict
+
+require "logger"
+require "json"
+
+module Exa
+  module Middleware
+    # Pretty request/response logging middleware for Faraday
+    #
+    # Logs all HTTP requests and responses with timing, status codes, and
+    # optional body logging. Useful for debugging and monitoring API usage.
+    #
+    # @example Enable logging on client
+    #   client = Exa::Client.new(api_key: key) do |config|
+    #     config.logger = Logger.new($stdout)
+    #     config.log_level = :info
+    #     config.log_bodies = true
+    #   end
+    #
+    # @example Custom logger
+    #   client = Exa::Client.new(api_key: key) do |config|
+    #     config.logger = Rails.logger
+    #     config.log_level = :debug
+    #   end
+    class RequestLogger < Faraday::Middleware
+      DEFAULT_OPTIONS = {
+        log_level: :info,
+        log_headers: false,
+        log_bodies: false,
+        filter_keys: %w[api_key x-api-key authorization password secret token].freeze,
+        max_body_size: 2000
+      }.freeze
+
+      # @param app [Faraday::Middleware] Next middleware in chain
+      # @param logger [Logger] Logger instance
+      # @param options [Hash] Configuration options
+      def initialize(app, logger: nil, **options)
+        super(app)
+        @logger = logger || default_logger
+        @options = DEFAULT_OPTIONS.merge(options)
+      end
+
+      def call(env)
+        start_time = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+        request_id = generate_request_id
+
+        log_request(env, request_id)
+
+        @app.call(env).on_complete do |response_env|
+          elapsed = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time) * 1000).round(2)
+          log_response(response_env, request_id, elapsed)
+        end
+      rescue StandardError => e
+        elapsed = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start_time) * 1000).round(2)
+        log_error(env, request_id, elapsed, e)
+        raise
+      end
+
+      private
+
+      def log_request(env, request_id)
+        method = env[:method].to_s.upcase
+        url = env[:url].to_s.gsub(/api_key=[^&]+/, "api_key=[FILTERED]")
+
+        message = "[Exa] #{request_id} → #{method} #{url}"
+
+        if @options[:log_headers]
+          headers = filter_sensitive(env[:request_headers] || {})
+          message += "\n  Headers: #{headers.to_json}"
+        end
+
+        if @options[:log_bodies] && env[:body]
+          body = truncate_body(filter_body(env[:body]))
+          message += "\n  Body: #{body}"
+        end
+
+        log(message)
+      end
+
+      def log_response(env, request_id, elapsed)
+        status = env[:status]
+        status_emoji = status_indicator(status)
+
+        message = "[Exa] #{request_id} ← #{status_emoji} #{status} (#{elapsed}ms)"
+
+        if @options[:log_bodies] && env[:body]
+          body = truncate_body(env[:body].to_s)
+          message += "\n  Body: #{body}"
+        end
+
+        log(message)
+      end
+
+      def log_error(env, request_id, elapsed, error)
+        message = "[Exa] #{request_id} ✗ ERROR after #{elapsed}ms: #{error.class}: #{error.message}"
+        log(message, :error)
+      end
+
+      def log(message, level = nil)
+        level ||= @options[:log_level]
+
+        case level
+        when :debug
+          @logger.debug(message)
+        when :info
+          @logger.info(message)
+        when :warn
+          @logger.warn(message)
+        when :error
+          @logger.error(message)
+        else
+          @logger.info(message)
+        end
+      end
+
+      def status_indicator(status)
+        case status
+        when 200..299 then "✓"
+        when 300..399 then "↻"
+        when 400..499 then "⚠"
+        when 500..599 then "✗"
+        else "?"
+        end
+      end
+
+      def generate_request_id
+        format("%06x", rand(0xFFFFFF))
+      end
+
+      def filter_sensitive(hash)
+        hash.each_with_object({}) do |(key, value), result|
+          if @options[:filter_keys].any? { |k| key.to_s.downcase.include?(k.downcase) }
+            result[key] = "[FILTERED]"
+          else
+            result[key] = value
+          end
+        end
+      end
+
+      def filter_body(body)
+        return body unless body.is_a?(Hash)
+
+        body.each_with_object({}) do |(key, value), result|
+          if @options[:filter_keys].any? { |k| key.to_s.downcase.include?(k.downcase) }
+            result[key] = "[FILTERED]"
+          else
+            result[key] = value
+          end
+        end.to_json
+      end
+
+      def truncate_body(body)
+        return body if body.length <= @options[:max_body_size]
+
+        "#{body[0, @options[:max_body_size]]}... (truncated)"
+      end
+
+      def default_logger
+        logger = Logger.new($stdout)
+        logger.formatter = proc { |_, _, _, msg| "#{msg}\n" }
+        logger
+      end
+    end
+
+    # Register middleware with Faraday
+    Faraday::Middleware.register_middleware(exa_logger: RequestLogger)
+  end
+end

data/lib/exa/middleware/response_cache.rb

@@ -0,0 +1,226 @@
+# frozen_string_literal: true
+
+# typed: strict
+
+require "digest"
+require "json"
+
+module Exa
+  module Middleware
+    # Response caching middleware for Exa API calls
+    #
+    # Caches GET requests and idempotent POST requests (like search)
+    # to reduce API costs and improve response times.
+    #
+    # @example Enable caching with memory store
+    #   client = Exa::Client.new(api_key: key) do |config|
+    #     config.cache = Exa::Cache::MemoryStore.new
+    #     config.cache_ttl = 300  # 5 minutes
+    #   end
+    #
+    # @example With Redis
+    #   require 'redis'
+    #   client = Exa::Client.new(api_key: key) do |config|
+    #     config.cache = Exa::Cache::RedisStore.new(Redis.new)
+    #     config.cache_ttl = 3600  # 1 hour
+    #   end
+    class ResponseCache < Faraday::Middleware
+      # Cacheable endpoints (idempotent operations)
+      CACHEABLE_PATHS = %w[
+        /search
+        /contents
+        /findSimilar
+      ].freeze
+
+      # @param app [Faraday::Middleware] Next middleware
+      # @param cache [Object] Cache store (must respond to get/set)
+      # @param ttl [Integer] Time to live in seconds
+      # @param cacheable_paths [Array<String>] Paths to cache
+      def initialize(app, cache:, ttl: 300, cacheable_paths: CACHEABLE_PATHS)
+        super(app)
+        @cache = cache
+        @ttl = ttl
+        @cacheable_paths = cacheable_paths
+      end
+
+      def call(env)
+        return @app.call(env) unless cacheable?(env)
+
+        cache_key = build_cache_key(env)
+        
+        # Try to get from cache
+        cached = @cache.get(cache_key)
+        if cached
+          return build_cached_response(env, cached)
+        end
+
+        # Make request and cache response
+        @app.call(env).on_complete do |response_env|
+          if response_env[:status] == 200
+            cache_response(cache_key, response_env)
+          end
+        end
+      end
+
+      private
+
+      def cacheable?(env)
+        path = env[:url].path
+        @cacheable_paths.any? { |p| path.include?(p) }
+      end
+
+      def build_cache_key(env)
+        # Include method, path, and body in cache key
+        components = [
+          env[:method].to_s,
+          env[:url].path,
+          env[:body].to_s
+        ]
+        
+        digest = Digest::SHA256.hexdigest(components.join("|"))
+        "exa:cache:#{digest}"
+      end
+
+      def cache_response(key, env)
+        data = {
+          status: env[:status],
+          headers: env[:response_headers].to_h,
+          body: env[:body]
+        }
+        
+        @cache.set(key, data.to_json, ttl: @ttl)
+      end
+
+      def build_cached_response(env, cached_data)
+        data = JSON.parse(cached_data, symbolize_names: true)
+        
+        env[:status] = data[:status]
+        env[:response_headers] = Faraday::Utils::Headers.new(data[:headers])
+        env[:body] = data[:body]
+        
+        # Add cache hit header
+        env[:response_headers]["X-Exa-Cache"] = "HIT"
+        
+        Faraday::Response.new(env)
+      end
+    end
+
+    Faraday::Middleware.register_middleware(exa_cache: ResponseCache)
+  end
+
+  module Cache
+    # In-memory cache store with TTL support
+    #
+    # Thread-safe, suitable for single-process applications.
+    # For multi-process or distributed apps, use RedisStore.
+    class MemoryStore
+      def initialize
+        @store = {}
+        @expirations = {}
+        @mutex = Mutex.new
+      end
+
+      # Get value from cache
+      # @param key [String] Cache key
+      # @return [String, nil] Cached value or nil
+      def get(key)
+        @mutex.synchronize do
+          cleanup_expired
+          @store[key]
+        end
+      end
+
+      # Set value in cache
+      # @param key [String] Cache key
+      # @param value [String] Value to cache
+      # @param ttl [Integer] Time to live in seconds
+      def set(key, value, ttl: 300)
+        @mutex.synchronize do
+          @store[key] = value
+          @expirations[key] = Time.now + ttl
+        end
+      end
+
+      # Delete from cache
+      # @param key [String] Cache key
+      def delete(key)
+        @mutex.synchronize do
+          @store.delete(key)
+          @expirations.delete(key)
+        end
+      end
+
+      # Clear entire cache
+      def clear
+        @mutex.synchronize do
+          @store.clear
+          @expirations.clear
+        end
+      end
+
+      # Get cache statistics
+      # @return [Hash] Stats including size and hit count
+      def stats
+        @mutex.synchronize do
+          cleanup_expired
+          { size: @store.size }
+        end
+      end
+
+      private
+
+      def cleanup_expired
+        now = Time.now
+        expired_keys = @expirations.select { |_, exp| exp < now }.keys
+        expired_keys.each do |key|
+          @store.delete(key)
+          @expirations.delete(key)
+        end
+      end
+    end
+
+    # Redis cache store for distributed caching
+    #
+    # Requires redis gem to be installed.
+    #
+    # @example
+    #   require 'redis'
+    #   cache = Exa::Cache::RedisStore.new(Redis.new)
+    class RedisStore
+      # @param redis [Redis] Redis client instance
+      # @param prefix [String] Key prefix
+      def initialize(redis, prefix: "exa")
+        @redis = redis
+        @prefix = prefix
+      end
+
+      def get(key)
+        @redis.get(prefixed(key))
+      end
+
+      def set(key, value, ttl: 300)
+        @redis.setex(prefixed(key), ttl, value)
+      end
+
+      def delete(key)
+        @redis.del(prefixed(key))
+      end
+
+      def clear
+        keys = @redis.keys("#{@prefix}:*")
+        @redis.del(*keys) if keys.any?
+      end
+
+      def stats
+        keys = @redis.keys("#{@prefix}:*")
+        { size: keys.size }
+      end
+
+      private
+
+      def prefixed(key)
+        key.start_with?(@prefix) ? key : "#{@prefix}:#{key}"
+      end
+    end
+  end
+end

data/lib/exa/rails.rb

@@ -0,0 +1,157 @@
+# frozen_string_literal: true
+
+# typed: strict
+
+module Exa
+  # Rails integration for the Exa gem
+  #
+  # Provides generators, Active Job adapters, and Rails-specific configuration.
+  #
+  # Install with: rails generate exa:install
+  module Rails
+    class Railtie < ::Rails::Railtie
+      # Initialize Exa with Rails configuration
+      initializer "exa.configure" do |app|
+        # Load config from Rails credentials or environment
+        Exa.configure do |config|
+          config.api_key = rails_api_key(app)
+          config.logger = ::Rails.logger if defined?(::Rails.logger)
+          config.timeout = ENV.fetch("EXA_TIMEOUT", 60).to_i
+        end
+      end
+
+      # Add Exa rake tasks
+      rake_tasks do
+        namespace :exa do
+          desc "Verify Exa API connection"
+          task verify: :environment do
+            begin
+              client = Exa::Client.new
+              result = client.search("test", num_results: 1)
+              puts "✓ Exa API connection successful"
+              puts "  Request ID: #{result.request_id}"
+            rescue Exa::Error => e
+              puts "✗ Exa API error: #{e.message}"
+              exit 1
+            end
+          end
+
+          desc "Show Exa configuration"
+          task config: :environment do
+            puts "Exa Configuration:"
+            puts "  API Key: #{Exa.configuration&.api_key&.slice(0, 8)}..."
+            puts "  Base URL: #{Exa.configuration&.base_url}"
+            puts "  Timeout: #{Exa.configuration&.timeout}s"
+          end
+        end
+      end
+
+      private
+
+      def rails_api_key(app)
+        # Try Rails credentials first
+        if app.credentials.respond_to?(:exa)
+          app.credentials.exa[:api_key]
+        elsif app.credentials.respond_to?(:dig)
+          app.credentials.dig(:exa, :api_key)
+        end || ENV["EXA_API_KEY"]
+      end
+    end
+
+    # Active Job adapter for async research tasks
+    #
+    # @example Create a research job
+    #   class ExaResearchJob < ApplicationJob
+    #     include Exa::Rails::ResearchJob
+    #
+    #     def perform(instructions)
+    #       research(instructions) do |task|
+    #         # Called when research completes
+    #         save_results(task.output)
+    #       end
+    #     end
+    #   end
+    module ResearchJob
+      extend ActiveSupport::Concern
+
+      included do
+        queue_as :exa_research
+        retry_on Exa::RateLimitError, wait: :exponentially_longer, attempts: 5
+        discard_on Exa::AuthenticationError
+      end
+
+      # Start a research task and poll until complete
+      #
+      # @param instructions [String] Research instructions
+      # @param model [String] Research model
+      # @param poll_interval [Integer] Seconds between status checks
+      #
+      # @yield [Exa::Resources::ResearchTask] Called when complete
+      def research(instructions, model: "exa-research", poll_interval: 5)
+        client = Exa::Client.new
+        task = client.create_research(instructions: instructions, model: model)
+
+        loop do
+          task = client.get_research(task.research_id)
+
+          case task.status
+          when "completed"
+            yield task if block_given?
+            return task
+          when "failed", "canceled"
+            raise Exa::Error, "Research task #{task.status}: #{task.error_message}"
+          else
+            sleep poll_interval
+          end
+        end
+      end
+    end
+
+    # Action Cable channel for real-time updates
+    #
+    # @example Create a channel
+    #   class ExaSearchChannel < ApplicationCable::Channel
+    #     include Exa::Rails::StreamingChannel
+    #
+    #     def search(data)
+    #       stream_answer(data["query"])
+    #     end
+    #   end
+    module StreamingChannel
+      extend ActiveSupport::Concern
+
+      # Stream answer tokens to the client
+      #
+      # @param query [String] Question to answer
+      def stream_answer(query)
+        Exa::Utils::SSEClient.stream_answer(
+          api_key: ENV["EXA_API_KEY"],
+          query: query
+        ) do |event|
+          case event[:type]
+          when :token
+            transmit({ type: "token", data: event[:data] })
+          when :citation
+            transmit({ type: "citation", data: event[:data] })
+          when :done
+            transmit({ type: "done" })
+          when :error
+            transmit({ type: "error", data: event[:data] })
+          end
+        end
+      end
+
+      # Stream research progress to the client
+      #
+      # @param instructions [String] Research instructions
+      def stream_research(instructions)
+        Exa::Utils::SSEClient.stream_research(
+          api_key: ENV["EXA_API_KEY"],
+          instructions: instructions
+        ) do |event|
+          transmit({ type: event[:type].to_s, data: event[:data] })
+        end
+      end
+    end
+  end
+end

data/lib/exa/utils/parallel.rb

@@ -0,0 +1,135 @@
+# frozen_string_literal: true
+
+# typed: strict
+
+require "concurrent"
+
+module Exa
+  module Utils
+    # Parallel request executor for batch operations
+    #
+    # Execute multiple Exa API calls concurrently with configurable
+    # concurrency limits and automatic error handling.
+    #
+    # @example Parallel searches
+    #   queries = ["AI research", "ML trends", "NLP papers"]
+    #   
+    #   results = Exa::Utils::Parallel.map(queries, client: client) do |query|
+    #     client.search(query, num_results: 10)
+    #   end
+    #
+    # @example With concurrency limit
+    #   results = Exa::Utils::Parallel.map(urls, concurrency: 5) do |url|
+    #     client.get_contents([url])
+    #   end
+    #
+    # @example Error handling
+    #   results = Exa::Utils::Parallel.map(queries, on_error: :skip) do |query|
+    #     client.search(query)
+    #   end
+    class Parallel
+      # Default concurrency limit
+      DEFAULT_CONCURRENCY = 10
+
+      class << self
+        # Execute block for each item in parallel
+        #
+        # @param items [Array] Items to process
+        # @param concurrency [Integer] Max concurrent requests
+        # @param on_error [Symbol] Error handling: :raise, :skip, :return_nil
+        # @param timeout [Integer, nil] Timeout per request in seconds
+        #
+        # @yield [item] Block to execute for each item
+        # @return [Array] Results in same order as input
+        #
+        # @example
+        #   Exa::Utils::Parallel.map(queries) { |q| client.search(q) }
+        def map(items, concurrency: DEFAULT_CONCURRENCY, on_error: :raise, timeout: nil)
+          return [] if items.empty?
+
+          pool = Concurrent::FixedThreadPool.new(concurrency)
+          futures = []
+
+          items.each_with_index do |item, idx|
+            future = Concurrent::Future.execute(executor: pool) do
+              if timeout
+                Concurrent::Promises.future { yield(item) }.value!(timeout)
+              else
+                yield(item)
+              end
+            end
+            futures << { index: idx, future: future }
+          end
+
+          # Collect results in order
+          results = Array.new(items.length)
+          
+          futures.each do |entry|
+            begin
+              results[entry[:index]] = entry[:future].value!
+            rescue StandardError => e
+              case on_error
+              when :raise
+                pool.shutdown
+                raise
+              when :skip
+                next
+              when :return_nil
+                results[entry[:index]] = nil
+              else
+                raise
+              end
+            end
+          end
+
+          pool.shutdown
+          pool.wait_for_termination
+
+          results
+        end
+
+        # Execute block for each item in parallel, returning only successful results
+        #
+        # @param items [Array] Items to process
+        # @param concurrency [Integer] Max concurrent requests
+        #
+        # @yield [item] Block to execute for each item
+        # @return [Array] Successful results only
+        def map_compact(items, concurrency: DEFAULT_CONCURRENCY, &block)
+          map(items, concurrency: concurrency, on_error: :return_nil, &block).compact
+        end
+
+        # Execute block for each item in parallel, ignoring return values
+        #
+        # @param items [Array] Items to process
+        # @param concurrency [Integer] Max concurrent requests
+        #
+        # @yield [item] Block to execute for each item
+        # @return [void]
+        def each(items, concurrency: DEFAULT_CONCURRENCY, &block)
+          map(items, concurrency: concurrency, on_error: :skip, &block)
+          nil
+        end
+
+        # Execute multiple different operations in parallel
+        #
+        # @param operations [Array<Proc>] Procs to execute
+        # @param concurrency [Integer] Max concurrent requests
+        #
+        # @return [Array] Results from each proc
+        #
+        # @example
+        #   results = Exa::Utils::Parallel.all(
+        #     -> { client.search("AI") },
+        #     -> { client.search("ML") },
+        #     -> { client.get_contents([url]) }
+        #   )
+        def all(*operations, concurrency: DEFAULT_CONCURRENCY)
+          operations = operations.first if operations.first.is_a?(Array)
+          
+          map(operations, concurrency: concurrency) { |op| op.call }
+        end
+      end
+    end
+  end
+end

data/lib/exa/version.rb

@@ -3,5 +3,5 @@
 # typed: strict
 
 module Exa
-  VERSION = "1.1.0"
+  VERSION = "1.2.0"
 end

data/lib/exa.rb

@@ -9,6 +9,13 @@ require_relative "exa/configuration"
 require_relative "exa/utils/parameter_converter"
 require_relative "exa/utils/webhook_handler"
 require_relative "exa/utils/sse_client"
+require_relative "exa/utils/parallel"
+
+# Middleware (loaded after Faraday is available in Client)
+require_relative "exa/middleware/request_logger"
+require_relative "exa/middleware/rate_limiter"
+require_relative "exa/middleware/response_cache"
+require_relative "exa/middleware/instrumentation"
 
 # Optional: Sorbet types (only loaded if sorbet-runtime is available)
 begin
@@ -18,6 +25,15 @@ rescue LoadError
   # sorbet-runtime not installed, types module not available
 end
 
+# Optional: Rails integration (only loaded if Rails is available)
+begin
+  if defined?(::Rails::Railtie)
+    require_relative "exa/rails"
+  end
+rescue LoadError
+  # Rails not available
+end
+
 require_relative "exa/resources/base"
 require_relative "exa/resources/search_result"
 require_relative "exa/resources/search_response"

data/lib/generators/exa/install_generator.rb

@@ -0,0 +1,94 @@
+# frozen_string_literal: true
+
+require "rails/generators/base"
+
+module Exa
+  module Generators
+    # Rails generator for Exa gem installation
+    #
+    # Usage:
+    #   rails generate exa:install
+    #
+    # This creates:
+    #   - config/initializers/exa.rb
+    #   - Adds EXA_API_KEY to .env if using dotenv
+    class InstallGenerator < ::Rails::Generators::Base
+      source_root File.expand_path("templates", __dir__)
+
+      desc "Creates an Exa initializer and credentials setup"
+
+      def create_initializer
+        create_file "config/initializers/exa.rb", <<~RUBY
+          # frozen_string_literal: true
+
+          # Exa API Configuration
+          # Documentation: https://docs.exa.ai
+          # Gem: https://github.com/tigel-agm/exaonruby
+
+          Exa.configure do |config|
+            # API Key (from Rails credentials or environment)
+            # Run: rails credentials:edit
+            # Add: exa: { api_key: "your-key" }
+            config.api_key = Rails.application.credentials.dig(:exa, :api_key) || ENV["EXA_API_KEY"]
+
+            # Request timeout in seconds (default: 60)
+            config.timeout = 60
+
+            # Maximum retry attempts for failed requests
+            config.max_retries = 3
+
+            # Optional: Enable request logging in development
+            if Rails.env.development?
+              config.logger = Rails.logger
+            end
+
+            # Optional: Enable caching to reduce API costs
+            # config.cache = Exa::Cache::MemoryStore.new
+            # config.cache_ttl = 300  # 5 minutes
+
+            # Optional: Rate limiting (requests per second)
+            # config.rate_limit = 10
+            # config.rate_limit_burst = 20
+          end
+        RUBY
+        say "Created config/initializers/exa.rb", :green
+      end
+
+      def add_to_env
+        if File.exist?(".env")
+          append_to_file ".env" do
+            "\n# Exa API Key\nEXA_API_KEY=your-api-key-here\n"
+          end
+          say "Added EXA_API_KEY to .env", :green
+        elsif File.exist?(".env.example")
+          append_to_file ".env.example" do
+            "\n# Exa API Key\nEXA_API_KEY=\n"
+          end
+          say "Added EXA_API_KEY to .env.example", :green
+        end
+      end
+
+      def show_instructions
+        say ""
+        say "Exa gem installed successfully!", :green
+        say ""
+        say "Next steps:"
+        say "  1. Add your API key to Rails credentials:"
+        say "     rails credentials:edit"
+        say ""
+        say "     exa:"
+        say "       api_key: your-api-key-here"
+        say ""
+        say "  2. Or set the EXA_API_KEY environment variable"
+        say ""
+        say "  3. Test the connection:"
+        say "     rails exa:verify"
+        say ""
+        say "Usage examples:"
+        say "  client = Exa::Client.new"
+        say "  results = client.search('AI research', num_results: 10)"
+        say ""
+      end
+    end
+  end
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: exaonruby
 version: !ruby/object:Gem::Version
-  version: 1.1.0
+  version: 1.2.0
 platform: ruby
 authors:
 - tigel-agm
@@ -69,10 +69,32 @@ dependencies:
     - - "<"
       - !ruby/object:Gem::Version
         version: '3.0'
+- !ruby/object:Gem::Dependency
+  name: concurrent-ruby
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.2'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '1.2'
+    - - "<"
+      - !ruby/object:Gem::Version
+        version: '2.0'
 description: A production-ready Ruby gem wrapper for the Exa.ai Search and Websets
   APIs. Features neural search, LLM-powered answers, async research tasks, Websets
-  management (monitors, imports, webhooks), SSE streaming, Sorbet type definitions,
-  and a beautiful CLI. Includes n8n/Zapier webhook signature verification utilities.
+  management (monitors, imports, webhooks), SSE streaming, request logging, rate limiting,
+  response caching, OpenTelemetry instrumentation, parallel requests, Rails integration,
+  Sorbet types, and a beautiful CLI. Includes n8n/Zapier webhook signature verification
+  utilities.
 email: []
 executables:
 - exa
@@ -102,6 +124,11 @@ files:
 - lib/exa/endpoints/webset_searches.rb
 - lib/exa/endpoints/websets.rb
 - lib/exa/errors.rb
+- lib/exa/middleware/instrumentation.rb
+- lib/exa/middleware/rate_limiter.rb
+- lib/exa/middleware/request_logger.rb
+- lib/exa/middleware/response_cache.rb
+- lib/exa/rails.rb
 - lib/exa/resources/answer_response.rb
 - lib/exa/resources/base.rb
 - lib/exa/resources/contents_response.rb
@@ -116,10 +143,12 @@ files:
 - lib/exa/resources/webset.rb
 - lib/exa/resources/webset_item.rb
 - lib/exa/types.rb
+- lib/exa/utils/parallel.rb
 - lib/exa/utils/parameter_converter.rb
 - lib/exa/utils/sse_client.rb
 - lib/exa/utils/webhook_handler.rb
 - lib/exa/version.rb
+- lib/generators/exa/install_generator.rb
 homepage: https://github.com/tigel-agm/exaonruby
 licenses:
 - MIT
@@ -145,5 +174,5 @@ required_rubygems_version: !ruby/object:Gem::Requirement
 requirements: []
 rubygems_version: 4.0.2
 specification_version: 4
-summary: Complete Ruby client for the Exa.ai API with CLI and SSE streaming
+summary: Complete Ruby client for the Exa.ai API with CLI, middleware, and Rails integration
 test_files: []