prescient 0.6.0 → 0.8.0

data/README.md

@@ -2,25 +2,116 @@
 
 [![Gem Version](https://img.shields.io/gem/v/prescient?logo=rubygems&logoColor=white)](https://rubygems.org/gems/prescient)
 [![Requires Ruby 3.1+](https://img.shields.io/badge/Requires-Ruby%203.1%2B-CC342D?logo=ruby&logoColor=white)](https://www.ruby-lang.org/)
+[![Docker](https://img.shields.io/badge/Docker-GHCR-2496ED?logo=docker&logoColor=white)](https://github.com/kanutocd/prescient/pkgs/container/prescient)
 [![CI](https://github.com/kanutocd/prescient/actions/workflows/ci.yml/badge.svg)](https://github.com/kanutocd/prescient/actions/workflows/ci.yml)
 [![Security](https://img.shields.io/github/actions/workflow/status/kanutocd/prescient/security.yml?branch=main&event=push&label=Security)](https://github.com/kanutocd/prescient/actions/workflows/security.yml)
 [![License](https://img.shields.io/badge/License-MIT-22C55E)](LICENSE.txt)
 
-Prescient is a boring AI provider abstraction for Ruby. Configure your AI providers once, then use the same interface regardless of whether the request is handled by OpenAI, Anthropic, Ollama, Hugging Face, Google Gemini, Mistral, DeepSeek, or xAI. Prescient handles provider selection, retries, health checks, and fallback.
+Prescient is a boring AI provider gateway implemented in Ruby. Configure your AI providers once, then use them through a consistent Ruby API, CLI, or REST API. Prescient handles provider selection, retries, health checks, and fallback across configured providers, including OpenAI, Anthropic, Ollama, Hugging Face, Google Gemini, Mistral, DeepSeek, and xAI.
 
 For focused guidance, see the **[examples guide](https://github.com/kanutocd/prescient/tree/main/examples)**,
 **[Rails integration guide](https://github.com/kanutocd/prescient/blob/main/INTEGRATION_GUIDE.md)**, and
 **[pgvector guide](https://github.com/kanutocd/prescient/blob/main/VECTOR_SEARCH_GUIDE.md)**.
 
+Prescient has three primary application entry points:
+
+- **Ruby gem** — Use `Prescient` directly from a Ruby application.
+- **CLI** — Run provider operations and configuration checks from scripts or a
+  terminal.
+- **Rack-compatible REST API** — Mount `Prescient::API` in a Rack, Rails, or
+  other Rack-compatible application.
+
+The Ruby entry point does not load the CLI, REST API, or MCP transport. Those
+surfaces are loaded lazily when referenced or explicitly required.
+MCP is an optional protocol integration surface available through explicit
+`prescient/mcp` loading; it does not change the three primary entry points.
+
+## Contents
+
+- [Quick Start](#quick-start)
+- [Features](#features)
+- [Supported Providers](#supported-providers)
+- [Installation](#installation)
+- [Command-Line Interface](#command-line-interface)
+- [REST API](#rest-api)
+- [Configuration](#configuration)
+- [Ruby API](#ruby-api)
+- [Agent runtime](#agent-runtime)
+- [Custom Prompt Templates](#custom-prompt-templates)
+- [Custom Context Configurations](#custom-context-configurations)
+- [Vector Database Integration](#vector-database-integration-pgvector)
+- [Testing](#testing)
+- [Development](#development)
+
+## Quick Start
+
+Prescient reads provider credentials from the environment. The following
+example uses OpenAI; replace it with a configured provider when appropriate.
+
+```bash
+export OPENAI_API_KEY=your_api_key
+```
+
+Create `example.rb`:
+
+```ruby
+require "prescient"
+
+Prescient.configure do |config|
+  config.default_provider = :openai
+  config.add_provider(
+    :openai,
+    Prescient::Provider::OpenAI,
+    api_key: ENV.fetch("OPENAI_API_KEY"),
+    chat_model: ENV.fetch("OPENAI_CHAT_MODEL", "gpt-4.1-mini")
+  )
+end
+
+result = Prescient.generate_response(
+  "Draft a concise maintenance notice for a scheduled database upgrade."
+)
+
+puts result[:response]
+```
+
+Run it with:
+
+```bash
+bundle exec ruby example.rb
+```
+
+`generate_response` returns a normalized hash containing the generated
+response, provider, model, and provider metadata. Use the client directly when
+you need repeated operations or provider-specific fallback control:
+
+```ruby
+client = Prescient.client(:openai)
+response = client.generate_response("Summarize this release note.")
+embedding = client.generate_embedding("A searchable release note.")
+```
+
+For local development, configure Ollama instead of a hosted provider:
+
+```bash
+docker compose up -d ollama
+docker compose run --rm ollama-init
+```
+
+See the [examples guide](examples/README.md) for complete scripts and the
+[Configuration](#configuration) section for YAML, environment references,
+fallbacks, prompt templates, and external tools.
+
 ## Features
 
-- **Unified Interface**: Single API for multiple AI providers
-- **Local and Cloud Support**: Ollama for local/private deployments, cloud APIs for scale
-- **Embedding Generation**: Vector embeddings for semantic search and AI applications
-- **Text Completion**: Chat completions with context support
-- **Error Handling**: Robust error handling with automatic retries
-- **Health Monitoring**: Built-in health checks for all providers
-- **Flexible Configuration**: YAML, environment variable, and programmatic configuration
+- **Provider abstraction** — One consistent interface across supported AI providers
+- **Multiple interfaces** — Ruby API, CLI, and Rack-compatible REST API
+- **Text and embeddings** — Generate responses and embeddings with provider/model selection
+- **Reliability controls** — Retries, health checks, and fallback across configured providers
+- **Declarative configuration** — Versioned YAML, environment references, and JSON Schema validation
+- **Prompt and context customization** — Configurable prompt templates and context formatting
+- **External tools** — Explicit, normalized web-search integration with SearXNG and SearchApi
+- **Local and cloud support** — Ollama alongside hosted providers
+- **Optional integrations** — Docker deployment and pgvector support without making either mandatory
 
 ## Supported Providers
 
@@ -103,6 +194,8 @@ prescient config validate
 prescient config example
 prescient generate "Explain Ruby Ractors"
 prescient embed "Ruby is a programming language"
+prescient search "Ruby HTTP clients"
+prescient agent "Summarize the account status" --tool accounts
 ```
 
 Supported options include:
@@ -122,8 +215,15 @@ Supported options include:
 --api-key KEY            Use an API key for the operation
 --api-key-env NAME       Read the API key from an environment variable
 --format FORMAT          Select text or json output
+--json-file PATH         Load JSON documents as generation context
+--no-fallback             Disable provider fallback
 ```
 
+Search-specific options are `--tool NAME`, `--generate`, and `--limit COUNT`.
+Agent-specific options are repeatable `--tool NAME` and `--max-loops COUNT`.
+The full command-specific help is available with `prescient search --help` or
+`prescient agent --help`.
+
 Use `--api-key-env` to source credentials from an environment variable. The
 direct `--api-key` option is available for ephemeral automation but may be
 visible in shell history or process listings. Use `--format json` for
@@ -242,6 +342,9 @@ run Prescient::API.new(
 Available endpoints include:
 
 - **`POST /v1/generate`**
+- **`POST /v1/search`**
+- **`POST /v1/search/generate`**
+- **`POST /v1/agent`**
 - **`POST /v1/embeddings`**
 - **`POST /v1/embeddings/batch`**
 - **`GET /v1/providers`**
@@ -347,25 +450,139 @@ YAML values, then built-in defaults. Use `prescient config validate` to check a
 configuration before running an operation, or `prescient config example` to
 generate an annotated starter file.
 
-Prompt templates can also be configured per provider. Use the YAML mapping for
-multiline templates, or pass a template file to a single CLI operation:
+### External Tools
+
+External tools are opt-in capability adapters, separate from AI providers.
+Supported web-search adapters are [SearXNG](https://searxng.org/) and
+[SearchApi](https://www.searchapi.io/):
+
+#### SearXNG
+
+Setting `SEARXNG_URL` registers the default `web_search` tool for CLI and Ruby
+environment-based configuration. YAML or programmatic configuration can be
+used when more control is needed.
 
 ```yaml
-providers:
-  openai:
-    type: openai
-    api_key_env: OPENAI_API_KEY
-    chat_model: gpt-4.1-mini
-    prompt_templates:
-      system_prompt: You are a concise assistant.
-      no_context_template: "%{system_prompt}\n\nUser: %{query}"
-      with_context_template: "%{system_prompt}\n\nContext:\n%{context}\n\nUser: %{query}"
+tools:
+  web_search:
+    type: searxng
+    url_env: SEARXNG_URL
+    language: en
+    categories: [general, news]
+    timeout: 5
+    max_results: 5
 ```
 
+Run the SearXNG-backed tool:
+
 ```bash
-prescient generate --prompt-templates-file prompts.yml "Summarize this"
+docker compose up -d searxng
+SEARXNG_URL=http://localhost:8080 bundle exec prescient search \
+  --format json "Ruby HTTP clients"
+```
+
+#### SearchApi
+
+[SearchApi](https://www.searchapi.io/) uses a hosted Google search engine and
+requires an API key:
+
+```yaml
+tools:
+  web_search:
+    type: searchapi
+    api_key_env: SEARCHAPI_API_KEY
+    engine: google
+    location: New York
+    hl: en
+    gl: us
+    timeout: 10
+    max_results: 5
+```
+
+The `engine` value can select another SearchApi web or product engine when its
+response uses `organic_results`, such as `bing`, `yahoo`, `yandex`,
+`amazon_search`, or `walmart_search`.
+
+Adapters can be grouped under one logical capability for ordered fallback:
+
+```yaml
+tools:
+  web_search:
+    adapters:
+      - type: searxng
+        url_env: SEARXNG_URL
+      - type: searchapi
+        api_key_env: SEARCHAPI_API_KEY
+        engine: google
+```
+
+Adapters are tried in order. Fallback is limited to transient connection and
+rate-limit failures; invalid configuration, authentication failures, and
+malformed responses are not retried with another adapter.
+
+Run the SearchApi-backed tool:
+
+```bash
+SEARCHAPI_API_KEY=your-key bundle exec prescient search \
+  --config prescient.yml --format json "Ruby HTTP clients"
+```
+
+Invoke a configured tool explicitly from Ruby or the CLI:
+
+```ruby
+result = Prescient.tool(:web_search).search('Ruby HTTP clients', limit: 3)
+```
+
+```bash
+bundle exec prescient search --config prescient.yml \
+  --tool web_search --format json "Ruby HTTP clients"
+```
+
+Results use a normalized envelope containing `tool`, `query`, `source`, and
+`results` entries with `title`, `url`, `snippet`, and `source`. Requests have
+bounded query length, timeout, result count, and response size. Tool execution
+is explicit; Prescient does not autonomously invoke tools, and tool endpoints
+are not exposed as raw tool endpoints through `Prescient::API`; the API exposes
+the explicit combined search-and-generation operation below. Other adapters can
+implement the same contract without changing provider integrations.
+
+Search results are not sent to an AI provider by default. Opt in when you want
+the normalized results assembled as generation context:
+
+```ruby
+response = Prescient.search_and_generate(
+  'Ruby HTTP clients',
+  tool: :web_search,
+  provider: :openai,
+)
+```
+
+The CLI exposes the same opt-in behavior with `--generate`:
+
+```bash
+prescient search --generate --provider openai "Ruby HTTP clients"
+```
+
+The REST API exposes the same opt-in behavior:
+
+Raw normalized search results are available without generation:
+
+```bash
+curl -X POST http://localhost:9292/v1/search \
+  -H 'Content-Type: application/json' \
+  -d '{"query":"Ruby HTTP clients","limit":5}'
+```
+
+```bash
+curl -X POST http://localhost:9292/v1/search/generate \
+  -H 'Content-Type: application/json' \
+  -d '{"query":"Ruby HTTP clients","provider":"openai","limit":5}'
 ```
 
+Use `fallback: false` to disable provider fallback for the request. The
+response is the normalized AI provider response; omit this endpoint and use
+`POST /v1/generate` when search context is not wanted.
+
 ### Programmatic Configuration
 
 ```ruby
@@ -465,6 +682,117 @@ client_no_fallback = Prescient::Client.new(:primary, enable_fallback: false)
 response = Prescient.generate_response("Hello", provider: :primary, enable_fallback: true)
 ```
 
+### JSON document context
+
+For small documentation sets, you can load JSON objects from a file instead of
+creating embeddings:
+
+```ruby
+documents = Prescient::DocumentSource::JsonFile.new(path: "docs.json").fetch
+response = Prescient.generate_response("How do I reset my password?", documents)
+```
+
+The CLI supports the same flow with bounded document loading:
+
+```bash
+prescient generate --json-file docs.json "How do I reset my password?"
+```
+
+Redis-backed documents use an injected Redis-compatible client, so Redis remains
+optional for gem consumers:
+
+```ruby
+source = Prescient::DocumentSource::RedisJson.new(client: redis, key: "docs:acme")
+response = Prescient.generate_response("Summarize the docs", source.fetch)
+```
+
+The REST API accepts the same documents inline as `documents` on
+`POST /v1/generate`. Sources enforce document-count and serialized-byte limits;
+use retrieval or embeddings when the documentation no longer fits comfortably
+in the provider's context window.
+
+## Agent runtime
+
+Prescient also provides an optional, bounded agent runtime. It is loaded
+explicitly and can execute allowlisted search tools through the existing
+provider routing:
+
+```ruby
+require "prescient/agent"
+
+agent = Prescient::Agent::Runtime.new(
+  provider:   :openai,
+  tool_names: [:web_search],
+  configuration: Prescient::Agent::Configuration.new(max_loops: 5)
+)
+result = agent.run("Find and summarize the latest Ruby release")
+puts result.response
+```
+
+The runtime supports one validated JSON action per iteration and stops at its
+loop limit. It does not provide arbitrary Ruby, shell, filesystem, browser, or
+database access. The agent namespace is not loaded by `require "prescient"`.
+
+The same bounded runtime is available from the CLI:
+
+```bash
+prescient agent "Summarize the account status" --provider openai --tool accounts
+```
+
+It is also available through `POST /v1/agent`. Supply an explicit `tools` array;
+the REST API does not grant access to configured tools implicitly:
+
+```json
+{"prompt":"Summarize the account status","provider":"openai","tools":[]}
+```
+
+Applications can provide `authorization` and `request_context` hooks to enforce
+tenant/principal policy, and a bounded `telemetry` hook receives only event
+metadata such as loop count, action names, and success status. The authorization
+hook receives `tool:`, copied `arguments:`, and copied request-scoped
+`context:`; only an exact `true` result permits invocation. Prescient does not
+guess roles, ownership, or organization boundaries, so the host application
+must make the access decision.
+
+For durable, privacy-preserving audit records, pass an opt-in JSONL sink:
+
+```ruby
+audit_log = Prescient::Agent::AuditLog.new(path: "tmp/prescient-agent.jsonl")
+Prescient::Agent::Runtime.new(audit_log:).run("Summarize account status")
+```
+
+The sink persists event metadata and timestamps, never prompts, documents, or
+raw observations. Audit storage failures are isolated from agent execution;
+configure retention, access controls, and storage protection in the host.
+
+For MCP hosts, load the optional dependency-free adapter explicitly with
+`require "prescient/mcp"`, or run `prescient-mcp` for newline-delimited
+JSON-RPC over stdio. MCP exposes only explicitly enabled capabilities and never
+returns credentials or raw provider failure bodies.
+
+The optional Rack adapter provides a bounded Streamable HTTP surface with
+bearer-token authentication, Origin allowlisting, initialize-assigned sessions,
+`Mcp-Session-Id` lifecycle management, notifications, and one-shot JSON or
+server-sent-event responses:
+
+```ruby
+require "prescient/mcp"
+
+app = Prescient::MCP::Rack.new(
+  authentication: Prescient::MCP::Authentication::BearerToken.new(
+    token: ENV.fetch("MCP_TOKEN"),
+    principal: { id: "mcp-client" }
+  ),
+  allowed_origins: ["https://admin.example.com"]
+)
+run app
+```
+
+Clients must send `MCP-Protocol-Version: 2025-06-18` and the returned
+`Mcp-Session-Id` on requests after `initialize`. `DELETE` terminates a session.
+The adapter does not implement a long-lived notification queue or replace a
+host application's token provisioning and authorization policy.
+
 **Fallback Behavior:**
 - When a provider fails with a persistent error, Prescient automatically tries the next available provider
 - Configured fallback providers are tried in order; the provider operation determines availability
@@ -473,7 +801,7 @@ response = Prescient.generate_response("Hello", provider: :primary, enable_fallb
 - Provider-service failures, connection failures, rate limits, and unavailable models may trigger fallback; authentication and invalid-request errors are returned to the caller
 - The fallback process preserves all method arguments and options
 
-## Usage
+## Ruby API
 
 ### Quick Start
 
@@ -602,6 +930,24 @@ client = Prescient.client(:customer_service)
 response = client.generate_response("What's your return policy?")
 ```
 
+Templates can also be configured in YAML or overridden for one CLI operation:
+
+```yaml
+providers:
+  openai:
+    type: openai
+    api_key_env: OPENAI_API_KEY
+    chat_model: gpt-4.1-mini
+    prompt_templates:
+      system_prompt: You are a concise assistant.
+      no_context_template: "%{system_prompt}\n\nUser: %{query}"
+      with_context_template: "%{system_prompt}\n\nContext:\n%{context}\n\nUser: %{query}"
+```
+
+```bash
+prescient generate --prompt-templates-file prompts.yml "Summarize this"
+```
+
 ### Template Placeholders
 
 - `%{system_prompt}` - The system/role instruction
@@ -1132,6 +1478,8 @@ The included `docker-compose.yml` provides:
 
 - **ollama**: Ollama AI service with persistent model storage
 - **ollama-init**: Automatically pulls required models on startup
+- **searxng**: Optional SearXNG web-search service with JSON output enabled
+- **postgres**: Optional PostgreSQL database with pgvector support
 - **redis**: Optional caching layer for embeddings
 - **prescient-app**: Example Ruby application container
 

data/Steepfile

@@ -1,17 +1,17 @@
 # frozen_string_literal: true
 
 target :prescient do
-  signature 'sig'
-  library 'json'
-  library 'net-http'
+  signature "sig"
+  library "json"
+  library "net-http"
 
-  check 'lib/prescient/version.rb'
-  check 'lib/prescient/errors.rb'
-  check 'lib/prescient.rb'
-  check 'lib/prescient/client.rb'
-  check 'lib/prescient/base.rb'
-  check 'lib/prescient/provider/openai.rb'
-  check 'lib/prescient/provider/ollama.rb'
-  check 'lib/prescient/provider/anthropic.rb'
-  check 'lib/prescient/provider/huggingface.rb'
+  check "lib/prescient/version.rb"
+  check "lib/prescient/errors.rb"
+  check "lib/prescient.rb"
+  check "lib/prescient/client.rb"
+  check "lib/prescient/base.rb"
+  check "lib/prescient/provider/openai.rb"
+  check "lib/prescient/provider/ollama.rb"
+  check "lib/prescient/provider/anthropic.rb"
+  check "lib/prescient/provider/huggingface.rb"
 end

data/db/migrate/001_create_prescient_tables.rb

@@ -2,11 +2,10 @@
 
 # Rails migration for Prescient gem vector database tables
 # Copy this file to your Rails db/migrate directory and adjust the timestamp
-
 class CreatePrescientTables < ActiveRecord::Migration[7.0]
   def up
     # Enable pgvector extension
-    enable_extension 'vector'
+    enable_extension "vector"
 
     # Documents table to store original content
     create_table :documents do |t|
@@ -76,14 +75,14 @@ class CreatePrescientTables < ActiveRecord::Migration[7.0]
     add_index :documents, :metadata, using: :gin
 
     add_index :document_embeddings, :document_id
-    add_index :document_embeddings, [:embedding_provider, :embedding_model], name: 'idx_doc_embeddings_provider_model'
+    add_index :document_embeddings, %i[embedding_provider embedding_model], name: "idx_doc_embeddings_provider_model"
     add_index :document_embeddings, :embedding_dimensions
 
-    add_index :document_chunks, [:document_id, :chunk_index], unique: true
+    add_index :document_chunks, %i[document_id chunk_index], unique: true
 
     add_index :chunk_embeddings, :chunk_id
     add_index :chunk_embeddings, :document_id
-    add_index :chunk_embeddings, [:embedding_provider, :embedding_model], name: 'idx_chunk_embeddings_provider_model'
+    add_index :chunk_embeddings, %i[embedding_provider embedding_model], name: "idx_chunk_embeddings_provider_model"
 
     add_index :search_queries, :created_at
     add_index :query_results, :query_id
@@ -94,30 +93,30 @@ class CreatePrescientTables < ActiveRecord::Migration[7.0]
 
     # Vector indexes for document embeddings
     execute <<-SQL
-      CREATE INDEX idx_document_embeddings_cosine#{' '}
-      ON document_embeddings#{' '}
+      CREATE INDEX idx_document_embeddings_cosine#{" "}
+      ON document_embeddings#{" "}
       USING hnsw (embedding vector_cosine_ops)
       WITH (m = 16, ef_construction = 64);
     SQL
 
     execute <<-SQL
-      CREATE INDEX idx_document_embeddings_l2#{' '}
-      ON document_embeddings#{' '}
+      CREATE INDEX idx_document_embeddings_l2#{" "}
+      ON document_embeddings#{" "}
       USING hnsw (embedding vector_l2_ops)
       WITH (m = 16, ef_construction = 64);
     SQL
 
     # Vector indexes for chunk embeddings
     execute <<-SQL
-      CREATE INDEX idx_chunk_embeddings_cosine#{' '}
-      ON chunk_embeddings#{' '}
+      CREATE INDEX idx_chunk_embeddings_cosine#{" "}
+      ON chunk_embeddings#{" "}
       USING hnsw (embedding vector_cosine_ops)
       WITH (m = 16, ef_construction = 64);
     SQL
 
     execute <<-SQL
-      CREATE INDEX idx_chunk_embeddings_l2#{' '}
-      ON chunk_embeddings#{' '}
+      CREATE INDEX idx_chunk_embeddings_l2#{" "}
+      ON chunk_embeddings#{" "}
       USING hnsw (embedding vector_l2_ops)
       WITH (m = 16, ef_construction = 64);
     SQL
@@ -150,9 +149,9 @@ class CreatePrescientTables < ActiveRecord::Migration[7.0]
     drop_table :document_embeddings
     drop_table :documents
 
-    execute 'DROP FUNCTION IF EXISTS cosine_similarity(vector, vector);'
-    execute 'DROP FUNCTION IF EXISTS euclidean_distance(vector, vector);'
+    execute "DROP FUNCTION IF EXISTS cosine_similarity(vector, vector);"
+    execute "DROP FUNCTION IF EXISTS euclidean_distance(vector, vector);"
 
-    disable_extension 'vector'
+    disable_extension "vector"
   end
 end

data/docker-compose.yml

@@ -91,6 +91,26 @@ services:
       retries: 3
       start_period: 30s
 
+  # Optional SearXNG instance for the external web-search example.
+  searxng:
+    image: searxng/searxng:latest
+    container_name: prescient-searxng
+    ports:
+      - "8080:8080"
+    environment:
+      - SEARXNG_BASE_URL=http://localhost:8080/
+      - SEARXNG_SECRET=${SEARXNG_SECRET:-prescient-development-secret}
+    volumes:
+      - ./searxng/settings.yml:/etc/searxng/settings.yml:ro
+      - searxng_cache:/var/cache/searxng
+    restart: unless-stopped
+    healthcheck:
+      test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/')"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 30s
+
   # Optional: Redis for caching embeddings (useful for development)
   redis:
     image: redis:7-alpine
@@ -145,6 +165,8 @@ volumes:
     driver: local
   redis_data:
     driver: local
+  searxng_cache:
+    driver: local
 
 networks:
   default:

data/examples/README.md

@@ -18,7 +18,12 @@ bundle install
 - `vector_search.rb` — `Prescient::Pgvector::Store` PostgreSQL/pgvector storage
   and similarity search.
 - `rest_api.ru` — a tiny Rack-compatible application that mounts
-  `Prescient::API` and lists its endpoints at `/`.
+  `Prescient::API` and lists its endpoints at `/`, including the bounded agent
+  and search-generation routes.
+- `web_search.rb` — explicit SearXNG tool invocation with normalized JSON output.
+
+The same `web_search` capability can use SearchApi instead of SearXNG when the
+tool is configured with `type: searchapi` and `SEARCHAPI_API_KEY`.
 
 Run the REST API example with a Rack server such as `rackup`:
 
@@ -29,6 +34,17 @@ PRESCIENT_API_TOKEN=change-me BUNDLE_WITH=rack_example \
 curl http://localhost:9292/
 ```
 
+The endpoint catalog includes `POST /v1/search/generate`. With a configured
+SearXNG tool and AI provider, call it explicitly to feed search results into
+generation:
+
+```bash
+curl -X POST http://localhost:9292/v1/search/generate \
+  -H 'Authorization: Bearer change-me' \
+  -H 'Content-Type: application/json' \
+  -d '{"query":"Ruby HTTP clients","provider":"openai","limit":5}'
+```
+
 Running `bundle exec ruby examples/rest_api.ru` directly prints the same
 endpoint catalog without starting a server.
 
@@ -57,6 +73,25 @@ configuration. The scripts are demonstrations rather than isolated test
 fixtures; they may make real provider requests when the configured service is
 available.
 
+The web-search example requires a reachable SearXNG instance:
+
+```bash
+docker compose up -d searxng
+SEARXNG_URL=http://localhost:8080 bundle exec ruby examples/web_search.rb "Ruby HTTP clients"
+```
+
+The example returns normalized search results directly by default. Opt in to
+feeding those results to the configured AI provider with `--generate`:
+
+```bash
+SEARXNG_URL=http://localhost:8080 PRESCIENT_PROVIDER=openai \
+  bundle exec ruby examples/web_search.rb --generate "Ruby HTTP clients"
+```
+
+Omit `--generate` to keep the search results direct. `PRESCIENT_PROVIDER` is
+only used with `--generate` and may be omitted when the default provider is
+configured.
+
 See the [main README](../README.md) for configuration, fallback behavior,
 prompt templates, context exclusions, embeddings, and the public API. Rails
 applications can also use the [integration guide](../INTEGRATION_GUIDE.md),