prescient 0.3.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a54256e23ba55fc2492fae38d687b275dcb2362df73449d3212bd03659150717
-  data.tar.gz: 73ab805b7a5781a1d03488605ae8d7890feda7344b41b1ac550343484b3b597a
+  metadata.gz: 53a88a152c9fb689789c06dc11612905e30d76e997dbdcc1ca9906870bb01899
+  data.tar.gz: 4dba29dd0ad5ae0edbccd0f9d50d8e1eeb8815ba1aeadda838b327ec124111d6
 SHA512:
-  metadata.gz: e918a87b3e09affe9d5a002a3955051e7729ee565f84c42c534c99dc3fc70371fda930e63cc90801da07b4c4f5e48031c41e952088dfc3de24f0f38b579e9567
-  data.tar.gz: feb707947b7cc0b67d8bff933b4c584a69595b9b5e1dd3f214b4c4aa2d48c0577c996b542c238fbcef337f0f276a1bb995acc9653e8fad32262a6f6fa17caaf4
+  metadata.gz: 0beee5d0fc6ca63e1d2f881055402a18ab374d8ec71f17c6e706bba7a0ec8f565bbd77ff4f1b337f100597d357a07ffcc45b37e080429199603ce3147e292579
+  data.tar.gz: 2d8214bfb8d5f61536e5c79462d560cd4d6f83215dae54cffed821fe0f23d2959adbe02ec94f522d69bd2e04adb37b52087ba364752a98e093e6605ce204abfb

data/CHANGELOG.md

@@ -2,6 +2,40 @@
 
 ## Unreleased
 
+## [0.5.0] - 2025-08-14
+
+### Added
+
+- Added a versioned YAML configuration loader with environment-variable references,
+  configuration validation, precedence rules, and a packaged JSON Schema.
+- Added YAML-configurable prompt templates and CLI prompt overrides, including
+  support for loading multiline templates from a file.
+- Added `prescient config example` for generating an annotated schema-backed YAML configuration starter.
+- Added Google Gemini provider support for text generation, embeddings, health checks, and model listing.
+- Added Gemini environment-variable defaults and YAML configuration support.
+- Added Mistral provider support for text generation, embeddings, health checks, and model listing.
+- Added Mistral environment-variable defaults and YAML configuration support.
+- Added DeepSeek provider support for text generation, health checks, and model listing.
+- Documented DeepSeek's unsupported embedding capability explicitly.
+- Added xAI provider support for text generation, health checks, and model listing.
+- Documented xAI's unsupported embedding capability explicitly.
+
+### Changed
+
+- Raised YARD documentation coverage enforcement from 99% to 100%.
+
+## [0.4.0] - 2025-08-14
+
+### Added
+
+- Added the `prescient` CLI with provider listing, health checks, configuration validation, generation, embeddings, JSON output, stdin input, and exit-status handling.
+
+### Changed
+
+- Added CLI model overrides to provider generation and embedding operations.
+- Added secure CLI credential sourcing with `--api-key-env`, alongside direct `--api-key` support for ephemeral automation.
+- Documented all CLI automation overrides, including provider selection, generic and task-specific models, API keys, environment-backed credentials, and JSON output.
+
 ## [0.3.0] - 2025-08-14
 
 ### Added

data/INTEGRATION_GUIDE.md

@@ -13,7 +13,7 @@ and [examples guide](examples/README.md).
 # Add to your Gemfile
 gem 'prescient', path: './prescient_gem'  # Local development
 # OR when published:
-# gem 'prescient', '~> 0.3.0'
+# gem 'prescient', '~> 0.5.0'
 ```
 
 ### 2. Replace Existing AI Service
@@ -117,6 +117,17 @@ end
 Rails.application.config.default_ai_provider = :ollama
 ```
 
+For YAML-based deployments, use the versioned configuration format and keep
+credentials in environment variables:
+
+```ruby
+Prescient.load_configuration('prescient.yml')
+```
+
+Configuration precedence is CLI overrides, environment defaults and
+references, YAML values, then built-in defaults. The generated
+`prescient config example` file includes the current JSON Schema URL.
+
 ### 4. Update Environment Variables
 
 ```bash

data/README.md

@@ -1,10 +1,16 @@
 # Prescient
 
-Prescient is a boring AI provider abstraction for Ruby. Configure your AI providers once, then use the same interface regardless of whether the request is handled by OpenAI, Anthropic, Ollama, or Hugging Face. Prescient handles provider selection, retries, health checks, and fallback.
+[![Gem Version](https://img.shields.io/gem/v/prescient?logo=rubygems&logoColor=white)](https://rubygems.org/gems/prescient)
+[![Requires Ruby 3.1+](https://img.shields.io/badge/Requires-Ruby%203.1%2B-CC342D?logo=ruby&logoColor=white)](https://www.ruby-lang.org/)
+[![CI](https://github.com/kanutocd/prescient/actions/workflows/ci.yml/badge.svg)](https://github.com/kanutocd/prescient/actions/workflows/ci.yml)
+[![Security](https://img.shields.io/github/actions/workflow/status/kanutocd/prescient/security.yml?branch=main&event=push&label=Security)](https://github.com/kanutocd/prescient/actions/workflows/security.yml)
+[![License](https://img.shields.io/badge/License-MIT-22C55E)](LICENSE.txt)
 
-For focused guidance, see the [examples guide](examples/README.md),
-[Rails integration guide](INTEGRATION_GUIDE.md), and
-[pgvector guide](VECTOR_SEARCH_GUIDE.md).
+Prescient is a boring AI provider abstraction for Ruby. Configure your AI providers once, then use the same interface regardless of whether the request is handled by OpenAI, Anthropic, Ollama, Hugging Face, Google Gemini, Mistral, DeepSeek, or xAI. Prescient handles provider selection, retries, health checks, and fallback.
+
+For focused guidance, see the **[examples guide](https://github.com/kanutocd/prescient/tree/main/examples)**,
+**[Rails integration guide](https://github.com/kanutocd/prescient/blob/main/INTEGRATION_GUIDE.md)**, and
+**[pgvector guide](https://github.com/kanutocd/prescient/blob/main/VECTOR_SEARCH_GUIDE.md)**.
 
 ## Features
 
@@ -14,7 +20,7 @@ For focused guidance, see the [examples guide](examples/README.md),
 - **Text Completion**: Chat completions with context support
 - **Error Handling**: Robust error handling with automatic retries
 - **Health Monitoring**: Built-in health checks for all providers
-- **Flexible Configuration**: Environment variable and programmatic configuration
+- **Flexible Configuration**: YAML, environment variable, and programmatic configuration
 
 ## Supported Providers
 
@@ -42,6 +48,30 @@ For focused guidance, see the [examples guide](examples/README.md),
 - **Capabilities**: Embeddings, Text Generation
 - **Use Case**: Open-source models, research
 
+### Google Gemini
+
+- **Models**: Gemini generation and embedding models
+- **Capabilities**: Embeddings, Text Generation, Model Listing
+- **Use Case**: Google AI hosted models
+
+### Mistral
+
+- **Models**: Mistral chat and embedding models
+- **Capabilities**: Embeddings, Text Generation, Model Listing
+- **Use Case**: Mistral AI hosted models
+
+### DeepSeek
+
+- **Models**: DeepSeek chat models
+- **Capabilities**: Text Generation, Model Listing (no embeddings)
+- **Use Case**: DeepSeek hosted reasoning and chat models
+
+### xAI
+
+- **Models**: Grok chat models
+- **Capabilities**: Text Generation, Model Listing (no embeddings)
+- **Use Case**: xAI hosted reasoning and chat models
+
 ## Installation
 
 Add this line to your application's Gemfile:
@@ -62,6 +92,137 @@ Or install it yourself as:
 gem install prescient
 ```
 
+## Command-Line Interface
+
+Prescient includes a thin CLI for provider inspection and common operations:
+
+```bash
+prescient providers
+prescient health
+prescient config validate
+prescient config example
+prescient generate "Explain Ruby Ractors"
+prescient embed "Ruby is a programming language"
+```
+
+Supported options include:
+
+```text
+--provider NAME          Select a provider
+--model NAME             Override the selected operation's model
+--chat-model NAME        Override the chat model for generation
+--embedding-model NAME   Override the embedding model
+--system-prompt TEXT     Override the system prompt
+--no-context-template TEXT
+                         Override the no-context prompt template
+--with-context-template TEXT
+                         Override the with-context prompt template
+--prompt-templates-file PATH
+                         Load prompt templates from a YAML file
+--api-key KEY            Use an API key for the operation
+--api-key-env NAME       Read the API key from an environment variable
+--format FORMAT          Select text or json output
+```
+
+Use `--api-key-env` to source credentials from an environment variable. The
+direct `--api-key` option is available for ephemeral automation but may be
+visible in shell history or process listings. Use `--format json` for
+machine-readable output and stdin for shell pipelines:
+
+```bash
+printf '%s' "Explain PostgreSQL logical replication" | \
+  prescient generate --provider openai --format json
+```
+
+OpenAI example JSON output:
+
+```json
+{
+    "response": "PostgreSQL logical replication is a method of replicating data between PostgreSQL databases at a logical level, allowing fine-grained control over which data is replicated and how. Unlike physical replication, which copies the entire database cluster’s data files at the storage level, logical replication works by sending changes to data (such as INSERT, UPDATE, DELETE operations) based on logical changes in the database.\n\n### Key Features of PostgreSQL Logical Replication\n\n1. **Row-Level Replication:** Logical replication replicates data changes at the row level. It streams changes to individual tables rather than the entire database.\n\n2. **Selective Replication:** You can choose specific tables to replicate rather than the whole database. This makes it useful for replicating subsets of data.\n\n3. **Asynchronous Replication:** Changes are sent asynchronously from the publisher (source) to the subscriber (target). This means there may be a slight delay between when changes are made and when they appear on the subscriber.\n\n4. **Supports Heterogeneous Setups:** Logical replication can be used between different major versions of PostgreSQL, allowing upgrades with minimal downtime. It can also be used for replication between different architectures or operating systems.\n\n5. **Bidirectional Replication:** By configuring multiple publishers and subscribers, logical replication can support multi-master setups, although care must be taken to avoid conflicts.\n\n### How Logical Replication Works\n\n- **Publisher:** The database that sends data changes. It defines one or more publications, which specify which tables and changes (inserts, updates, deletes) to replicate.\n  \n- **Subscriber:** The database that receives and applies the changes. It subscribes to one or more publications from the publisher.\n\nWhen a change occurs on the publisher's table, the change is captured and sent to the subscriber, where it is applied to the corresponding table.\n\n### Setting Up Logical Replication (Basic Steps)\n\n1. **Enable required settings:** Ensure the PostgreSQL server has `wal_level` set to `logical`, and configure `max_replication_slots` and `max_wal_senders` appropriately.\n\n2. **Create a publication on the publisher:**\n\n   ```sql\n   CREATE PUBLICATION my_publication FOR TABLE my_table;\n   ```\n\n3. **Create a subscription on the subscriber:**\n\n   ```sql\n   CREATE SUBSCRIPTION my_subscription\n   CONNECTION 'host=publisher_host dbname=publisher_db user=replicator password=secret'\n   PUBLICATION my_publication;\n   ```\n\nOnce set up, changes to `my_table` on the publisher will be replicated to the subscriber.\n\n### Use Cases\n\n- **Selective data replication:** Replicating only certain tables or rows.\n- **Data integration:** Feeding data from multiple sources into a central database.\n- **Upgrading PostgreSQL versions:** Using logical replication to migrate data with minimal downtime.\n- **Multi-datacenter replication:** Replicating data across geographically distributed systems.\n\n---\n\nIn summary, PostgreSQL logical replication is a flexible, table-level replication mechanism that allows selective, asynchronous replication of data changes between PostgreSQL databases, useful for upgrades, distributed architectures, and data integration scenarios.",
+    "model": "gpt-4.1-mini",
+    "provider": "openai",
+    "processing_time": null,
+    "metadata": {
+        "usage": {
+            "prompt_tokens": 38,
+            "completion_tokens": 632,
+            "total_tokens": 670,
+            "prompt_tokens_details": {
+                "cached_tokens": 0,
+                "audio_tokens": 0
+            },
+            "completion_tokens_details": {
+                "reasoning_tokens": 0,
+                "audio_tokens": 0,
+                "accepted_prediction_tokens": 0,
+                "rejected_prediction_tokens": 0
+            }
+        },
+        "finish_reason": "stop"
+    }
+}
+```
+
+Anthropic example JSON output:
+
+```json
+{
+    "response": "PostgreSQL logical replication is a method of replicating data between PostgreSQL databases that allows fine-grained control over which data is replicated and how it is applied. Unlike physical replication, which copies the entire database cluster at the storage level, logical replication works at the level of individual database changes, such as INSERT, UPDATE, and DELETE operations.\n\n### Key Features of PostgreSQL Logical Replication:\n\n1. **Row-Level Changes:** Logical replication replicates changes at the row level, meaning only the actual data changes are sent to the subscriber.\n\n2. **Selective Replication:** You can replicate specific tables rather than the entire database. This allows partial replication tailored to your needs.\n\n3. **Asynchronous Replication:** Logical replication is asynchronous, so there might be a slight delay between the publisher and subscriber.\n\n4. **Bidirectional Replication (with care):** While PostgreSQL does not natively support multi-master replication, logical replication can be configured to allow bidirectional replication setups with caution to avoid conflicts.\n\n5. **Decoupling of Replication:** Logical replication decouples the replication from the physical storage, enabling replication across different PostgreSQL versions (within compatibility limits).\n\n### How Logical Replication Works:\n\n- **Publisher:** The source database that sends changes. It publishes a set of changes based on one or more publications.\n- **Publication:** A set of changes (typically from specific tables) that the publisher makes available to subscribers.\n- **Subscriber:** The target database that receives changes and applies them.\n- **Subscription:** A configuration on the subscriber that connects to a publication and applies changes.\n\n### Use Cases:\n\n- Replicating specific tables or subsets of data.\n- Migrating data between PostgreSQL versions or clusters.\n- Distributing data geographically.\n- Implementing data warehousing or reporting solutions with up-to-date data.\n- Supporting microservices architectures where different services own different parts of the data.\n\n### Basic Setup Example:\n\n1. **On the Publisher:**\n\n```sql\nCREATE PUBLICATION my_publication FOR TABLE my_table;\n```\n\n2. **On the Subscriber:**\n\n```sql\nCREATE SUBSCRIPTION my_subscription\nCONNECTION 'host=publisher_host dbname=mydb user=replicator password=secret'\nPUBLICATION my_publication;\n```\n\nOnce set up, changes to `my_table` on the publisher will be sent and applied to the subscriber.\n\n### Important Notes:\n\n- Logical replication requires WAL (Write-Ahead Logging) to be configured properly with `wal_level = logical`.\n- Some DDL changes (like adding columns) need careful handling as logical replication primarily replicates DML changes.\n- Logical replication does not replicate sequences, large objects, or certain system catalogs automatically.\n- Conflict resolution is mostly manual; the subscriber applies changes as received.\n\n---\n\nIn summary, PostgreSQL logical replication provides a flexible, table-level replication mechanism that supports selective and version-independent replication of data changes, suitable for many modern replication and data distribution scenarios.",
+    "model": "gpt-4.1-mini",
+    "provider": "openai",
+    "processing_time": null,
+    "metadata": {
+        "usage": {
+            "prompt_tokens": 38,
+            "completion_tokens": 597,
+            "total_tokens": 635,
+            "prompt_tokens_details": {
+                "cached_tokens": 0,
+                "audio_tokens": 0
+            },
+            "completion_tokens_details": {
+                "reasoning_tokens": 0,
+                "audio_tokens": 0,
+                "accepted_prediction_tokens": 0,
+                "rejected_prediction_tokens": 0
+            }
+        },
+        "finish_reason": "stop"
+    }
+}
+```
+
+For automated model and credential overrides:
+
+```bash
+prescient generate \
+  --provider openai \
+  --chat-model gpt-4.1-mini \
+  --api-key-env OPENAI_API_KEY \
+  --format json \
+  "Explain PostgreSQL logical replication"
+
+prescient embed \
+  --provider openai \
+  --embedding-model text-embedding-3-small \
+  --api-key-env OPENAI_API_KEY \
+  "Ruby is a programming language"
+```
+
+The CLI writes results to stdout, diagnostics to stderr, and returns a
+non-zero status for invalid usage, provider errors, or unreachable health
+checks. It uses the same `Prescient::Client` execution path as Ruby callers.
+
+Generate a schema-backed, annotated starter configuration with:
+
+```bash
+prescient config example > prescient.yml
+```
+
+The generated file points YAML language servers at the latest schema on the
+main branch. Pin the schema URL to a release tag when reproducible tooling is
+required.
+
 ## Configuration
 
 ### Environment Variables
@@ -85,6 +246,68 @@ OPENAI_CHAT_MODEL=gpt-4.1-mini
 HUGGINGFACE_API_KEY=your_api_key
 HUGGINGFACE_EMBEDDING_MODEL=sentence-transformers/all-MiniLM-L6-v2
 HUGGINGFACE_CHAT_MODEL=google/gemma-2-2b-it
+
+# Google Gemini
+GEMINI_API_KEY=your_api_key
+GEMINI_EMBEDDING_MODEL=gemini-embedding-001
+GEMINI_CHAT_MODEL=gemini-2.5-flash
+
+# Mistral
+MISTRAL_API_KEY=your_api_key
+MISTRAL_EMBEDDING_MODEL=mistral-embed
+MISTRAL_CHAT_MODEL=mistral-large-latest
+
+# DeepSeek
+DEEPSEEK_API_KEY=your_api_key
+DEEPSEEK_CHAT_MODEL=deepseek-v4-flash
+
+# xAI
+XAI_API_KEY=your_api_key
+XAI_CHAT_MODEL=grok-4.5
+```
+
+### YAML Configuration
+
+Load a versioned YAML configuration with environment-backed credentials:
+
+```yaml
+# yaml-language-server: $schema=https://raw.githubusercontent.com/kanutocd/prescient/refs/heads/main/schema/prescient.configuration.schema.json
+version: 1
+default_provider: openai
+providers:
+  openai:
+    type: openai
+    api_key_env: OPENAI_API_KEY
+    embedding_model: text-embedding-3-small
+    chat_model: gpt-4.1-mini
+```
+
+```ruby
+Prescient.load_configuration('prescient.yml')
+```
+
+Configuration precedence is CLI overrides, environment defaults and references,
+YAML values, then built-in defaults. Use `prescient config validate` to check a
+configuration before running an operation, or `prescient config example` to
+generate an annotated starter file.
+
+Prompt templates can also be configured per provider. Use the YAML mapping for
+multiline templates, or pass a template file to a single CLI operation:
+
+```yaml
+providers:
+  openai:
+    type: openai
+    api_key_env: OPENAI_API_KEY
+    chat_model: gpt-4.1-mini
+    prompt_templates:
+      system_prompt: You are a concise assistant.
+      no_context_template: "%{system_prompt}\n\nUser: %{query}"
+      with_context_template: "%{system_prompt}\n\nContext:\n%{context}\n\nUser: %{query}"
+```
+
+```bash
+prescient generate --prompt-templates-file prompts.yml "Summarize this"
 ```
 
 ### Programmatic Configuration
@@ -118,6 +341,32 @@ Prescient.configure do |config|
     embedding_model: 'text-embedding-3-small',
     chat_model: 'gpt-4.1-mini'
   )
+
+  # Add Google Gemini
+  config.add_provider(:gemini, Prescient::Provider::Gemini,
+    api_key: ENV['GEMINI_API_KEY'],
+    embedding_model: 'gemini-embedding-001',
+    chat_model: 'gemini-2.5-flash'
+  )
+
+  # Add Mistral
+  config.add_provider(:mistral, Prescient::Provider::Mistral,
+    api_key: ENV['MISTRAL_API_KEY'],
+    embedding_model: 'mistral-embed',
+    chat_model: 'mistral-large-latest'
+  )
+
+  # Add DeepSeek
+  config.add_provider(:deepseek, Prescient::Provider::DeepSeek,
+    api_key: ENV['DEEPSEEK_API_KEY'],
+    chat_model: 'deepseek-v4-flash'
+  )
+
+  # Add xAI
+  config.add_provider(:xai, Prescient::Provider::XAI,
+    api_key: ENV['XAI_API_KEY'],
+    chat_model: 'grok-4.5'
+  )
 end
 ```
 
@@ -273,14 +522,14 @@ Prescient.configure do |config|
     prompt_templates: {
       system_prompt: 'You are a friendly customer service representative.',
       no_context_template: <<~TEMPLATE.strip,
-        %{ system_prompt }
+        %{system_prompt}
 
         Customer Question: %{query}
 
         Please provide a helpful response.
       TEMPLATE
       with_context_template: <<~TEMPLATE.strip
-        %{ system_prompt } Use the company info below to help answer.
+        %{system_prompt} Use the company info below to help answer.
 
         Company Information:
         %{context}
@@ -450,8 +699,9 @@ results = store.search(embedding:, provider: :openai, model: 'text-embedding-3-s
 ```
 
 Every vector must exactly match the store's configured dimensions; Prescient
-never pads or truncates vectors. The existing application-schema example below
-remains available for projects that need documents, chunks, and custom metadata.
+never pads or truncates vectors. The application-schema example below is
+optional integration material for projects that need documents, chunks, and
+custom metadata; it is not managed by `Prescient::Pgvector::Store`.
 
 ### Setup with Docker
 
@@ -470,7 +720,7 @@ docker compose up -d postgres
 
 ### Database Schema
 
-The setup creates these key tables:
+The optional Docker demo creates these application-owned tables:
 
 - **`documents`** - Store original content and metadata
 - **`document_embeddings`** - Store vector embeddings for documents
@@ -652,10 +902,9 @@ DB_HOST=localhost ruby examples/vector_search.rb
 
 The example demonstrates:
 
-- Document embedding generation and storage
-- Similarity search with different distance functions
-- Metadata filtering and advanced queries
-- Performance comparison between approaches
+- Embedding generation through `Prescient::Client`
+- Dimension-validated storage through `Prescient::Pgvector::Store`
+- Similarity search with provider and model filters
 
 ## Advanced Usage
 

data/Rakefile

@@ -32,7 +32,7 @@ namespace :yard do
     abort("Unable to determine YARD coverage") unless match
 
     coverage = match[1].to_f
-    minimum = 99.0
+    minimum = 100.0
     if coverage < minimum
       message = format(
         "YARD coverage %<coverage>.2f%% is below %<minimum>.2f%%",

data/VECTOR_SEARCH_GUIDE.md

@@ -27,6 +27,10 @@ store.create_index!(metric: :cosine)
 or chunk tables, connections, migrations outside that table, or the `pg` gem.
 Use `#upsert` and `#search` with embeddings of exactly the configured dimension.
 
+The remaining sections describe an optional application-owned document schema
+used by the repository's Docker demo. They are not tables created or managed
+by `Prescient::Pgvector::Store`.
+
 ### 1. Start Services
 
 ```bash
@@ -55,12 +59,12 @@ export DB_HOST=localhost
 export OLLAMA_URL=http://localhost:11434
 
 # Run the example
-ruby examples/vector_search.rb
+bundle exec ruby examples/vector_search.rb
 ```
 
 ## Architecture Overview
 
-### Database Schema
+### Application-Owned Example Schema
 
 ```
 documents
@@ -103,7 +107,7 @@ chunk_embeddings
 
 ### Vector Indexes
 
-The setup automatically creates HNSW indexes for optimal performance:
+The optional Docker demo creates HNSW indexes for the application-owned tables:
 
 - **Cosine Distance**: `embedding <=> query_vector`
 - **L2 Distance**: `embedding <-> query_vector`

data/examples/README.md

@@ -15,7 +15,8 @@ bundle install
 - `custom_prompts.rb` — system prompts and no-context/with-context templates.
 - `custom_contexts.rb` — explicit context types, field matching, formatting,
   and embedding field selection.
-- `vector_search.rb` — PostgreSQL/pgvector storage and similarity search.
+- `vector_search.rb` — `Prescient::Pgvector::Store` PostgreSQL/pgvector storage
+  and similarity search.
 
 The first three examples use Ollama by default. Start Ollama and pull the
 current local models before running them:

data/examples/basic_usage.rb

@@ -67,7 +67,7 @@ end
 # Example 3: Provider comparison (if multiple providers configured)
 puts "\n=== Example 3: Provider Health Check ==="
 
-providers = [:ollama, :anthropic, :openai, :huggingface]
+providers = %i[ollama anthropic openai huggingface gemini mistral deepseek xai]
 
 providers.each do |provider_name|
   begin

data/examples/custom_contexts.rb

@@ -228,8 +228,8 @@ rescue Prescient::Error => e
   puts "❌ Error: #{e.message}"
 end
 
-# Example 4: Embedding Text Extraction
-puts "\n--- Example 4: Embedding Text Extraction ---"
+# Example 4: Embedding Text Selection
+puts "\n--- Example 4: Embedding Text Selection ---"
 
 begin
   # Configure a provider with context configs
@@ -251,7 +251,7 @@ begin
   client = Prescient.client(:embedding_demo)
   
   if client.available?
-    # Test embedding text extraction
+    # The configured embedding_fields select title, content, and tags.
     blog_post = {
       'type' => 'blog_post',
       'title' => 'Getting Started with AI',
@@ -262,12 +262,10 @@ begin
       'publish_date' => '2024-01-15'
     }
 
-    # The extract_embedding_text method will only use title, content, and tags
-    # This demonstrates how embedding generation can focus on specific fields
-    embedding_text = client.provider.send(:extract_embedding_text, blog_post)
-    puts "📊 Embedding Text Extracted:"
+    embedding_text = [blog_post['title'], blog_post['content'], blog_post['tags']].join(' ')
+    puts "📊 Embedding Text Selected:"
     puts "\"#{embedding_text}\""
-    puts "\n(Notice how only title, content, and tags are included - not author, category, or date)"
+    puts "\n(Only title, content, and tags are included in the embedding input)"
 
     # Generate actual embedding
     puts "\n🔢 Generating embedding..."
@@ -317,19 +315,6 @@ begin
     puts "🔧 Raw data (no context config):"
     random_data.each { |item| puts "   #{item}" }
     
-    puts "\n📄 How items are formatted without context config:"
-    random_data.each do |item|
-      formatted = client.provider.send(:format_context_item, item)
-      puts "   #{formatted}"
-    end
-    
-    puts "\n🔤 Embedding text extraction (automatic field filtering):"
-    random_data.each do |item|
-      embedding_text = client.provider.send(:extract_embedding_text, item)
-      puts "   \"#{embedding_text}\""
-      puts "   (Notice: excludes 'created_at', 'timestamp' - common metadata fields)"
-    end
-    
     response = client.generate_response("Summarize the key issues", random_data)
     puts "\n🤖 AI Response (using default formatting):"
     puts response[:response]