htm 0.0.17 → 0.0.18

data/examples/config_file_example/README.md

@@ -0,0 +1,256 @@
+# HTM Configuration Example
+
+This example demonstrates how HTM uses the `anyway_config` gem for flexible, layered configuration management.
+
+## Configuration Sources (Priority Order)
+
+HTM loads configuration from multiple sources, merged in order from lowest to highest priority:
+
+| Priority | Source | Location | Description |
+|----------|--------|----------|-------------|
+| 1 | Bundled Defaults | `lib/htm/config/defaults.yml` | Ships with the gem, defines schema |
+| 2 | XDG User Config | `~/.config/htm/htm.yml` | User-wide settings |
+| 3 | Project Config | `./config/htm.yml` | Project-specific settings |
+| 4 | Local Overrides | `./config/htm.local.yml` | Local dev settings (gitignored) |
+| 5 | HTM_CONF File | Path in `HTM_CONF` env var | Custom config file path |
+| 6 | Environment Variables | `HTM_*` | Runtime overrides |
+| 7 | Code Block | `HTM.configure { }` | Programmatic configuration |
+
+Higher priority sources override values from lower priority sources.
+
+## Generating a Config File
+
+> **Tip:** Use the `htm_mcp config` command to output the complete default configuration to STDOUT. Redirect it to any location with any filename:
+>
+> ```bash
+> # Create a project config file
+> htm_mcp config > ./config/htm.yml
+>
+> # Create a local overrides file
+> htm_mcp config > ./config/htm.local.yml
+>
+> # Create a user-wide XDG config
+> mkdir -p ~/.config/htm
+> htm_mcp config > ~/.config/htm/htm.yml
+>
+> # Create a custom config for HTM_CONF
+> htm_mcp config > /path/to/my_custom_config.yml
+> ```
+>
+> This gives you a complete, documented template with all available settings that you can customize for your needs.
+
+## Standard Config File Locations
+
+### 1. Bundled Defaults (`lib/htm/config/defaults.yml`)
+
+The gem ships with a defaults file that defines the complete configuration schema and sensible defaults. This is always loaded first.
+
+### 2. XDG User Config (`~/.config/htm/htm.yml`)
+
+User-wide configuration following the [XDG Base Directory Specification](https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html). On macOS, also checks `~/Library/Application Support/htm/htm.yml`.
+
+```yaml
+# ~/.config/htm/htm.yml
+embedding:
+  provider: openai
+  model: text-embedding-3-small
+
+providers:
+  openai:
+    api_key: sk-your-api-key
+```
+
+### 3. Project Config (`./config/htm.yml`)
+
+Project-specific configuration. Supports environment-specific sections:
+
+```yaml
+# ./config/htm.yml
+defaults:
+  database:
+    host: localhost
+    port: 5432
+
+development:
+  database:
+    name: myapp_development
+
+production:
+  database:
+    name: myapp_production
+    pool_size: 25
+```
+
+### 4. Local Overrides (`./config/htm.local.yml`)
+
+Developer-specific overrides, typically added to `.gitignore`:
+
+```yaml
+# ./config/htm.local.yml
+database:
+  name: my_local_db
+
+log_level: debug
+```
+
+## Using HTM_CONF for Custom Config Files
+
+The `HTM_CONF` environment variable lets you specify any YAML file path:
+
+```bash
+# Relative path
+HTM_CONF=./custom_config.yml ruby show_config.rb
+
+# Absolute path
+HTM_CONF=/etc/htm/production.yml ruby show_config.rb
+
+# Different config per environment
+HTM_CONF=./config/staging.yml HTM_ENV=staging ruby show_config.rb
+```
+
+This is useful for:
+- Testing different configurations
+- CI/CD pipelines with config files in non-standard locations
+- Docker deployments with mounted config files
+
+## Environment Variables
+
+Environment variables use double underscores (`__`) for nested values:
+
+```bash
+# Database configuration
+export HTM_DATABASE__URL="postgresql://user:pass@host:5432/dbname"
+export HTM_DATABASE__POOL_SIZE=25
+
+# Embedding configuration
+export HTM_EMBEDDING__PROVIDER=openai
+export HTM_EMBEDDING__MODEL=text-embedding-3-small
+export HTM_EMBEDDING__DIMENSIONS=1536
+
+# Provider credentials
+export HTM_PROVIDERS__OPENAI__API_KEY=sk-your-key
+export HTM_PROVIDERS__OLLAMA__URL=http://localhost:11434
+
+# General settings
+export HTM_LOG_LEVEL=debug
+export HTM_TELEMETRY_ENABLED=true
+```
+
+## Code-Level Configuration
+
+The highest priority configuration is done programmatically:
+
+```ruby
+require 'htm'
+
+HTM.configure do |config|
+  # Database
+  config.database.host = 'localhost'
+  config.database.name = 'my_database'
+
+  # Embedding provider
+  config.embedding.provider = :openai
+  config.embedding.model = 'text-embedding-3-small'
+  config.embedding.dimensions = 1536
+
+  # Tag extraction
+  config.tag.provider = :anthropic
+  config.tag.model = 'claude-3-haiku-20240307'
+
+  # Provider credentials
+  config.providers.openai.api_key = ENV['OPENAI_API_KEY']
+
+  # General settings
+  config.log_level = :debug
+  config.telemetry_enabled = false
+
+  # Custom callables
+  config.logger = Logger.new($stdout)
+end
+```
+
+## Environment Selection
+
+HTM detects the environment using these variables (in order):
+
+1. `HTM_ENV`
+2. `RAILS_ENV`
+3. `RACK_ENV`
+4. Defaults to `development`
+
+```bash
+HTM_ENV=production ruby show_config.rb
+HTM_ENV=test ruby show_config.rb
+```
+
+## Running the Example
+
+```bash
+cd examples/config_file_example
+
+# Basic usage - loads ./config/htm.local.yml automatically
+ruby show_config.rb
+
+# Different environment
+HTM_ENV=production ruby show_config.rb
+
+# Override with environment variable
+HTM_EMBEDDING__MODEL=mxbai-embed-large ruby show_config.rb
+
+# Use custom config file
+HTM_CONF=./custom_config.yml ruby show_config.rb
+
+# Combine multiple overrides
+HTM_CONF=./custom_config.yml HTM_ENV=test HTM_LOG_LEVEL=warn ruby show_config.rb
+```
+
+## Example Output
+
+The `show_config.rb` script displays the loaded configuration with source tracing:
+
+```
+HTM Configuration Example
+============================================================
+
+Environment: development
+
+Config sources checked:
+  - Bundled defaults: lib/htm/config/defaults.yml
+  - XDG config: ~/.config/htm/htm.yml
+  - Project config: ./config/htm.yml
+  - Local overrides: ./config/htm.local.yml
+  - HTM_CONF override: (not set)
+  - Environment variables: HTM_*
+
+------------------------------------------------------------
+Configuration with Sources:
+------------------------------------------------------------
+
+database:
+  host: "localhost"  # from: htm.local.yml
+  port: 5432  # from: htm.local.yml
+  name: "htm_example"  # from: htm.local.yml
+  pool_size: 10  # from: bundled_defaults
+embedding:
+  provider: "ollama"  # from: htm.local.yml
+  model: "nomic-embed-text:latest"  # from: htm.local.yml
+  dimensions: 768  # from: bundled_defaults
+...
+```
+
+## Files in This Example
+
+```
+config_file_example/
+├── README.md              # This file
+├── show_config.rb         # Demo script with source tracing
+├── custom_config.yml      # Example for HTM_CONF usage
+└── config/
+    └── htm.local.yml      # Auto-loaded local overrides
+```
+
+## See Also
+
+- [anyway_config documentation](https://github.com/palkan/anyway_config)
+- [HTM Configuration Guide](../../docs/configuration.md)
+- [HTM README](../../README.md)

data/examples/config_file_example/config/htm.local.yml

@@ -0,0 +1,34 @@
+# HTM Local Configuration Example
+#
+# This file demonstrates local configuration overrides.
+# Place in ./config/htm.local.yml (typically gitignored)
+#
+# Bundled defaults are loaded automatically from:
+#   lib/htm/config/defaults.yml
+#
+# This file overrides specific values for local development.
+
+# Override database settings for this example
+database:
+  host: localhost
+  port: 5432
+  name: htm_example
+
+# Use local Ollama for embeddings
+embedding:
+  provider: ollama
+  model: nomic-embed-text:latest
+
+# Use local Ollama for tag extraction
+tag:
+  provider: ollama
+  model: gemma3:latest
+
+# Local development settings
+log_level: debug
+telemetry_enabled: false
+
+# Provider endpoints
+providers:
+  ollama:
+    url: http://localhost:11434

data/examples/config_file_example/custom_config.yml

@@ -0,0 +1,22 @@
+# Custom HTM Configuration
+#
+# Use this file with: HTM_CONF=custom_config.yml ruby show_config.rb
+#
+# This demonstrates loading config from an arbitrary file path
+# using the HTM_CONF environment variable.
+
+database:
+  host: custom-db.example.com
+  port: 5433
+  name: htm_custom
+
+embedding:
+  provider: openai
+  model: text-embedding-3-small
+  dimensions: 1536
+
+tag:
+  provider: anthropic
+  model: claude-3-haiku-20240307
+
+log_level: warn

data/examples/config_file_example/show_config.rb

@@ -0,0 +1,125 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Show Config Example
+#
+# Demonstrates how anyway_config automatically loads configuration
+# from standard locations and traces the source of each value.
+#
+# Config loading priority (lowest to highest):
+#   1. Bundled defaults: lib/htm/config/defaults.yml (ships with gem)
+#   2. XDG user config: ~/.config/htm/htm.yml
+#   3. Project config: ./config/htm.yml
+#   4. Local overrides: ./config/htm.local.yml  <-- This example uses this
+#   5. HTM_CONF file: path specified by HTM_CONF env var (overrides above)
+#   6. Environment variables (HTM_*)
+#
+# Usage:
+#   ruby show_config.rb
+#   HTM_ENV=production ruby show_config.rb
+#   HTM_EMBEDDING__MODEL=mxbai-embed-large ruby show_config.rb
+#   HTM_CONF=/path/to/custom.yml ruby show_config.rb
+
+require_relative '../../lib/htm'
+require 'amazing_print'
+
+puts <<~HEADER
+
+  HTM Configuration Example
+  #{'=' * 60}
+
+  Environment: #{HTM.config.environment}
+
+  Config sources checked:
+    - Bundled defaults: lib/htm/config/defaults.yml
+    - XDG config: #{HTM::Config.xdg_config_file}
+    - Project config: ./config/htm.yml
+    - Local overrides: ./config/htm.local.yml
+    - HTM_CONF override: #{ENV['HTM_CONF'] || '(not set)'}
+    - Environment variables: HTM_*
+
+  Active XDG config: #{HTM::Config.active_xdg_config_file || '(none found)'}
+
+  #{'-' * 60}
+  Configuration with Sources:
+  #{'-' * 60}
+
+HEADER
+
+# Get the source trace from anyway_config
+trace = HTM.config.to_source_trace
+
+# Helper to format source information
+def format_source(source)
+  return "unknown" unless source
+
+  case source[:type]
+  when :defaults
+    "defaults"
+  when :yml
+    path = source[:path]
+    # Shorten the path for readability
+    if path.include?('defaults.yml')
+      "defaults.yml"
+    elsif path.include?('htm.local.yml')
+      "htm.local.yml"
+    elsif path.include?('htm.yml')
+      "htm.yml"
+    else
+      File.basename(path)
+    end
+  when :env
+    "ENV[#{source[:key]}]"
+  when :user
+    "code"
+  else
+    source[:type].to_s
+  end
+end
+
+# Recursively print config with sources
+def print_config(trace, indent = 0)
+  trace.each do |key, data|
+    prefix = "  " * indent
+
+    if data.is_a?(Hash) && data[:value].nil? && data[:source].nil?
+      # Nested section
+      puts "\n#{prefix}#{key}:"
+      print_config(data, indent + 1)
+    elsif data.is_a?(Hash) && data.key?(:value)
+      # Leaf value with source
+      value = data[:value]
+      source = format_source(data[:source])
+
+      # Format value for display
+      display_value = case value
+      when nil then "nil"
+      when String then value.empty? ? '""' : "\"#{value}\""
+      when Symbol then ":#{value}"
+      else value.inspect
+      end
+
+      # Truncate long values
+      display_value = display_value[0..50] + "..." if display_value.length > 50
+
+      puts "#{prefix}#{key}: #{display_value}  # from: #{source}"
+    else
+      # Fallback for unexpected structure
+      puts "#{prefix}#{key}: #{data.inspect}"
+    end
+  end
+end
+
+print_config(trace)
+
+puts <<~LEGEND
+
+  #{'-' * 60}
+  Legend:
+    defaults.yml  = bundled gem defaults
+    htm.local.yml = ./config/htm.local.yml
+    htm.yml       = ./config/htm.yml or ~/.config/htm/htm.yml
+    ENV[KEY]      = environment variable
+    code          = set programmatically
+
+LEGEND

data/examples/custom_llm_configuration.rb

@@ -118,15 +118,15 @@ puts "\n5. Configuring Default Provider Settings"
 puts "-" * 50
 
 HTM.configure do |config|
-  # Customize the default RubyLLM configuration
-  config.embedding_provider = :ollama
-  config.embedding_model = 'nomic-embed-text'
-  config.embedding_dimensions = 768
+  # Customize the default RubyLLM configuration using nested syntax
+  config.embedding.provider = :ollama
+  config.embedding.model = 'nomic-embed-text'
+  config.embedding.dimensions = 768
 
-  config.tag_provider = :ollama
-  config.tag_model = 'llama3'
+  config.tag.provider = :ollama
+  config.tag.model = 'llama3'
 
-  config.ollama_url = ENV['OLLAMA_URL'] || 'http://localhost:11434'
+  config.providers.ollama.url = ENV['OLLAMA_URL'] || 'http://localhost:11434'
 
   # Reset to use these new settings with default implementations
   config.reset_to_defaults

data/examples/example_app/Rakefile

@@ -13,7 +13,7 @@ namespace :app do
   desc "Start the example application"
   task :start do
     puts "\n=== Starting Example HTM Application ==="
-    puts "Database configured: #{ENV['HTM_DBURL'] ? 'Yes' : 'No'}"
+    puts "Database configured: #{ENV['HTM_DATABASE__URL'] ? 'Yes' : 'No'}"
     puts "\nRun 'rake -T' to see all available tasks"
     puts "Run 'rake -T htm:db' to see HTM database tasks\n\n"
 
@@ -63,7 +63,7 @@ task :help do
       rake htm:db:reset      Drop and recreate database
 
     Environment:
-      export HTM_DBURL="postgresql://user:pass@host:port/dbname"
+      export HTM_DATABASE__URL="postgresql://user:pass@host:port/dbname"
 
     See docs/using_rake_tasks_in_your_app.md for more information.
 

data/examples/example_app/app.rb

@@ -16,8 +16,8 @@ class ExampleApp
     config = HTM::Database.default_config
     unless config
       puts "\n⚠ Database not configured!"
-      puts "Set HTM_DBURL environment variable:"
-      puts "  export HTM_DBURL='postgresql://user:pass@host:port/dbname'"
+      puts "Set HTM_DATABASE__URL environment variable:"
+      puts "  export HTM_DATABASE__URL='postgresql://user:pass@host:port/dbname'"
       puts "\nOr run: rake app:bootstrap"
       return
     end
@@ -32,14 +32,14 @@ class ExampleApp
       c.logger.level = Logger::INFO
 
       # Configure embedding generation (using Ollama)
-      c.embedding_provider = :ollama
-      c.embedding_model = 'nomic-embed-text'
-      c.embedding_dimensions = 768
-      c.ollama_url = ENV['OLLAMA_URL'] || 'http://localhost:11434'
+      c.embedding.provider = :ollama
+      c.embedding.model = 'nomic-embed-text'
+      c.embedding.dimensions = 768
+      c.providers.ollama.url = ENV['OLLAMA_URL'] || 'http://localhost:11434'
 
       # Configure tag extraction (using Ollama - using smaller/faster model)
-      c.tag_provider = :ollama
-      c.tag_model = 'gemma3'  # Smaller model (3.3GB) for faster tag generation
+      c.tag.provider = :ollama
+      c.tag.model = 'gemma3'  # Smaller model (3.3GB) for faster tag generation
 
       # Apply default implementations
       c.reset_to_defaults

data/examples/file_loader_usage.rb

@@ -10,7 +10,7 @@
 # - Unloading files
 #
 # Prerequisites:
-# 1. Set HTM_DBURL environment variable (see SETUP.md)
+# 1. Set HTM_DATABASE__URL environment variable (see SETUP.md)
 # 2. Initialize database schema: rake db_setup
 # 3. Install dependencies: bundle install
 
@@ -22,9 +22,9 @@ puts "HTM File Loader Example"
 puts "=" * 60
 
 # Check environment
-unless ENV['HTM_DBURL']
-  puts "ERROR: HTM_DBURL not set. Please set it:"
-  puts "  export HTM_DBURL=\"postgresql://postgres@localhost:5432/htm_development\""
+unless ENV['HTM_DATABASE__URL']
+  puts "ERROR: HTM_DATABASE__URL not set. Please set it:"
+  puts "  export HTM_DATABASE__URL=\"postgresql://postgres@localhost:5432/htm_development\""
   puts "See SETUP.md for details."
   exit 1
 end
@@ -33,11 +33,11 @@ begin
   # Configure HTM globally (uses Ollama by default)
   puts "\n1. Configuring HTM with Ollama provider..."
   HTM.configure do |config|
-    config.embedding_provider = :ollama
-    config.embedding_model = 'nomic-embed-text:latest'
-    config.embedding_dimensions = 768
-    config.tag_provider = :ollama
-    config.tag_model = 'gemma3:latest'
+    config.embedding.provider = :ollama
+    config.embedding.model = 'nomic-embed-text:latest'
+    config.embedding.dimensions = 768
+    config.tag.provider = :ollama
+    config.tag.model = 'gemma3:latest'
     config.reset_to_defaults
   end
   puts "   Configured with Ollama provider"

data/examples/mcp_client.rb

@@ -10,7 +10,7 @@
 # Prerequisites:
 # 1. Install gems: gem install ruby_llm-mcp
 # 2. Have Ollama running with gpt-oss model: ollama pull gpt-oss
-# 3. Set HTM_DBURL environment variable
+# 3. Set HTM_DATABASE__URL environment variable
 # 4. The htm_mcp executable must be available (this client will launch it)
 #
 # Usage:
@@ -57,9 +57,9 @@ class HTMMcpClient
   private
 
   def validate_environment
-    unless ENV['HTM_DBURL']
-      warn 'Error: HTM_DBURL not set.'
-      warn '  export HTM_DBURL="postgresql://postgres@localhost:5432/htm_development"'
+    unless ENV['HTM_DATABASE__URL']
+      warn 'Error: HTM_DATABASE__URL not set.'
+      warn '  export HTM_DATABASE__URL="postgresql://postgres@localhost:5432/htm_development"'
       exit 1
     end
 
@@ -103,7 +103,7 @@ class HTMMcpClient
         command: RbConfig.ruby,
         args: [MCP_SERVER_PATH],
         env: {
-          'HTM_DBURL' => ENV['HTM_DBURL'],
+          'HTM_DATABASE__URL' => ENV['HTM_DATABASE__URL'],
           'OLLAMA_URL' => OLLAMA_URL
         }
       }