htm 0.0.2 → 0.0.11

data/docs/api/yard/HTM/ActiveRecordConfig.md

@@ -0,0 +1,23 @@
+# Class: HTM::ActiveRecordConfig
+**Inherits:** Object
+    
+
+ActiveRecord database configuration and model loading
+
+
+# Class Methods
+## connected?() {: #method-c-connected? }
+Check if connection is established and active
+**`@return`** [Boolean] 
+
+## connection_stats() {: #method-c-connection_stats }
+Get connection pool statistics
+## disconnect!() {: #method-c-disconnect! }
+Close all database connections
+## establish_connection!() {: #method-c-establish_connection! }
+Establish database connection from config/database.yml
+## load_database_config() {: #method-c-load_database_config }
+Load and parse database configuration from YAML with ERB
+## verify_extensions!() {: #method-c-verify_extensions! }
+Verify required extensions are available
+

data/docs/api/yard/HTM/AuthorizationError.md

@@ -0,0 +1,11 @@
+# Exception: HTM::AuthorizationError
+**Inherits:** HTM::Error
+    
+
+Raised when an operation is not authorized
+
+Reserved for future multi-tenant scenarios where access control may restrict
+certain operations.
+
+
+

data/docs/api/yard/HTM/CircuitBreaker.md

@@ -0,0 +1,92 @@
+# Class: HTM::CircuitBreaker
+**Inherits:** Object
+    
+
+Circuit Breaker - Prevents cascading failures from external LLM services
+
+Implements the circuit breaker pattern to protect against repeated failures
+when calling external LLM APIs for embeddings or tag extraction.
+
+States:
+*   :closed - Normal operation, requests flow through
+*   :open - Circuit tripped, requests fail fast with CircuitBreakerOpenError
+*   :half_open - Testing if service recovered, allows limited requests
+
+
+**`@example`**
+```ruby
+breaker = HTM::CircuitBreaker.new(name: 'embedding')
+result = breaker.call { external_api_call }
+```
+**`@example`**
+```ruby
+breaker = HTM::CircuitBreaker.new(
+  name: 'tag_extraction',
+  failure_threshold: 3,
+  reset_timeout: 30
+)
+```
+# Attributes
+## failure_count[RW] {: #attribute-i-failure_count }
+Returns the value of attribute failure_count.
+
+## last_failure_time[RW] {: #attribute-i-last_failure_time }
+Returns the value of attribute last_failure_time.
+
+## name[RW] {: #attribute-i-name }
+Returns the value of attribute name.
+
+## state[RW] {: #attribute-i-state }
+Returns the value of attribute state.
+
+
+# Instance Methods
+## call() {: #method-i-call }
+Execute a block with circuit breaker protection
+
+**`@raise`** [CircuitBreakerOpenError] If circuit is open
+
+**`@raise`** [StandardError] If the block raises an error (after recording failure)
+
+**`@return`** [Object] Result of the block if successful
+
+**`@yield`** [] Block containing the protected operation
+
+## closed?() {: #method-i-closed? }
+Check if circuit is currently closed (normal operation)
+
+**`@return`** [Boolean] true if circuit is closed
+
+## half_open?() {: #method-i-half_open? }
+Check if circuit is in half-open state (testing recovery)
+
+**`@return`** [Boolean] true if circuit is half-open
+
+## initialize(name:, failure_threshold:DEFAULT_FAILURE_THRESHOLD, reset_timeout:DEFAULT_RESET_TIMEOUT, half_open_max_calls:DEFAULT_HALF_OPEN_MAX_CALLS) {: #method-i-initialize }
+Initialize a new circuit breaker
+
+**`@param`** [String] Identifier for this circuit breaker (for logging)
+
+**`@param`** [Integer] Number of failures before opening circuit
+
+**`@param`** [Integer] Seconds to wait before attempting recovery
+
+**`@param`** [Integer] Successful calls needed to close circuit
+
+**`@return`** [CircuitBreaker] a new instance of CircuitBreaker
+
+## open?() {: #method-i-open? }
+Check if circuit is currently open
+
+**`@return`** [Boolean] true if circuit is open
+
+## reset!() {: #method-i-reset! }
+Manually reset the circuit breaker to closed state
+
+**`@return`** [void] 
+
+## stats() {: #method-i-stats }
+Get current circuit breaker statistics
+
+**`@return`** [Hash] Statistics including state, failure count, etc.
+

data/docs/api/yard/HTM/CircuitBreakerOpenError.md

@@ -0,0 +1,34 @@
+# Exception: HTM::CircuitBreakerOpenError
+**Inherits:** HTM::Error
+    
+
+Raised when circuit breaker is open due to repeated failures
+
+The circuit breaker pattern protects against cascading failures when external
+LLM services are unavailable. When too many consecutive failures occur, the
+circuit "opens" and subsequent calls fail fast without attempting the
+operation.
+
+Circuit states:
+*   :closed - Normal operation, requests flow through
+*   :open - Too many failures, requests fail immediately
+*   :half_open - Testing if service recovered
+
+After a reset timeout (default: 60 seconds), the circuit transitions to
+half-open and tests if the service has recovered.
+
+**`@see`** [] 
+
+**`@see`** [] 
+
+
+**`@example`**
+```ruby
+begin
+  htm.remember("new content")
+rescue HTM::CircuitBreakerOpenError
+  # LLM service unavailable, but node is still saved
+  # Embeddings/tags will be generated later when service recovers
+end
+```
+

data/docs/api/yard/HTM/Configuration.md

@@ -0,0 +1,175 @@
+# Class: HTM::Configuration
+**Inherits:** Object
+    
+
+HTM Configuration
+
+HTM uses RubyLLM for multi-provider LLM support. Supported providers:
+*   :openai (OpenAI API)
+*   :anthropic (Anthropic Claude)
+*   :gemini (Google Gemini)
+*   :azure (Azure OpenAI)
+*   :ollama (Local Ollama - default)
+*   :huggingface (HuggingFace Inference API)
+*   :openrouter (OpenRouter)
+*   :bedrock (AWS Bedrock)
+*   :deepseek (DeepSeek)
+
+
+**`@example`**
+```ruby
+HTM.configure do |config|
+  config.embedding_provider = :openai
+  config.embedding_model = 'text-embedding-3-small'
+  config.tag_provider = :openai
+  config.tag_model = 'gpt-4o-mini'
+  config.openai_api_key = ENV['OPENAI_API_KEY']
+end
+```
+**`@example`**
+```ruby
+HTM.configure do |config|
+  config.embedding_provider = :ollama
+  config.embedding_model = 'nomic-embed-text'
+  config.tag_provider = :ollama
+  config.tag_model = 'llama3'
+  config.ollama_url = 'http://localhost:11434'
+end
+```
+**`@example`**
+```ruby
+HTM.configure do |config|
+  config.embedding_provider = :openai
+  config.embedding_model = 'text-embedding-3-small'
+  config.openai_api_key = ENV['OPENAI_API_KEY']
+  config.tag_provider = :anthropic
+  config.tag_model = 'claude-3-haiku-20240307'
+  config.anthropic_api_key = ENV['ANTHROPIC_API_KEY']
+end
+```
+**`@example`**
+```ruby
+HTM.configure do |config|
+  config.embedding_generator = ->(text) {
+    MyApp::LLMService.embed(text)  # Returns Array<Float>
+  }
+  config.tag_extractor = ->(text, ontology) {
+    MyApp::LLMService.extract_tags(text, ontology)  # Returns Array<String>
+  }
+  config.logger = Rails.logger
+end
+```
+# Attributes
+## anthropic_api_key[RW] {: #attribute-i-anthropic_api_key }
+Returns the value of attribute anthropic_api_key.
+
+## azure_api_key[RW] {: #attribute-i-azure_api_key }
+Returns the value of attribute azure_api_key.
+
+## azure_api_version[RW] {: #attribute-i-azure_api_version }
+Returns the value of attribute azure_api_version.
+
+## azure_endpoint[RW] {: #attribute-i-azure_endpoint }
+Returns the value of attribute azure_endpoint.
+
+## bedrock_access_key[RW] {: #attribute-i-bedrock_access_key }
+Returns the value of attribute bedrock_access_key.
+
+## bedrock_region[RW] {: #attribute-i-bedrock_region }
+Returns the value of attribute bedrock_region.
+
+## bedrock_secret_key[RW] {: #attribute-i-bedrock_secret_key }
+Returns the value of attribute bedrock_secret_key.
+
+## connection_timeout[RW] {: #attribute-i-connection_timeout }
+Returns the value of attribute connection_timeout.
+
+## deepseek_api_key[RW] {: #attribute-i-deepseek_api_key }
+Returns the value of attribute deepseek_api_key.
+
+## embedding_dimensions[RW] {: #attribute-i-embedding_dimensions }
+Returns the value of attribute embedding_dimensions.
+
+## embedding_generator[RW] {: #attribute-i-embedding_generator }
+Returns the value of attribute embedding_generator.
+
+## embedding_model[RW] {: #attribute-i-embedding_model }
+Returns the value of attribute embedding_model.
+
+## embedding_provider[RW] {: #attribute-i-embedding_provider }
+Returns the value of attribute embedding_provider.
+
+## embedding_timeout[RW] {: #attribute-i-embedding_timeout }
+Returns the value of attribute embedding_timeout.
+
+## gemini_api_key[RW] {: #attribute-i-gemini_api_key }
+Returns the value of attribute gemini_api_key.
+
+## huggingface_api_key[RW] {: #attribute-i-huggingface_api_key }
+Returns the value of attribute huggingface_api_key.
+
+## job_backend[RW] {: #attribute-i-job_backend }
+Returns the value of attribute job_backend.
+
+## logger[RW] {: #attribute-i-logger }
+Returns the value of attribute logger.
+
+## ollama_url[RW] {: #attribute-i-ollama_url }
+Returns the value of attribute ollama_url.
+
+## openai_api_key[RW] {: #attribute-i-openai_api_key }
+Provider-specific API keys and endpoints
+
+## openai_organization[RW] {: #attribute-i-openai_organization }
+Provider-specific API keys and endpoints
+
+## openai_project[RW] {: #attribute-i-openai_project }
+Provider-specific API keys and endpoints
+
+## openrouter_api_key[RW] {: #attribute-i-openrouter_api_key }
+Returns the value of attribute openrouter_api_key.
+
+## tag_extractor[RW] {: #attribute-i-tag_extractor }
+Returns the value of attribute tag_extractor.
+
+## tag_model[RW] {: #attribute-i-tag_model }
+Returns the value of attribute tag_model.
+
+## tag_provider[RW] {: #attribute-i-tag_provider }
+Returns the value of attribute tag_provider.
+
+## tag_timeout[RW] {: #attribute-i-tag_timeout }
+Returns the value of attribute tag_timeout.
+
+## token_counter[RW] {: #attribute-i-token_counter }
+Returns the value of attribute token_counter.
+
+## week_start[RW] {: #attribute-i-week_start }
+Returns the value of attribute week_start.
+
+
+# Instance Methods
+## configure_ruby_llm(providernil) {: #method-i-configure_ruby_llm }
+Configure RubyLLM with the appropriate provider credentials
+
+**`@param`** [Symbol] The provider to configure (:openai, :anthropic, etc.)
+
+## initialize() {: #method-i-initialize }
+**`@return`** [Configuration] a new instance of Configuration
+
+## normalize_ollama_model(model_name) {: #method-i-normalize_ollama_model }
+Normalize Ollama model name to include tag if missing
+
+Ollama models require a tag (e.g., :latest, :7b, :13b). If the user specifies
+a model without a tag, we append :latest by default.
+
+**`@param`** [String] Original model name
+
+**`@return`** [String] Normalized model name with tag
+
+## reset_to_defaults() {: #method-i-reset_to_defaults }
+Reset to default RubyLLM-based implementations
+
+## validate!() {: #method-i-validate! }
+Validate configuration
+

data/docs/api/yard/HTM/Database.md

@@ -0,0 +1,99 @@
+# Class: HTM::Database
+**Inherits:** Object
+    
+
+Database setup and configuration for HTM Handles schema creation and database
+initialization
+
+
+# Class Methods
+## default_config() {: #method-c-default_config }
+Get default database configuration
+**`@return`** [Hash, nil] Connection configuration hash
+
+## drop(db_url nil) {: #method-c-drop }
+Drop all HTM tables
+**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+
+**`@return`** [void] 
+
+## dump_schema(db_url nil) {: #method-c-dump_schema }
+Dump current database schema to db/schema.sql
+
+Uses pg_dump to create a clean SQL schema file without data
+**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+
+**`@return`** [void] 
+
+## generate_docs(db_url nil) {: #method-c-generate_docs }
+Generate database documentation using tbls
+
+Uses .tbls.yml configuration file for output directory and settings. Creates
+comprehensive database documentation including:
+*   Entity-relationship diagrams
+*   Table schemas with comments
+*   Index information
+*   Relationship diagrams
+**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+
+**`@return`** [void] 
+
+## info(db_url nil) {: #method-c-info }
+Show database info
+**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+
+**`@return`** [void] 
+
+## load_schema(db_url nil) {: #method-c-load_schema }
+Load schema from db/schema.sql
+
+Uses psql to load the schema file
+**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+
+**`@return`** [void] 
+
+## migrate(db_url nil) {: #method-c-migrate }
+Run pending database migrations
+**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+
+**`@return`** [void] 
+
+## migration_status(db_url nil) {: #method-c-migration_status }
+Show migration status
+**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+
+**`@return`** [void] 
+
+## parse_connection_params() {: #method-c-parse_connection_params }
+Build config from individual environment variables
+**`@return`** [Hash, nil] Connection configuration hash
+
+## parse_connection_url(url ) {: #method-c-parse_connection_url }
+Parse database connection URL
+**`@param`** [String] Connection URL (e.g., postgresql://user:pass@host:port/dbname)
+
+**`@raise`** [ArgumentError] If URL format is invalid
+
+**`@return`** [Hash, nil] Connection configuration hash
+
+## seed(db_url nil) {: #method-c-seed }
+Seed database with sample data
+
+Loads and executes db/seeds.rb file following Rails conventions. All seeding
+logic is contained in db/seeds.rb and reads data from markdown files in
+db/seed_data/ directory.
+**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+
+**`@return`** [void] 
+
+## setup(db_url nil, run_migrations: true, dump_schema: false) {: #method-c-setup }
+Set up the HTM database schema
+**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+
+**`@param`** [Boolean] Whether to run migrations (default: true)
+
+**`@param`** [Boolean] Whether to dump schema to db/schema.sql after setup (default: false)
+
+**`@return`** [void] 
+
+

data/docs/api/yard/HTM/DatabaseError.md

@@ -0,0 +1,14 @@
+# Exception: HTM::DatabaseError
+**Inherits:** HTM::Error
+    
+
+Raised when database operations fail
+
+Common causes:
+*   Connection failures
+*   Query syntax errors
+*   Constraint violations
+*   Extension not installed (pgvector, pg_trgm)
+
+
+

data/docs/api/yard/HTM/EmbeddingError.md

@@ -0,0 +1,18 @@
+# Exception: HTM::EmbeddingError
+**Inherits:** HTM::Error
+    
+
+Raised when embedding generation fails
+
+Common causes:
+*   LLM provider API errors
+*   Invalid embedding response format
+*   Network connectivity issues
+*   Model not available
+
+Note: This error is distinct from CircuitBreakerOpenError. EmbeddingError
+indicates a single failure, while CircuitBreakerOpenError indicates repeated
+failures have triggered protective circuit breaking.
+
+
+

data/docs/api/yard/HTM/EmbeddingService.md

@@ -0,0 +1,58 @@
+# Class: HTM::EmbeddingService
+**Inherits:** Object
+    
+
+Embedding Service - Processes and validates vector embeddings
+
+This service wraps the configured embedding generator and provides:
+*   Response validation
+*   Dimension handling (padding/truncation)
+*   Error handling and logging
+*   Storage formatting
+*   Circuit breaker protection for external LLM failures
+
+The actual LLM call is delegated to HTM.configuration.embedding_generator
+
+
+# Class Methods
+## circuit_breaker() {: #method-c-circuit_breaker }
+Get or create the circuit breaker for embedding service
+**`@return`** [HTM::CircuitBreaker] The circuit breaker instance
+
+## format_for_storage(embedding ) {: #method-c-format_for_storage }
+Format embedding for database storage
+**`@param`** [Array<Float>] Padded embedding
+
+**`@return`** [String] PostgreSQL array format
+
+## generate(text ) {: #method-c-generate }
+Generate embedding with validation and processing
+**`@param`** [String] Text to embed
+
+**`@raise`** [CircuitBreakerOpenError] If circuit breaker is open
+
+**`@return`** [Hash] Processed embedding with metadata
+{
+  embedding: Array<Float>,           # Original embedding
+  dimension: Integer,                # Original dimension
+  storage_embedding: String,         # Formatted for database storage
+  storage_dimension: Integer         # Padded dimension (2000)
+}
+
+## pad_embedding(embedding ) {: #method-c-pad_embedding }
+Pad embedding to MAX_DIMENSION with zeros
+**`@param`** [Array<Float>] Original embedding
+
+**`@return`** [Array<Float>] Padded embedding
+
+## reset_circuit_breaker!() {: #method-c-reset_circuit_breaker! }
+Reset the circuit breaker (useful for testing)
+**`@return`** [void] 
+
+## validate_embedding!(embedding ) {: #method-c-validate_embedding! }
+Validate embedding response format
+**`@param`** [Object] Raw embedding from generator
+
+**`@raise`** [HTM::EmbeddingError] if invalid
+
+

data/docs/api/yard/HTM/Error.md

@@ -0,0 +1,11 @@
+# Exception: HTM::Error
+**Inherits:** StandardError
+    
+
+Base error class for all HTM errors
+
+All custom HTM errors inherit from this class, providing a common ancestor for
+error handling.
+
+
+

data/docs/api/yard/HTM/JobAdapter.md

@@ -0,0 +1,39 @@
+# Module: HTM::JobAdapter
+    
+
+Job adapter for pluggable background job backends
+
+Supports multiple job backends to work seamlessly across different application
+types (CLI, Sinatra, Rails).
+
+Supported backends:
+*   :active_job - Rails ActiveJob (recommended for Rails apps)
+*   :sidekiq - Direct Sidekiq integration (recommended for Sinatra apps)
+*   :inline - Synchronous execution (recommended for CLI and tests)
+*   :thread - Background thread (legacy, for standalone apps)
+
+**`@see`** [] Async Embedding and Tag Generation
+
+
+**`@example`**
+```ruby
+HTM.configure do |config|
+  config.job_backend = :active_job
+end
+```
+**`@example`**
+```ruby
+HTM::JobAdapter.enqueue(HTM::Jobs::GenerateEmbeddingJob, node_id: 123)
+```
+# Class Methods
+## enqueue(job_class , **params ) {: #method-c-enqueue }
+Enqueue a background job using the configured backend
+**`@param`** [Class] Job class to enqueue (must respond to :perform)
+
+**`@param`** [Hash] Parameters to pass to the job
+
+**`@raise`** [HTM::Error] If job backend is unknown
+
+**`@return`** [void] 
+
+