htm 0.0.11 → 0.0.15

data/docs/api/yard/HTM/Configuration.md

@@ -81,6 +81,21 @@ Returns the value of attribute bedrock_region.
 ## bedrock_secret_key[RW] {: #attribute-i-bedrock_secret_key }
 Returns the value of attribute bedrock_secret_key.
 
+## chunk_overlap[RW] {: #attribute-i-chunk_overlap }
+Character overlap between chunks (default: 64)
+
+## chunk_size[RW] {: #attribute-i-chunk_size }
+Chunking configuration (for file loading)
+
+## circuit_breaker_failure_threshold[RW] {: #attribute-i-circuit_breaker_failure_threshold }
+Circuit breaker configuration
+
+## circuit_breaker_half_open_max_calls[RW] {: #attribute-i-circuit_breaker_half_open_max_calls }
+Successes to close (default: 3)
+
+## circuit_breaker_reset_timeout[RW] {: #attribute-i-circuit_breaker_reset_timeout }
+Seconds before half-open (default: 60)
+
 ## connection_timeout[RW] {: #attribute-i-connection_timeout }
 Returns the value of attribute connection_timeout.
 
@@ -102,6 +117,9 @@ Returns the value of attribute embedding_provider.
 ## embedding_timeout[RW] {: #attribute-i-embedding_timeout }
 Returns the value of attribute embedding_timeout.
 
+## extract_propositions[RW] {: #attribute-i-extract_propositions }
+Returns the value of attribute extract_propositions.
+
 ## gemini_api_key[RW] {: #attribute-i-gemini_api_key }
 Returns the value of attribute gemini_api_key.
 
@@ -114,6 +132,12 @@ Returns the value of attribute job_backend.
 ## logger[RW] {: #attribute-i-logger }
 Returns the value of attribute logger.
 
+## max_embedding_dimension[RW] {: #attribute-i-max_embedding_dimension }
+Limit configuration
+
+## max_tag_depth[RW] {: #attribute-i-max_tag_depth }
+Max tag hierarchy depth (default: 4)
+
 ## ollama_url[RW] {: #attribute-i-ollama_url }
 Returns the value of attribute ollama_url.
 
@@ -129,6 +153,33 @@ Provider-specific API keys and endpoints
 ## openrouter_api_key[RW] {: #attribute-i-openrouter_api_key }
 Returns the value of attribute openrouter_api_key.
 
+## proposition_extractor[RW] {: #attribute-i-proposition_extractor }
+Returns the value of attribute proposition_extractor.
+
+## proposition_model[RW] {: #attribute-i-proposition_model }
+Returns the value of attribute proposition_model.
+
+## proposition_provider[RW] {: #attribute-i-proposition_provider }
+Returns the value of attribute proposition_provider.
+
+## proposition_timeout[RW] {: #attribute-i-proposition_timeout }
+Returns the value of attribute proposition_timeout.
+
+## relevance_access_weight[RW] {: #attribute-i-relevance_access_weight }
+Access frequency weight (default: 0.1)
+
+## relevance_recency_half_life_hours[RW] {: #attribute-i-relevance_recency_half_life_hours }
+Decay half-life in hours (default: 168 = 1 week)
+
+## relevance_recency_weight[RW] {: #attribute-i-relevance_recency_weight }
+Temporal freshness weight (default: 0.1)
+
+## relevance_semantic_weight[RW] {: #attribute-i-relevance_semantic_weight }
+Relevance scoring weights (must sum to 1.0)
+
+## relevance_tag_weight[RW] {: #attribute-i-relevance_tag_weight }
+Tag overlap weight (default: 0.3)
+
 ## tag_extractor[RW] {: #attribute-i-tag_extractor }
 Returns the value of attribute tag_extractor.
 
@@ -141,6 +192,9 @@ Returns the value of attribute tag_provider.
 ## tag_timeout[RW] {: #attribute-i-tag_timeout }
 Returns the value of attribute tag_timeout.
 
+## telemetry_enabled[RW] {: #attribute-i-telemetry_enabled }
+Enable OpenTelemetry metrics (default: false)
+
 ## token_counter[RW] {: #attribute-i-token_counter }
 Returns the value of attribute token_counter.
 

data/docs/api/yard/HTM/Database.md

@@ -8,20 +8,23 @@ initialization
 
 # Class Methods
 ## default_config() {: #method-c-default_config }
-Get default database configuration
-**`@return`** [Hash, nil] Connection configuration hash
+Get default database configuration (respects RAILS_ENV)
+
+Uses ActiveRecordConfig which reads from config/database.yml and respects
+RAILS_ENV for environment-specific database selection.
+**`@return`** [Hash, nil] Connection configuration hash with PG-style keys
 
 ## drop(db_url nil) {: #method-c-drop }
-Drop all HTM tables
-**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+Drop all HTM tables (respects RAILS_ENV)
+**`@param`** [String] Database connection URL (uses default_config if not provided)
 
 **`@return`** [void] 
 
 ## dump_schema(db_url nil) {: #method-c-dump_schema }
-Dump current database schema to db/schema.sql
+Dump current database schema to db/schema.sql (respects RAILS_ENV)
 
 Uses pg_dump to create a clean SQL schema file without data
-**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+**`@param`** [String] Database connection URL (uses default_config if not provided)
 
 **`@return`** [void] 
 
@@ -39,16 +42,16 @@ comprehensive database documentation including:
 **`@return`** [void] 
 
 ## info(db_url nil) {: #method-c-info }
-Show database info
-**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+Show database info (respects RAILS_ENV)
+**`@param`** [String] Database connection URL (uses default_config if not provided)
 
 **`@return`** [void] 
 
 ## load_schema(db_url nil) {: #method-c-load_schema }
-Load schema from db/schema.sql
+Load schema from db/schema.sql (respects RAILS_ENV)
 
 Uses psql to load the schema file
-**`@param`** [String] Database connection URL (uses ENV['HTM_DBURL'] if not provided)
+**`@param`** [String] Database connection URL (uses default_config if not provided)
 
 **`@return`** [void] 
 

data/docs/api/yard/HTM/EmbeddingService.md

@@ -39,8 +39,12 @@ Generate embedding with validation and processing
   storage_dimension: Integer         # Padded dimension (2000)
 }
 
+## max_dimension() {: #method-c-max_dimension }
+Maximum embedding dimension (configurable, default 2000)
+**`@return`** [Integer] Max dimensions for pgvector HNSW index
+
 ## pad_embedding(embedding ) {: #method-c-pad_embedding }
-Pad embedding to MAX_DIMENSION with zeros
+Pad embedding to max_dimension with zeros
 **`@param`** [Array<Float>] Original embedding
 
 **`@return`** [Array<Float>] Padded embedding

data/docs/api/yard/HTM/LongTermMemory.md

@@ -1,6 +1,8 @@
 # Class: HTM::LongTermMemory
 **Inherits:** Object
     
+**Includes:** FulltextSearch, HybridSearch, NodeOperations, RelevanceScorer, RobotOperations, TagOperations, VectorSearch
+  
 
 Long-term Memory - PostgreSQL/TimescaleDB-backed permanent storage
 
@@ -13,6 +15,21 @@ LongTermMemory provides durable storage for all memory nodes with:
 *   ActiveRecord ORM for data access
 *   Query result caching for efficiency
 
+This class uses standalone utility classes and modules:
+
+Standalone classes (used via class methods or instances):
+*   HTM::SqlBuilder: SQL condition building helpers (class methods)
+*   HTM::QueryCache: Query result caching (instantiated as @cache)
+
+Included modules:
+*   RelevanceScorer: Dynamic relevance scoring
+*   NodeOperations: Node CRUD operations
+*   RobotOperations: Robot registration and activity
+*   TagOperations: Tag management
+*   VectorSearch: Vector similarity search
+*   FulltextSearch: Full-text search
+*   HybridSearch: Combined search strategies
+
 
 # Attributes
 ## query_timeout[RW] {: #attribute-i-query_timeout }
@@ -20,104 +37,11 @@ Returns the value of attribute query_timeout.
 
 
 # Instance Methods
-## add(content:, token_count:0, robot_id:, embedding:nil, metadata:{}) {: #method-i-add }
-Add a node to long-term memory (with deduplication)
-
-If content already exists (by content_hash), links the robot to the existing
-node and updates timestamps. Otherwise creates a new node.
-
-**`@param`** [String] Conversation message/utterance
-
-**`@param`** [Integer] Token count
-
-**`@param`** [Integer] Robot identifier
-
-**`@param`** [Array<Float>, nil] Pre-generated embedding vector
-
-**`@param`** [Hash] Flexible metadata for the node (default: {})
-
-**`@return`** [Hash] { node_id:, is_new:, robot_node: }
-
-## add_tag(node_id:, tag:) {: #method-i-add_tag }
-Add a tag to a node
-
-**`@param`** [Integer] Node database ID
-
-**`@param`** [String] Tag name
-
-**`@return`** [void] 
-
-## batch_load_node_tags(node_ids) {: #method-i-batch_load_node_tags }
-Batch load tags for multiple nodes (avoids N+1 queries)
-
-**`@param`** [Array<Integer>] Node database IDs
-
-**`@return`** [Hash<Integer, Array<String>>] Map of node_id to array of tag names
-
-## calculate_relevance(node:, query_tags:[], vector_similarity:nil, node_tags:nil) {: #method-i-calculate_relevance }
-Calculate dynamic relevance score for a node given query context
-
-Combines multiple signals:
-*   Vector similarity (semantic match)
-*   Tag overlap (categorical match)
-*   Recency (freshness)
-*   Access frequency (popularity/utility)
-
-**`@param`** [Hash] Node data with similarity, tags, created_at, access_count
-
-**`@param`** [Array<String>] Tags associated with the query
-
-**`@param`** [Float, nil] Pre-computed vector similarity (0-1)
-
-**`@param`** [Array<String>, nil] Pre-loaded tags for this node (avoids N+1 query)
-
-**`@return`** [Float] Composite relevance score (0-10)
-
 ## clear_cache!() {: #method-i-clear_cache! }
-Clear the query cache
-
-Call this after any operation that modifies data (soft delete, restore, etc.)
-to ensure subsequent queries see fresh results.
-
-**`@return`** [void] 
-
-## delete(node_id) {: #method-i-delete }
-Delete a node
-
-**`@param`** [Integer] Node database ID
+Clear the query result cache
 
 **`@return`** [void] 
 
-## exists?(node_id) {: #method-i-exists? }
-Check if a node exists
-
-**`@param`** [Integer] Node database ID
-
-**`@return`** [Boolean] True if node exists
-
-## find_query_matching_tags(query, include_extracted:false) {: #method-i-find_query_matching_tags }
-Find tags that match terms in the query
-
-Searches the tags table for tags where any hierarchy level matches query
-words. For example, query "PostgreSQL database" would match tags like
-"database:postgresql", "database:sql", etc. Find tags matching a query using
-semantic extraction
-
-**`@param`** [String] Search query
-
-**`@param`** [Boolean] If true, returns hash with :extracted and :matched keys
-
-**`@return`** [Array<String>] Matching tag names (default)
-
-**`@return`** [Hash] If include_extracted: { extracted: [...], matched: [...] }
-
-## get_node_tags(node_id) {: #method-i-get_node_tags }
-Get tags for a specific node
-
-**`@param`** [Integer] Node database ID
-
-**`@return`** [Array<String>] Tag names
-
 ## initialize(config, pool_size:nil, query_timeout:DEFAULT_QUERY_TIMEOUT, cache_size:DEFAULT_CACHE_SIZE, cache_ttl:DEFAULT_CACHE_TTL) {: #method-i-initialize }
 Initialize long-term memory storage
 
@@ -146,160 +70,9 @@ ltm = LongTermMemory.new(config, cache_size: 500, cache_ttl: 600)
 ```ruby
 ltm = LongTermMemory.new(config, cache_size: 0)
 ```
-## link_robot_to_node(robot_id:, node:, working_memory:false) {: #method-i-link_robot_to_node }
-Link a robot to a node (create or update robot_node record)
-
-**`@param`** [Integer] Robot ID
-
-**`@param`** [HTM::Models::Node] Node to link
-
-**`@param`** [Boolean] Whether node is in working memory (default: false)
-
-**`@return`** [HTM::Models::RobotNode] The robot_node link record
-
-## mark_evicted(robot_id:, node_ids:) {: #method-i-mark_evicted }
-Mark nodes as evicted from working memory
-
-Sets working_memory = false on the robot_nodes join table for the specified
-robot and node IDs.
-
-**`@param`** [Integer] Robot ID whose working memory is being evicted
-
-**`@param`** [Array<Integer>] Node IDs to mark as evicted
-
-**`@return`** [void] 
-
-## node_topics(node_id) {: #method-i-node_topics }
-Get topics for a specific node
-
-**`@param`** [Integer] Node database ID
-
-**`@return`** [Array<String>] Topic paths
-
-## nodes_by_topic(topic_path, exact:false, limit:50) {: #method-i-nodes_by_topic }
-Retrieve nodes by ontological topic
-
-**`@param`** [String] Topic hierarchy path
-
-**`@param`** [Boolean] Exact match or prefix match
-
-**`@param`** [Integer] Maximum results
-
-**`@return`** [Array<Hash>] Matching nodes
-
-## ontology_structure() {: #method-i-ontology_structure }
-Get ontology structure view
-
-**`@return`** [Array<Hash>] Ontology structure
-
 ## pool_size() {: #method-i-pool_size }
 For backwards compatibility with tests/code that expect pool_size
 
-## popular_tags(limit:20, timeframe:nil) {: #method-i-popular_tags }
-Get most popular tags
-
-**`@param`** [Integer] Number of tags to return
-
-**`@param`** [Range, nil] Optional time range filter
-
-**`@return`** [Array<Hash>] Tags with usage counts
-
-## register_robot(robot_name) {: #method-i-register_robot }
-Register a robot
-
-**`@param`** [String] Robot identifier
-
-**`@param`** [String] Robot name
-
-**`@return`** [void] 
-
-## retrieve(node_id) {: #method-i-retrieve }
-Retrieve a node by ID
-
-Automatically tracks access by incrementing access_count and updating
-last_accessed
-
-**`@param`** [Integer] Node database ID
-
-**`@return`** [Hash, nil] Node data or nil
-
-## search(timeframe:, query:, limit:, embedding_service:, metadata:{}) {: #method-i-search }
-Vector similarity search
-
-**`@param`** [nil, Range, Array<Range>] Time range(s) to search (nil = no filter)
-
-**`@param`** [String] Search query
-
-**`@param`** [Integer] Maximum results
-
-**`@param`** [Object] Service to generate embeddings
-
-**`@param`** [Hash] Filter by metadata fields (default: {})
-
-**`@return`** [Array<Hash>] Matching nodes
-
-## search_by_tags(tags:, match_all:false, timeframe:nil, limit:20) {: #method-i-search_by_tags }
-Search nodes by tags
-
-**`@param`** [Array<String>] Tags to search for
-
-**`@param`** [Boolean] If true, match ALL tags; if false, match ANY tag
-
-**`@param`** [Range, nil] Optional time range filter
-
-**`@param`** [Integer] Maximum results
-
-**`@return`** [Array<Hash>] Matching nodes with relevance scores
-
-## search_fulltext(timeframe:, query:, limit:, metadata:{}) {: #method-i-search_fulltext }
-Full-text search
-
-**`@param`** [Range] Time range to search
-
-**`@param`** [String] Search query
-
-**`@param`** [Integer] Maximum results
-
-**`@param`** [Hash] Filter by metadata fields (default: {})
-
-**`@return`** [Array<Hash>] Matching nodes
-
-## search_hybrid(timeframe:, query:, limit:, embedding_service:, prefilter_limit:100, metadata:{}) {: #method-i-search_hybrid }
-Hybrid search (full-text + vector)
-
-**`@param`** [Range] Time range to search
-
-**`@param`** [String] Search query
-
-**`@param`** [Integer] Maximum results
-
-**`@param`** [Object] Service to generate embeddings
-
-**`@param`** [Integer] Candidates to consider (default: 100)
-
-**`@param`** [Hash] Filter by metadata fields (default: {})
-
-**`@return`** [Array<Hash>] Matching nodes
-
-## search_with_relevance(timeframe:, query:nil, query_tags:[], limit:20, embedding_service:nil, metadata:{}) {: #method-i-search_with_relevance }
-Search with dynamic relevance scoring
-
-Returns nodes with calculated relevance scores based on query context
-
-**`@param`** [nil, Range, Array<Range>] Time range(s) to search (nil = no filter)
-
-**`@param`** [String, nil] Search query
-
-**`@param`** [Array<String>] Tags to match
-
-**`@param`** [Integer] Maximum results
-
-**`@param`** [Object, nil] Service to generate embeddings
-
-**`@param`** [Hash] Filter by metadata fields (default: {})
-
-**`@return`** [Array<Hash>] Nodes with relevance scores
-
 ## shutdown() {: #method-i-shutdown }
 Shutdown - no-op with ActiveRecord (connection pool managed by ActiveRecord)
 
@@ -308,35 +81,3 @@ Get memory statistics
 
 **`@return`** [Hash] Statistics
 
-## topic_relationships(min_shared_nodes:2, limit:50) {: #method-i-topic_relationships }
-Get topic relationships (co-occurrence)
-
-**`@param`** [Integer] Minimum shared nodes
-
-**`@param`** [Integer] Maximum relationships
-
-**`@return`** [Array<Hash>] Topic relationships
-
-## track_access(node_ids) {: #method-i-track_access }
-Track access for multiple nodes (bulk operation)
-
-Updates access_count and last_accessed for all nodes in the array
-
-**`@param`** [Array<Integer>] Node IDs that were accessed
-
-**`@return`** [void] 
-
-## update_last_accessed(node_id) {: #method-i-update_last_accessed }
-Update last_accessed timestamp
-
-**`@param`** [Integer] Node database ID
-
-**`@return`** [void] 
-
-## update_robot_activity(robot_id) {: #method-i-update_robot_activity }
-Update robot activity timestamp
-
-**`@param`** [String] Robot identifier
-
-**`@return`** [void] 
-

data/docs/api/yard/HTM/PropositionError.md

@@ -0,0 +1,18 @@
+# Exception: HTM::PropositionError
+**Inherits:** HTM::Error
+    
+
+Raised when proposition extraction fails
+
+Common causes:
+*   LLM provider API errors
+*   Invalid proposition response format
+*   Network connectivity issues
+*   Model not available
+
+Note: This error is distinct from CircuitBreakerOpenError. PropositionError
+indicates a single failure, while CircuitBreakerOpenError indicates repeated
+failures have triggered protective circuit breaking.
+
+
+

data/docs/api/yard/HTM/PropositionService.md

@@ -0,0 +1,66 @@
+# Class: HTM::PropositionService
+**Inherits:** Object
+    
+
+Proposition Service - Extracts atomic factual propositions from text
+
+This service breaks complex text into simple, self-contained factual
+statements that can be stored as independent memory nodes. Each proposition:
+*   Expresses a single fact
+*   Is understandable without context
+*   Uses full names, not pronouns
+*   Includes relevant dates/qualifiers
+*   Contains one subject-predicate relationship
+
+The actual LLM call is delegated to HTM.configuration.proposition_extractor
+
+
+**`@example`**
+```ruby
+propositions = HTM::PropositionService.extract(
+  "In 1969, Neil Armstrong became the first person to walk on the Moon during Apollo 11."
+)
+# => ["Neil Armstrong was an astronaut.",
+#     "Neil Armstrong walked on the Moon in 1969.",
+#     "Neil Armstrong was the first person to walk on the Moon.",
+#     "Neil Armstrong walked on the Moon during the Apollo 11 mission.",
+#     "The Apollo 11 mission occurred in 1969."]
+```
+# Class Methods
+## circuit_breaker() {: #method-c-circuit_breaker }
+Get or create the circuit breaker for proposition service
+**`@return`** [HTM::CircuitBreaker] The circuit breaker instance
+
+## extract(content ) {: #method-c-extract }
+Extract propositions from text content
+**`@param`** [String] Text to analyze
+
+**`@raise`** [CircuitBreakerOpenError] If circuit breaker is open
+
+**`@raise`** [PropositionError] If extraction fails
+
+**`@return`** [Array<String>] Array of atomic propositions
+
+## parse_propositions(raw_propositions ) {: #method-c-parse_propositions }
+Parse proposition response (handles string or array input)
+**`@param`** [String, Array] Raw response from extractor
+
+**`@return`** [Array<String>] Parsed proposition strings
+
+## reset_circuit_breaker!() {: #method-c-reset_circuit_breaker! }
+Reset the circuit breaker (useful for testing)
+**`@return`** [void] 
+
+## valid_proposition?(proposition ) {: #method-c-valid_proposition? }
+Validate single proposition
+**`@param`** [String] Proposition to validate
+
+**`@return`** [Boolean] True if valid
+
+## validate_and_filter_propositions(propositions ) {: #method-c-validate_and_filter_propositions }
+Validate and filter propositions
+**`@param`** [Array<String>] Parsed propositions
+
+**`@return`** [Array<String>] Valid propositions only
+
+

data/docs/api/yard/HTM/QueryCache.md

@@ -0,0 +1,88 @@
+# Class: HTM::QueryCache
+**Inherits:** Object
+    
+
+Thread-safe query result cache with TTL and statistics
+
+Provides LRU caching for expensive query results with:
+*   Configurable size and TTL
+*   Thread-safe statistics tracking
+*   Fast cache key generation (using Ruby's built-in hash)
+*   Selective cache invalidation by method type
+
+
+**`@example`**
+```ruby
+cache = HTM::QueryCache.new(size: 1000, ttl: 300)
+```
+**`@example`**
+```ruby
+result = cache.fetch(:search, timeframe, query, limit) do
+  expensive_search_operation
+end
+```
+**`@example`**
+```ruby
+cache.stats
+# => { hits: 42, misses: 10, hit_rate: 80.77, size: 52 }
+```
+**`@example`**
+```ruby
+cache.invalidate_methods!(:search, :hybrid)  # Only invalidate search-related entries
+```
+# Attributes
+## enabled[RW] {: #attribute-i-enabled }
+Returns the value of attribute enabled.
+
+
+# Instance Methods
+## clear!() {: #method-i-clear! }
+Clear all cached entries
+
+**`@return`** [void] 
+
+## enabled?() {: #method-i-enabled? }
+Check if cache is enabled
+
+**`@return`** [Boolean] 
+
+## fetch(method, *args, &block) {: #method-i-fetch }
+Fetch a value from cache or execute block
+
+**`@param`** [Symbol] Method name for cache key
+
+**`@param`** [Array] Arguments for cache key
+
+**`@return`** [Object] Cached or computed value
+
+**`@yield`** [] Block that computes the value if not cached
+
+## initialize(size:1000, ttl:300) {: #method-i-initialize }
+Initialize a new query cache
+
+**`@param`** [Integer] Maximum number of entries (default: 1000, use 0 to disable)
+
+**`@param`** [Integer] Time-to-live in seconds (default: 300)
+
+**`@return`** [QueryCache] a new instance of QueryCache
+
+## invalidate!() {: #method-i-invalidate! }
+Invalidate cache (alias for clear!)
+
+**`@return`** [void] 
+
+## invalidate_methods!(*methods) {: #method-i-invalidate_methods! }
+Invalidate cache entries for specific methods only
+
+More efficient than full invalidation when only certain types of cached data
+need to be refreshed.
+
+**`@param`** [Array<Symbol>] Method names to invalidate
+
+**`@return`** [Integer] Number of entries invalidated
+
+## stats() {: #method-i-stats }
+Get cache statistics
+
+**`@return`** [Hash, nil] Statistics hash or nil if disabled
+