htm 0.0.18 → 0.0.20

data/docs/examples/file-loading.md

@@ -0,0 +1,208 @@
+# File Loading Example
+
+This example demonstrates loading markdown files into HTM's long-term memory with automatic chunking, YAML frontmatter extraction, and source tracking.
+
+**Source:** [`examples/file_loader_usage.rb`](https://github.com/madbomber/htm/blob/main/examples/file_loader_usage.rb)
+
+## Overview
+
+The file loading example shows:
+
+- Loading single markdown files
+- Loading directories with glob patterns
+- YAML frontmatter extraction
+- Querying nodes from loaded files
+- Re-sync behavior for changed files
+- Unloading files from memory
+
+## Running the Example
+
+```bash
+export HTM_DATABASE__URL="postgresql://user@localhost:5432/htm_development"
+ruby examples/file_loader_usage.rb
+```
+
+## Code Walkthrough
+
+### Loading a Single File
+
+```ruby
+htm = HTM.new(robot_name: "FileLoaderDemo")
+
+# Load a markdown file
+result = htm.load_file("docs/guide.md")
+# => {
+#   file_source_id: 1,
+#   chunks_created: 5,
+#   chunks_updated: 0,
+#   skipped: false
+# }
+```
+
+### YAML Frontmatter
+
+Files with frontmatter have metadata extracted automatically:
+
+```markdown
+---
+title: PostgreSQL Guide
+author: HTM Team
+tags:
+  - database
+  - postgresql
+---
+
+# PostgreSQL Guide
+
+Content starts here...
+```
+
+Access frontmatter via FileSource:
+
+```ruby
+source = HTM::Models::FileSource.find(result[:file_source_id])
+source.title            # => "PostgreSQL Guide"
+source.author           # => "HTM Team"
+source.frontmatter_tags # => ["database", "postgresql"]
+source.frontmatter      # => { "title" => "...", ... }
+```
+
+### Loading a Directory
+
+```ruby
+# Load all markdown files
+results = htm.load_directory("docs/", pattern: "**/*.md")
+# => [
+#   { file_path: "docs/guide.md", chunks_created: 3, ... },
+#   { file_path: "docs/api.md", chunks_created: 5, ... }
+# ]
+
+# Load with specific pattern
+results = htm.load_directory("docs/guides/", pattern: "*.md")
+```
+
+### Querying Loaded Files
+
+```ruby
+# Get all nodes from a specific file
+nodes = htm.nodes_from_file("docs/guide.md")
+
+nodes.each do |node|
+  puts "#{node.id}: #{node.content[0..50]}..."
+end
+```
+
+### Re-Sync Behavior
+
+HTM tracks file modification times for efficient updates:
+
+```ruby
+# First load - creates chunks
+htm.load_file("docs/guide.md")
+# => { skipped: false, chunks_created: 5 }
+
+# Second load - skipped (unchanged)
+htm.load_file("docs/guide.md")
+# => { skipped: true }
+
+# After editing file - re-syncs
+htm.load_file("docs/guide.md")
+# => { skipped: false, chunks_updated: 2, chunks_created: 1 }
+
+# Force reload
+htm.load_file("docs/guide.md", force: true)
+```
+
+### Unloading Files
+
+```ruby
+# Soft delete all chunks from a file
+count = htm.unload_file("docs/guide.md")
+puts "Removed #{count} chunks"
+```
+
+## Chunking Configuration
+
+```ruby
+HTM.configure do |config|
+  config.chunk_size = 1024    # Characters per chunk (default)
+  config.chunk_overlap = 64   # Overlap between chunks (default)
+end
+```
+
+Or via environment variables:
+
+```bash
+export HTM_CHUNK_SIZE=512
+export HTM_CHUNK_OVERLAP=50
+```
+
+## Expected Output
+
+```
+HTM File Loader Example
+============================================================
+
+1. Configuring HTM with Ollama provider...
+   Configured with Ollama provider
+
+2. Initializing HTM...
+   Robot: FileLoaderDemo (ID: 1)
+
+3. Creating sample markdown files...
+   Created: /tmp/htm_demo/postgresql_guide.md
+   Created: /tmp/htm_demo/ruby_intro.md
+
+4. Loading single file with frontmatter...
+   File: postgresql_guide.md
+   Source ID: 1
+   Chunks created: 3
+   Frontmatter title: PostgreSQL Guide
+   Frontmatter author: HTM Team
+   Frontmatter tags: database, postgresql
+
+5. Loading directory...
+   Files processed: 2
+   - postgresql_guide.md: skipped
+   - ruby_intro.md: 2 chunks
+
+...
+
+============================================================
+Example completed successfully!
+```
+
+## Rake Tasks
+
+```bash
+# Load a single file
+rake 'htm:files:load[docs/guide.md]'
+
+# Load directory
+rake 'htm:files:load_dir[docs/]'
+rake 'htm:files:load_dir[docs/,**/*.md]'
+
+# List loaded files
+rake htm:files:list
+
+# Show file details
+rake 'htm:files:info[docs/guide.md]'
+
+# Unload a file
+rake 'htm:files:unload[docs/guide.md]'
+
+# Sync all files
+rake htm:files:sync
+
+# Show statistics
+rake htm:files:stats
+
+# Force reload
+FORCE=true rake 'htm:files:load[docs/guide.md]'
+```
+
+## See Also
+
+- [File Loading Guide](../guides/file-loading.md)
+- [Basic Usage Example](basic-usage.md)
+- [Markdown Chunking](../guides/file-loading.md#chunking-strategy)

data/docs/examples/index.md

@@ -0,0 +1,116 @@
+# Examples
+
+HTM includes working example programs that demonstrate various features and integration patterns. These examples show real-world usage patterns and can serve as templates for your own applications.
+
+## Running Examples
+
+All examples require the database to be configured:
+
+```bash
+export HTM_DATABASE__URL="postgresql://user@localhost:5432/htm_development"
+```
+
+Then run any example with:
+
+```bash
+ruby examples/<example_name>.rb
+```
+
+## Available Examples
+
+### Core Usage
+
+| Example | Description |
+|---------|-------------|
+| [Basic Usage](basic-usage.md) | Core HTM operations: remember, recall, forget |
+| [LLM Configuration](llm-configuration.md) | Configure providers, custom embeddings, and tag extractors |
+| [File Loading](file-loading.md) | Load markdown files with frontmatter and chunking |
+
+### Advanced Features
+
+| Example | Description |
+|---------|-------------|
+| [Timeframes](timeframes.md) | Natural language temporal queries |
+| [Robot Groups](robot-groups.md) | Multi-robot coordination with shared memory |
+| [MCP Client](mcp-client.md) | Interactive AI chat with memory tools |
+| [Telemetry](telemetry.md) | Prometheus metrics and Grafana visualization |
+
+## Quick Reference
+
+### Basic Operations
+
+```ruby
+# Initialize HTM
+htm = HTM.new(robot_name: "My Robot")
+
+# Remember information
+node_id = htm.remember("PostgreSQL supports vector search via pgvector.")
+
+# Recall memories
+results = htm.recall("database features", strategy: :hybrid, limit: 5)
+
+# Forget a memory
+htm.forget(node_id)
+```
+
+### Configuration
+
+```ruby
+HTM.configure do |config|
+  # Use any RubyLLM-supported provider
+  config.embedding.provider = :openai  # or :ollama, :anthropic, etc.
+  config.embedding.model = 'text-embedding-3-small'
+
+  config.tag.provider = :openai
+  config.tag.model = 'gpt-4o-mini'
+end
+```
+
+### File Loading
+
+```ruby
+# Load markdown files into memory
+htm.load_file("docs/guide.md")
+htm.load_directory("docs/", pattern: "**/*.md")
+
+# Query nodes from a file
+nodes = htm.nodes_from_file("docs/guide.md")
+```
+
+## Prerequisites
+
+Most examples require:
+
+1. **PostgreSQL** with pgvector extension
+2. **LLM Provider** - Ollama (default), OpenAI, Anthropic, etc.
+3. **Ruby 3.2+** with HTM gem installed
+
+### Ollama Setup (Default Provider)
+
+```bash
+# Install Ollama
+curl https://ollama.ai/install.sh | sh
+
+# Pull required models
+ollama pull nomic-embed-text  # For embeddings
+ollama pull gemma3:latest     # For tag extraction
+```
+
+### Using Cloud Providers
+
+```bash
+# OpenAI
+export OPENAI_API_KEY="your-key"
+
+# Anthropic
+export ANTHROPIC_API_KEY="your-key"
+
+# Google Gemini
+export GEMINI_API_KEY="your-key"
+```
+
+## See Also
+
+- [Getting Started Guide](../getting-started/quick-start.md)
+- [API Reference](../api/htm.md)
+- [Architecture Overview](../architecture/overview.md)

data/docs/examples/llm-configuration.md

@@ -0,0 +1,168 @@
+# LLM Configuration Example
+
+This example demonstrates the various ways to configure LLM providers for embeddings and tag extraction in HTM.
+
+**Source:** [`examples/custom_llm_configuration.rb`](https://github.com/madbomber/htm/blob/main/examples/custom_llm_configuration.rb)
+
+## Overview
+
+HTM uses RubyLLM for multi-provider LLM support. This example shows:
+
+- Using default configuration (Ollama)
+- Custom lambda-based embeddings and tags
+- Service object integration
+- Mixed configuration patterns
+- Provider-specific settings
+
+## Running the Example
+
+```bash
+ruby examples/custom_llm_configuration.rb
+```
+
+## Configuration Patterns
+
+### Default Configuration (Ollama)
+
+```ruby
+HTM.configure  # Uses defaults from config files
+
+htm = HTM.new(robot_name: "DefaultBot")
+# Uses: ollama/nomic-embed-text for embeddings
+# Uses: ollama/gemma3:latest for tag extraction
+```
+
+### Custom Lambda Functions
+
+```ruby
+HTM.configure do |config|
+  # Custom embedding generator
+  config.embedding_generator = lambda do |text|
+    # Your custom embedding logic
+    # Must return Array<Float>
+    MyEmbeddingService.embed(text)
+  end
+
+  # Custom tag extractor
+  config.tag_extractor = lambda do |text, existing_ontology|
+    # Your custom tag extraction logic
+    # Must return Array<String>
+    MyTagService.extract(text, ontology: existing_ontology)
+  end
+end
+```
+
+### Service Object Pattern
+
+```ruby
+class MyAppLLMService
+  def self.embed(text)
+    # Integrate with LangChain, LlamaIndex, or custom infrastructure
+    # Returns embedding vector as Array<Float>
+    Array.new(1024) { rand }
+  end
+
+  def self.extract_tags(text, ontology)
+    # Returns array of hierarchical tag strings
+    ['app:feature:memory', 'app:component:llm']
+  end
+end
+
+HTM.configure do |config|
+  config.embedding_generator = ->(text) { MyAppLLMService.embed(text) }
+  config.tag_extractor = ->(text, ontology) { MyAppLLMService.extract_tags(text, ontology) }
+end
+```
+
+### Provider Configuration
+
+```ruby
+HTM.configure do |config|
+  # Configure embedding provider
+  config.embedding.provider = :openai
+  config.embedding.model = 'text-embedding-3-small'
+  config.embedding.dimensions = 1536
+
+  # Configure tag extraction provider
+  config.tag.provider = :anthropic
+  config.tag.model = 'claude-3-haiku-20240307'
+
+  # Provider-specific settings
+  config.providers.ollama.url = 'http://localhost:11434'
+  config.providers.openai.api_key = ENV['OPENAI_API_KEY']
+end
+```
+
+### Mixed Configuration
+
+Use custom embedding with default tag extraction:
+
+```ruby
+HTM.configure do |config|
+  # Custom embedding
+  config.embedding_generator = ->(text) {
+    MyCustomEmbedder.generate(text)
+  }
+
+  # Keep default RubyLLM-based tag extraction
+  # (uses configured tag.provider and tag.model)
+end
+```
+
+## Supported Providers
+
+HTM uses RubyLLM which supports:
+
+| Provider | Embedding Models | Chat Models |
+|----------|-----------------|-------------|
+| Ollama (default) | `nomic-embed-text`, `mxbai-embed-large` | `gemma3`, `llama3`, `mistral` |
+| OpenAI | `text-embedding-3-small`, `text-embedding-3-large` | `gpt-4o-mini`, `gpt-4o` |
+| Anthropic | - | `claude-3-haiku`, `claude-3-sonnet` |
+| Google Gemini | `text-embedding-004` | `gemini-1.5-flash`, `gemini-1.5-pro` |
+| Azure OpenAI | Same as OpenAI | Same as OpenAI |
+| HuggingFace | Various | Various |
+| AWS Bedrock | Titan, Cohere | Claude, Llama |
+| DeepSeek | - | `deepseek-chat` |
+
+## Environment Variables
+
+```bash
+# Ollama (default)
+export OLLAMA_URL="http://localhost:11434"
+
+# OpenAI
+export OPENAI_API_KEY="sk-..."
+export OPENAI_ORGANIZATION="org-..."
+
+# Anthropic
+export ANTHROPIC_API_KEY="sk-ant-..."
+
+# Google Gemini
+export GEMINI_API_KEY="..."
+
+# Azure OpenAI
+export AZURE_OPENAI_API_KEY="..."
+export AZURE_OPENAI_ENDPOINT="https://....openai.azure.com/"
+```
+
+## Integration with HTM Operations
+
+When you call `htm.remember()`, HTM uses your configured generators:
+
+```ruby
+HTM.configure do |config|
+  config.embedding_generator = ->(text) {
+    puts "Embedding: #{text[0..40]}..."
+    MyService.embed(text)
+  }
+end
+
+# This triggers your custom embedding generator
+node_id = htm.remember("PostgreSQL supports vector search")
+```
+
+## See Also
+
+- [Basic Usage Example](basic-usage.md)
+- [Configuration Guide](../getting-started/quick-start.md)
+- [EmbeddingService API](../api/embedding-service.md)

data/docs/examples/mcp-client.md

@@ -0,0 +1,172 @@
+# MCP Client Example
+
+This example demonstrates using the HTM MCP (Model Context Protocol) server with an interactive AI chat interface.
+
+**Source:** [`examples/mcp_client.rb`](https://github.com/madbomber/htm/blob/main/examples/mcp_client.rb)
+
+## Overview
+
+The MCP Client example shows:
+
+- Connecting to the HTM MCP server via STDIO transport
+- Interactive chat with tool calling
+- Memory operations through natural language
+- Session persistence and restoration
+- Available tools and resources
+
+## Prerequisites
+
+```bash
+# Install the MCP client gem
+gem install ruby_llm-mcp
+
+# Have Ollama running with a chat model
+ollama pull gpt-oss  # or llama3, mistral, etc.
+
+# Set database connection
+export HTM_DATABASE__URL="postgresql://user@localhost:5432/htm_development"
+```
+
+## Running the Example
+
+```bash
+ruby examples/mcp_client.rb
+```
+
+## Code Walkthrough
+
+### MCP Client Setup
+
+```ruby
+# Configure RubyLLM for Ollama
+RubyLLM.configure do |config|
+  config.ollama_api_base = "http://localhost:11434/v1"
+end
+
+# Connect to HTM MCP server
+@mcp_client = RubyLLM::MCP.client(
+  name: 'htm-memory',
+  transport_type: :stdio,
+  request_timeout: 60_000,
+  config: {
+    command: RbConfig.ruby,
+    args: ['bin/htm_mcp'],
+    env: {
+      'HTM_DATABASE__URL' => ENV['HTM_DATABASE__URL'],
+      'OLLAMA_URL' => 'http://localhost:11434'
+    }
+  }
+)
+```
+
+### Set Robot Identity
+
+```ruby
+set_robot_tool = @tools.find { |t| t.name == 'SetRobotTool' }
+result = set_robot_tool.call(name: "My Assistant")
+```
+
+### Chat with Tools
+
+```ruby
+@chat = RubyLLM.chat(
+  model: 'gpt-oss:latest',
+  provider: :ollama,
+  assume_model_exists: true
+)
+
+# Attach MCP tools to chat
+@chat.with_tools(*@tools)
+
+# Natural language interactions
+response = @chat.ask("Remember that the API rate limit is 1000 requests per minute")
+# The LLM will call RememberTool automatically
+
+response = @chat.ask("What do you know about databases?")
+# The LLM will call RecallTool and summarize results
+```
+
+### Session Restoration
+
+The client can restore previous session context:
+
+```ruby
+get_wm_tool = @tools.find { |t| t.name == 'GetWorkingMemoryTool' }
+result = get_wm_tool.call({})
+
+if result['count'] > 0
+  # Restore previous memories to chat context
+  @chat.add_message(
+    role: :user,
+    content: "Previous session context: #{memories.join("\n")}"
+  )
+end
+```
+
+## Available Commands
+
+| Command | Description |
+|---------|-------------|
+| `/tools` | List available MCP tools |
+| `/resources` | List available MCP resources |
+| `/stats` | Show memory statistics |
+| `/tags` | List all tags |
+| `/clear` | Clear chat history |
+| `/help` | Show help |
+| `/exit` | Quit |
+
+## Available MCP Tools
+
+| Tool | Description |
+|------|-------------|
+| `SetRobotTool` | Set the current robot identity |
+| `RememberTool` | Store information in memory |
+| `RecallTool` | Query memories by topic |
+| `ForgetTool` | Delete a memory by ID |
+| `ListTagsTool` | List all hierarchical tags |
+| `StatsTool` | Show memory statistics |
+| `GetWorkingMemoryTool` | Get current working memory |
+
+## Example Interactions
+
+```
+you> Remember that the PostgreSQL connection string is in the DATABASE_URL env var
+
+[Tool Call] RememberTool
+  Arguments: {content: "PostgreSQL connection string is in DATABASE_URL env var"}
+[Tool Result] RememberTool
+  Result: {"success": true, "node_id": 42}
+
+Assistant> I've stored that information about the PostgreSQL connection string.
+
+you> What do you know about databases?
+
+[Tool Call] RecallTool
+  Arguments: {topic: "databases", limit: 5}
+[Tool Result] RecallTool
+  Result: {"memories": [...]}
+
+Assistant> Based on my memories, I know that:
+1. PostgreSQL connection string is stored in DATABASE_URL env var
+2. PostgreSQL supports vector search via pgvector
+...
+```
+
+## Configuration
+
+```bash
+# Use a different Ollama model
+export OLLAMA_MODEL="llama3:latest"
+
+# Use a different Ollama URL
+export OLLAMA_URL="http://192.168.1.100:11434"
+
+# Set robot name via environment
+export HTM_ROBOT_NAME="My Custom Bot"
+```
+
+## See Also
+
+- [MCP Server Guide](../guides/mcp-server.md)
+- [HTM API Reference](../api/htm.md)
+- [RubyLLM-MCP Documentation](https://github.com/contextco/ruby_llm-mcp)