htm 0.0.15 → 0.0.17

data/README.md

@@ -1,1657 +1,163 @@
 <div align="center">
   <h1>HTM</h1>
-  <!-- img src="docs/assets/images/htm.jpg" alt="HTM - Hierarchical Temporal Memory" width="600" -->
   <img src="docs/assets/images/htm_demo.gif" alt="Tree of Knowledge is Growing" width="400">
 
-  <p>A hierarchical and temporal system for encoding, storing, and retrieving information—operating across varying levels of abstraction (from simple to detailed concepts and their relationships) and across time (from the present to the past).<br/>
-  </p>
-</div>
-
-<br/><br/>
-
-> [!CAUTION]
-> This library is under active development and experimentation. APIs and features may change without notice. Not recommended for production use yet... and the documentation may lie to you!
-> <br /><br/>
-> Apologies to Jeff Hawkins for using his term in such a macro-superficial way.
-
-
-## What does it mean?
-
-- **Hierarchical**: operates across multiple levels of abstraction, from simple concepts to detailed relationships
-- **Temporal**: functions across time, from the present moment to historical data
-- **Memory function**: encodes, stores, and retrieves information
-
-**HTM**: a hierarchical and temporal memory system that organizes and recalls information at multiple levels of detail over extended timeframes.
-
-## Features
-
-- **Client-Side Embeddings**
-    - Automatic embedding generation before database insertion
-    - Uses the [ruby_llm](https://ruby_llm.com) gem for LLM access
-    - Configurable embedding providers and models
-
-- **Two-Tier Memory Architecture**
-    - Working Memory: Token-limited active context for immediate LLM use
-    - Long-term Memory: Durable PostgreSQL storage
-
-- **Never Forgets (Unless Told)**
-    - All memories persist in long-term storage
-    - Only explicit `forget()` commands delete data
-    - Working memory evicts to long-term, never deletes
-
-- **RAG-Based Retrieval**
-    - Vector similarity search (pgvector)
-    - Full-text search (PostgreSQL)
-    - Hybrid search (combines both)
-    - Temporal filtering ("last week", date ranges)
-    - Variable embedding dimensions (384 to 3072)
-
-- **Hive Mind**
-    - All robots share global memory
-    - Cross-robot context awareness
-    - Track which robot said what
-
-- **Robot Groups**
-    - Coordinate multiple robots with shared working memory
-    - Real-time synchronization via PostgreSQL LISTEN/NOTIFY
-    - Active/passive roles for instant failover
-    - Dynamic scaling (add/remove robots at runtime)
-
-- **LLM-Driven Tag Extraction**
-    - Automatic hierarchical tag extraction from content
-    - Tags in colon-delimited format (e.g., `database:postgresql:performance`)
-    - LLM-powered asynchronous processing
-    - Enables both structured navigation and semantic discovery
-    - Complements vector embeddings (symbolic + sub-symbolic retrieval)
-
-- **Knowledge Graph**
-    - Tag-based categorization
-    - Hierarchical tag structures
-
-- **File Loading**
-    - Load markdown files into long-term memory
-    - Automatic paragraph-based chunking
-    - Source file tracking with re-sync support
-    - YAML frontmatter extraction as metadata
-
-- **Telemetry (OpenTelemetry)**
-    - Optional metrics collection via OpenTelemetry
-    - Zero overhead when disabled (null object pattern)
-    - Works with 50+ backends (Jaeger, Prometheus, Datadog, etc.)
-    - Tracks job latency, search performance, and cache effectiveness
-
-## Installation
-
-Add this line to your application's Gemfile:
-
-```ruby
-gem 'htm'
-```
-
-And then execute:
-
-```bash
-bundle install
-```
-
-Or install it yourself as:
-
-```bash
-gem install htm
-```
-
-## Setup
-
-### 1. Database Configuration
-
-HTM uses PostgreSQL for durable long-term memory storage. Set up your database connection via environment variables:
-
-```bash
-# Preferred: Full connection URL
-export HTM_DBURL="postgresql://user:password@host:port/dbname?sslmode=require"
-
-# Alternative: Individual parameters
-export HTM_DBNAME="htm_production"
-export HTM_DBUSER="postgres"
-export HTM_DBPASS="your_password"
-export HTM_DBHOST="localhost"
-export HTM_DBPORT="5432"
-```
-
-See the [Environment Variables](#environment-variables) section below for complete details.
-
-### 2. Configure LLM Providers
-
-HTM uses LLM providers for embedding generation and tag extraction. By default, it uses Ollama (local).
-
-**Start Ollama (Default Configuration)**:
-```bash
-# Install Ollama
-curl https://ollama.ai/install.sh | sh
-
-# Start Ollama and pull models
-ollama serve
-ollama pull nomic-embed-text  # For embeddings (768 dimensions)
-ollama pull llama3           # For tag extraction
-```
-
-**Configure HTM** (optional - uses defaults if not configured):
-```ruby
-require 'htm'
-
-# Use defaults (Ollama with nomic-embed-text and llama3)
-HTM.configure
-
-# Or customize providers
-HTM.configure do |config|
-  # Embedding configuration
-  config.embedding_provider = :ollama  # or :openai
-  config.embedding_model = 'nomic-embed-text'
-  config.embedding_dimensions = 768
-  config.ollama_url = 'http://localhost:11434'
-
-  # Tag extraction configuration
-  config.tag_provider = :ollama  # or :openai
-  config.tag_model = 'llama3'
-
-  # Logger configuration (optional)
-  config.logger = Logger.new($stdout)
-  config.logger.level = Logger::INFO
-
-  # Custom embedding generator (advanced)
-  config.embedding_generator = ->(text) {
-    # Your custom implementation
-    # Must return Array<Float>
-  }
-
-  # Custom tag extractor (advanced)
-  config.tag_extractor = ->(text, ontology) {
-    # Your custom implementation
-    # Must return Array<String>
-  }
-
-  # Token counter (optional, defaults to Tiktoken)
-  config.token_counter = ->(text) {
-    # Your custom implementation
-    # Must return Integer
-  }
-end
-```
-
-See the [Configuration](#configuration) section below for complete details.
-
-### 3. Initialize Database Schema
-
-**Using Rake Tasks (Recommended)**:
-
-HTM provides comprehensive rake tasks for database management. Add this to your application's `Rakefile`:
-
-```ruby
-require 'htm/tasks'
-```
-
-Then use the tasks:
-
-```bash
-# Set up database schema and run migrations
-rake htm:db:setup
-
-# Verify connection
-rake htm:db:test
-
-# Check database status
-rake htm:db:info
-```
-
-**Or programmatically**:
-
-```ruby
-require 'htm'
-
-# Run once to set up database schema
-HTM::Database.setup
-```
-
-**Or from command line**:
-
-```bash
-ruby -r ./lib/htm -e "HTM::Database.setup"
-```
-
-See [Using HTM Rake Tasks in Your Application](docs/using_rake_tasks_in_your_app.md) for complete integration guide.
-
-### 4. Verify Setup
-
-```bash
-rake htm:db:test    # Using rake tasks
-# or
-ruby test_connection.rb
-```
-
-See [docs/setup_local_database.md](docs/setup_local_database.md) for detailed local database setup instructions.
-
-## Usage
-
-### Basic Example
-
-```ruby
-require 'htm'
-require 'ruby_llm'
-require 'logger'
-
-# Configure HTM with RubyLLM for embeddings and tag extraction
-HTM.configure do |config|
-  # Configure logger (optional - uses STDOUT at INFO level if not provided)
-  config.logger = Logger.new($stdout)
-  config.logger.level = Logger::INFO
-
-  # Configure embedding generation using RubyLLM with Ollama
-  config.embedding_provider = :ollama
-  config.embedding_model = 'nomic-embed-text'
-  config.embedding_dimensions = 768
-  config.ollama_url = ENV['OLLAMA_URL'] || 'http://localhost:11434'
-
-  # Configure tag extraction using RubyLLM with Ollama
-  config.tag_provider = :ollama
-  config.tag_model = 'llama3'
-
-  # Apply configuration (sets up default RubyLLM implementations)
-  config.reset_to_defaults
-end
-
-# Initialize HTM for your robot
-htm = HTM.new(
-  robot_name: "Code Helper",
-  working_memory_size: 128_000  # tokens
-)
-
-# Remember information - embeddings and tags generated asynchronously
-node_id = htm.remember(
-  "We decided to use PostgreSQL for HTM storage",
-  source: "architect"
-)
-
-# Recall from the past (uses semantic search with embeddings)
-memories = htm.recall(
-  "database decisions",
-  timeframe: "last week",
-  strategy: :hybrid  # Combines vector + full-text search
-)
-
-# Forget (explicit deletion only)
-htm.forget(node_id, confirm: :confirmed)
-```
-
-### Loading Files
-
-HTM can load text-based files (currently markdown) into long-term memory with automatic chunking and source tracking.
-
-```ruby
-htm = HTM.new(robot_name: "Document Loader")
-
-# Load a single markdown file
-result = htm.load_file("docs/guide.md")
-# => { file_source_id: 1, chunks_created: 5, chunks_updated: 0, chunks_deleted: 0 }
-
-# Load all markdown files in a directory
-results = htm.load_directory("docs/", pattern: "**/*.md")
-
-# Get nodes from a specific file
-nodes = htm.nodes_from_file("docs/guide.md")
-
-# Unload a file (soft deletes chunks)
-htm.unload_file("docs/guide.md")
-```
-
-**Features:**
-- **Paragraph chunking**: Text split by blank lines, code blocks preserved
-- **Source tracking**: Files tracked with mtime for automatic re-sync
-- **YAML frontmatter**: Extracted and stored as metadata
-- **Duplicate detection**: Content hash prevents duplicate nodes
-
-**Rake tasks:**
-```bash
-rake 'htm:files:load[docs/guide.md]'   # Load a single file
-rake 'htm:files:load_dir[docs/]'       # Load all markdown files from directory
-rake htm:files:list                     # List all loaded file sources
-rake htm:files:sync                     # Sync all files (reload changed)
-rake htm:files:stats                    # Show file loading statistics
-```
-
-### Robot Groups
-
-Robot Groups enable coordination of multiple robots with shared working memory and real-time synchronization. This is useful for high-availability setups, load balancing, and collaborative AI agents.
-
-**Key Classes:**
-- `HTM::RobotGroup` - Coordinates multiple robots with active/passive roles and shared working memory
-- `HTM::WorkingMemoryChannel` - PostgreSQL LISTEN/NOTIFY pub/sub for real-time cross-process synchronization
-
-**Basic Usage:**
-
-```ruby
-require 'htm'
-
-HTM.configure do |config|
-  config.embedding_provider = :ollama
-  config.embedding_model = 'nomic-embed-text'
-end
-
-# Create a robot group with primary + standby
-group = HTM::RobotGroup.new(
-  name: 'customer-support-ha',
-  active: ['support-primary'],
-  passive: ['support-standby'],
-  max_tokens: 8000
-)
-
-# Add memories to the shared working memory
-group.remember("Customer #123 prefers email communication", originator: 'support-primary')
-
-# All robots in the group can recall shared memories
-memories = group.recall('customer', limit: 5, strategy: :fulltext)
-
-# Simulate failover when primary fails
-group.failover!  # Promotes standby to active
-
-# Dynamic scaling - add more robots at runtime
-group.add_active('support-secondary')
-group.sync_robot('support-secondary')  # Sync existing memories
-
-# Check group status
-status = group.status
-# => { name: "customer-support-ha", active: [...], passive: [...], in_sync: true, ... }
-
-# Cleanup
-group.shutdown
-```
-
-**Cross-Process Synchronization:**
-
-For multi-process deployments, use `HTM::WorkingMemoryChannel` directly:
-
-```ruby
-# In each worker process
-channel = HTM::WorkingMemoryChannel.new('my-group', HTM::Database.default_config)
-
-# Listen for memory changes from other processes
-channel.on_change do |event, node_id, origin_robot_id|
-  case event
-  when :added    # Another robot added a memory
-  when :evicted  # Another robot evicted a memory
-  when :cleared  # Another robot cleared working memory
-  end
-end
-
-channel.start_listening
-
-# Notify other processes when you add a memory
-channel.notify(:added, node_id: node.id, robot_id: my_robot_id)
-```
-
-**Demo Applications:**
-
-See `examples/robot_groups/` for complete examples:
-- `same_process.rb` - Single-process demo with multiple robots
-- `multi_process.rb` - Multi-process demo with real-time sync
-
-### Automatic Tag Extraction
-
-HTM automatically extracts hierarchical tags from content using LLM analysis. Tags are inferred from the content itself - you never specify them manually.
-
-**Example:**
-```ruby
-htm.remember(
-  "User prefers dark mode for all interfaces",
-  source: "user"
-)
-# Tags like "preference", "ui", "dark-mode" may be auto-extracted by the LLM
-# The specific tags depend on the LLM's analysis of the content
-```
-
-### Async Job Processing
-
-HTM automatically generates embeddings and extracts tags asynchronously in background jobs. This avoids blocking the main request path.
-
-**Monitor job processing:**
-```bash
-# Show statistics for nodes and async processing
-rake htm:jobs:stats
-
-# Output:
-# Total nodes: 150
-# Nodes with embeddings: 145 (96.7%)
-# Nodes without embeddings: 5 (3.3%)
-# Nodes with tags: 140 (93.3%)
-# Total tags in ontology: 42
-```
-
-**Process pending jobs manually:**
-```bash
-# Process all pending embedding jobs
-rake htm:jobs:process_embeddings
-
-# Process all pending tag extraction jobs
-rake htm:jobs:process_tags
-
-# Process both embeddings and tags
-rake htm:jobs:process_all
-```
-
-**Reprocess all nodes (force regeneration):**
-```bash
-# Regenerate embeddings for ALL nodes
-rake htm:jobs:reprocess_embeddings
-
-# WARNING: Prompts for confirmation
-```
-
-**Show failed/stuck jobs:**
-```bash
-# Show nodes that failed async processing (>1 hour old without embeddings/tags)
-rake htm:jobs:failed
-```
-
-**Clear all for testing:**
-```bash
-# Clear ALL embeddings and tags (development/testing only)
-rake htm:jobs:clear_all
-
-# WARNING: Prompts for confirmation
-```
-
-See `rake -T htm:jobs` for complete list of job management tasks.
-
-## Telemetry (OpenTelemetry)
-
-HTM includes optional OpenTelemetry-based metrics for production observability. Telemetry is **disabled by default** with zero overhead when off.
-
-### Enabling Telemetry
-
-```ruby
-HTM.configure do |config|
-  config.telemetry_enabled = true
-end
-
-# Or via environment variable
-# HTM_TELEMETRY_ENABLED=true
-```
-
-### Configuring the Destination
-
-HTM emits metrics via standard OpenTelemetry protocols. Configure your destination using environment variables:
-
-```bash
-# Export to any OTLP-compatible backend
-export OTEL_METRICS_EXPORTER="otlp"
-export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"
-```
-
-### Available Metrics
-
-| Metric | Type | Attributes | Description |
-|--------|------|------------|-------------|
-| `htm.jobs` | Counter | `job`, `status` | Job execution counts (embedding, tags) |
-| `htm.embedding.latency` | Histogram | `provider`, `status` | Embedding generation time (ms) |
-| `htm.tag.latency` | Histogram | `provider`, `status` | Tag extraction time (ms) |
-| `htm.search.latency` | Histogram | `strategy` | Search operation time (ms) |
-| `htm.cache.operations` | Counter | `operation` | Cache hits/misses |
-
-### Compatible Backends
-
-HTM works with any OTLP-compatible observability platform:
-
-**Open Source:** Jaeger, Prometheus, Grafana Tempo/Mimir, SigNoz, Uptrace
-
-**Commercial:** Datadog, New Relic, Honeycomb, Splunk, Dynatrace, AWS X-Ray, Google Cloud Trace, Azure Monitor
-
-### Optional Dependencies
-
-Users who want telemetry should add these gems:
-
-```ruby
-gem 'opentelemetry-sdk'
-gem 'opentelemetry-metrics-sdk'
-gem 'opentelemetry-exporter-otlp'  # For OTLP export
-```
-
-### Design
-
-HTM uses a **null object pattern** for telemetry. When disabled or when the SDK is not installed:
-- All metric operations are no-ops
-- Zero runtime overhead
-- No errors or exceptions
-
-See [docs/telemetry.md](docs/telemetry.md) for detailed configuration and usage examples.
-
-## MCP Server (Model Context Protocol)
-
-HTM includes an MCP server that exposes memory capabilities to AI assistants like Claude Desktop, Claude Code, and AIA. This enables any MCP-compatible client to store, recall, and manage memories through a standardized protocol.
-
-### Available Tools
-
-| Tool | Description |
-|------|-------------|
-| `SetRobotTool` | Set the robot identity for the session (call first) |
-| `GetRobotTool` | Get current robot information |
-| `GetWorkingMemoryTool` | Get working memory contents for session restore |
-| `RememberTool` | Store information with optional tags and metadata |
-| `RecallTool` | Search memories using vector, fulltext, or hybrid strategies |
-| `ForgetTool` | Soft-delete a memory (recoverable) |
-| `RestoreTool` | Restore a soft-deleted memory |
-| `ListTagsTool` | List tags with optional prefix filtering |
-| `SearchTagsTool` | Fuzzy tag search with typo tolerance |
-| `FindByTopicTool` | Find nodes by topic with optional fuzzy matching |
-| `StatsTool` | Get memory usage statistics |
-
-### Available Resources
-
-| URI | Description |
-|-----|-------------|
-| `htm://statistics` | Memory statistics as JSON |
-| `htm://tags/hierarchy` | Tag hierarchy as text tree |
-| `htm://memories/recent` | Last 20 memories |
-
-### Client Configuration
-
-**Claude Desktop** (`~/.config/claude/claude_desktop_config.json`):
-```json
-{
-  "mcpServers": {
-    "htm-memory": {
-      "command": "htm_mcp.rb",
-      "env": {
-        "HTM_DBURL": "postgresql://user@localhost:5432/htm_development"
-      }
-    }
-  }
-}
-```
-
-**Claude Code** (`~/.claude/claude_code_config.json`):
-```json
-{
-  "mcpServers": {
-    "htm-memory": {
-      "command": "htm_mcp.rb",
-      "env": {
-        "HTM_DBURL": "postgresql://user@localhost:5432/htm_development"
-      }
-    }
-  }
-}
-```
-
-**AIA** (`~/.config/aia/config.yml`):
-```yaml
-mcp_servers:
-  htm-memory:
-    command: htm_mcp.rb
-    env:
-      HTM_DBURL: postgresql://user@localhost:5432/htm_development
-```
-
-See [docs/guides/mcp-server.md](docs/guides/mcp-server.md) for detailed configuration, usage examples, and troubleshooting.
-
-## Configuration
-
-HTM uses dependency injection for LLM access, allowing you to configure embedding generation, tag extraction, logging, and token counting.
-
-### Default Configuration
-
-By default, HTM uses:
-- **Embedding Provider**: Ollama (local) with `nomic-embed-text` model (768 dimensions)
-- **Tag Provider**: Ollama (local) with `llama3` model
-- **Logger**: Ruby's standard Logger to STDOUT at INFO level
-- **Token Counter**: Tiktoken with GPT-3.5-turbo encoding
-
-```ruby
-require 'htm'
-
-# Use defaults
-HTM.configure  # or omit this - configuration is lazy-loaded
-```
-
-### Custom Configuration
-
-Configure HTM before creating instances:
-
-#### Using Ollama (Default)
-
-```ruby
-HTM.configure do |config|
-  # Embedding configuration
-  config.embedding_provider = :ollama
-  config.embedding_model = 'nomic-embed-text'  # 768 dimensions
-  config.embedding_dimensions = 768
-  config.ollama_url = 'http://localhost:11434'
-
-  # Tag extraction configuration
-  config.tag_provider = :ollama
-  config.tag_model = 'llama3'
-
-  # Logger configuration
-  config.logger = Logger.new($stdout)
-  config.logger.level = Logger::INFO
-end
-```
-
-**Ollama Setup:**
-```bash
-# Install Ollama
-curl https://ollama.ai/install.sh | sh
-
-# Pull models
-ollama pull nomic-embed-text  # For embeddings
-ollama pull llama3           # For tag extraction
-
-# Verify Ollama is running
-curl http://localhost:11434/api/version
-```
-
-#### Using OpenAI
-
-```ruby
-HTM.configure do |config|
-  # Embedding configuration (OpenAI)
-  config.embedding_provider = :openai
-  config.embedding_model = 'text-embedding-3-small'  # 1536 dimensions
-  config.embedding_dimensions = 1536
-
-  # Tag extraction (can mix providers)
-  config.tag_provider = :openai
-  config.tag_model = 'gpt-4'
-
-  # Logger
-  config.logger = Rails.logger  # Use Rails logger
-end
-```
-
-**OpenAI Setup:**
-```bash
-# Set your OpenAI API key
-export OPENAI_API_KEY='sk-your-api-key-here'
-```
-
-**Important:** The database uses pgvector with HNSW indexing, which has a maximum dimension limit of 2000. Embeddings exceeding this limit will be automatically truncated with a warning.
-
-#### Custom Providers
-
-Provide your own LLM implementations:
-
-```ruby
-HTM.configure do |config|
-  # Custom embedding generator
-  config.embedding_generator = ->(text) {
-    # Call your custom LLM service
-    response = MyEmbeddingService.generate(text)
-
-    # Must return Array<Float>
-    response[:embedding]  # e.g., [0.123, -0.456, ...]
-  }
-
-  # Custom tag extractor
-  config.tag_extractor = ->(text, existing_ontology) {
-    # Call your custom LLM service for tag extraction
-    # existing_ontology is Array<String> of recent tags for context
-    response = MyTagService.extract(text, context: existing_ontology)
-
-    # Must return Array<String> in hierarchical format
-    # e.g., ["ai:llm:embeddings", "database:postgresql"]
-    response[:tags]
-  }
-
-  # Custom token counter (optional)
-  config.token_counter = ->(text) {
-    # Your token counting implementation
-    # Must return Integer
-    MyTokenizer.count(text)
-  }
-end
-```
-
-#### Logger Configuration
-
-Customize logging behavior:
-
-```ruby
-HTM.configure do |config|
-  # Use custom logger
-  config.logger = Logger.new('log/htm.log')
-  config.logger.level = Logger::DEBUG
-
-  # Or use Rails logger
-  config.logger = Rails.logger
-
-  # Or disable logging
-  config.logger = Logger.new(IO::NULL)
-  config.logger.level = Logger::FATAL
-end
-
-# Control log level via environment variable
-ENV['HTM_LOG_LEVEL'] = 'DEBUG'  # or INFO, WARN, ERROR
-HTM.configure  # Respects HTM_LOG_LEVEL
-```
-
-#### Service Layer Architecture
-
-HTM uses a service layer to process LLM responses:
-
-- **EmbeddingService**: Calls your configured `embedding_generator`, validates responses, handles padding/truncation, and formats for storage
-- **TagService**: Calls your configured `tag_extractor`, parses responses (String or Array), validates format, and filters invalid tags
-
-This separation allows you to provide raw LLM access while HTM handles response processing, validation, and storage formatting.
-
-### Recall Strategies
-
-HTM supports three retrieval strategies:
-
-```ruby
-# Vector similarity search (semantic) - uses configured embedding provider
-memories = htm.recall(
-  "HTM architecture",
-  timeframe: "last week",
-  strategy: :vector  # Semantic similarity using embeddings
-)
-
-# Full-text search (keyword matching) - PostgreSQL full-text search with trigrams
-memories = htm.recall(
-  "database performance",
-  timeframe: "last month",
-  strategy: :fulltext  # Keyword-based matching
-)
-
-# Hybrid (combines both) - best of both worlds
-memories = htm.recall(
-  "testing strategies",
-  timeframe: "yesterday",
-  strategy: :hybrid  # Weighted combination of vector + full-text
-)
-```
+  <p><strong>Give your AI tools a shared, persistent memory.</strong></p>
 
-**Strategy Details:**
-- `:vector` - Semantic search using pgvector cosine similarity (requires embeddings)
-- `:fulltext` - PostgreSQL tsvector with pg_trgm fuzzy matching
-- `:hybrid` - Combines both with weighted scoring (default: 70% vector, 30% full-text)
-
-## API Reference
-
-HTM provides a minimal, focused API with only 3 core instance methods for memory operations:
-
-### Core Memory Operations
-
-#### `remember(content, source: "", metadata: {})`
-
-Store information in memory. Embeddings and tags are automatically generated asynchronously.
-
-**Parameters:**
-- `content` (String, required) - The information to remember. Converted to string if nil. Returns ID of last node if empty.
-- `source` (String, optional) - Where the content came from (e.g., "user", "assistant", "system"). Defaults to empty string.
-- `metadata` (Hash, optional) - Arbitrary key-value metadata stored as JSONB. Keys must be strings or symbols. Defaults to `{}`.
-
-**Returns:** Integer - The node ID of the stored memory
-
-**Example:**
-```ruby
-# Store with source
-node_id = htm.remember("PostgreSQL is excellent for vector search with pgvector", source: "architect")
-
-# Store without source (uses default empty string)
-node_id = htm.remember("HTM uses two-tier memory architecture")
-
-# Store with metadata
-node_id = htm.remember(
-  "User prefers dark mode",
-  metadata: { category: "preference", priority: "high", version: 2 }
-)
-
-# Nil/empty handling
-node_id = htm.remember(nil)  # Returns ID of last node without creating duplicate
-node_id = htm.remember("")   # Returns ID of last node without creating duplicate
-```
-
----
-
-#### `recall(topic, timeframe: nil, limit: 20, strategy: :vector, with_relevance: false, query_tags: [], metadata: {})`
-
-Retrieve memories using temporal filtering and semantic/keyword search.
-
-**Parameters:**
-- `topic` (String, required) - Query text for semantic/keyword matching (first positional argument)
-- `timeframe` (Range, String, optional) - Time range to search within. Default: "last 7 days"
-  - Range: `(Time.now - 3600)..Time.now`
-  - String: `"last hour"`, `"last week"`, `"yesterday"`
-- `limit` (Integer, optional) - Maximum number of results. Default: 20
-- `strategy` (Symbol, optional) - Search strategy. Default: `:vector`
-  - `:vector` - Semantic search using embeddings (cosine similarity)
-  - `:fulltext` - Keyword search using PostgreSQL full-text + trigrams
-  - `:hybrid` - Weighted combination (70% vector, 30% full-text)
-- `with_relevance` (Boolean, optional) - Include dynamic relevance scores. Default: false
-- `query_tags` (Array<String>, optional) - Filter results by tags. Default: []
-- `metadata` (Hash, optional) - Filter results by metadata using JSONB containment (`@>`). Default: `{}`
-
-**Returns:** Array<Hash> - Matching memories with fields: `id`, `content`, `source`, `created_at`, `access_count`, `metadata`, (optionally `relevance`)
-
-**Example:**
-```ruby
-# Basic recall with time range
-memories = htm.recall(
-  "database architecture",
-  timeframe: (Time.now - 86400)..Time.now
-)
-
-# Using human-readable timeframe
-memories = htm.recall(
-  "PostgreSQL performance",
-  timeframe: "last week",
-  strategy: :hybrid,
-  limit: 10
-)
-
-# With relevance scoring
-memories = htm.recall(
-  "HTM design decisions",
-  timeframe: "last month",
-  with_relevance: true,
-  query_tags: ["architecture"]
-)
-# => [{ "id" => 123, "content" => "...", "relevance" => 0.92, ... }, ...]
-
-# Filter by metadata
-memories = htm.recall(
-  "user preferences",
-  metadata: { category: "preference" }
-)
-# => Returns only nodes with metadata containing { category: "preference" }
-
-# Combine metadata with other filters
-memories = htm.recall(
-  "settings",
-  timeframe: "last month",
-  strategy: :hybrid,
-  metadata: { priority: "high", version: 2 }
-)
-```
-
----
-
-#### `forget(node_id, confirm: false)`
-
-Permanently delete a memory from both working memory and long-term storage.
-
-**Parameters:**
-- `node_id` (Integer, required) - The ID of the node to delete
-- `confirm` (Symbol, Boolean, required) - Must be `:confirmed` or `true` to proceed
-
-**Returns:** Boolean - `true` if deleted, `false` if not found
-
-**Safety:** This is the ONLY way to delete memories. Requires explicit confirmation to prevent accidental deletion.
-
-**Example:**
-```ruby
-# Delete with symbol confirmation (recommended)
-htm.forget(node_id, confirm: :confirmed)
-
-# Delete with boolean confirmation
-htm.forget(node_id, confirm: true)
-
-# Will raise error - confirmation required
-htm.forget(node_id)  # ArgumentError: Must confirm deletion
-htm.forget(node_id, confirm: false)  # ArgumentError: Must confirm deletion
-```
-
----
-
-### Complete Usage Example
-
-```ruby
-require 'htm'
-
-# Configure once globally (optional - uses defaults if not called)
-HTM.configure do |config|
-  config.embedding_provider = :ollama
-  config.embedding_model = 'nomic-embed-text'
-  config.tag_provider = :ollama
-  config.tag_model = 'llama3'
-end
-
-# Initialize for your robot
-htm = HTM.new(robot_name: "Assistant", working_memory_size: 128_000)
-
-# Store information
-htm.remember("PostgreSQL with pgvector for vector search", source: "architect")
-htm.remember("User prefers dark mode", source: "user")
-htm.remember("Use debug_me for debugging, not puts", source: "system")
-
-# Retrieve by time + topic
-recent = htm.recall(
-  "PostgreSQL",
-  timeframe: "last week",
-  strategy: :hybrid,
-  limit: 5
-)
-
-# Delete if needed (requires node ID from remember or recall)
-htm.forget(node_id, confirm: :confirmed)
-```
-
-## Use with Rails
-
-HTM is designed to integrate seamlessly with Ruby on Rails applications. It uses ActiveRecord models and follows Rails conventions, making it easy to add AI memory capabilities to existing Rails apps.
-
-### Why HTM Works Well with Rails
-
-**1. Uses ActiveRecord**
-- HTM models are standard ActiveRecord classes with associations, validations, and scopes
-- Follows Rails naming conventions and patterns
-- Models are namespaced under `HTM::Models::` to avoid conflicts
-
-**2. Standard Rails Configuration**
-- Uses `config/database.yml` (Rails convention)
-- Respects `RAILS_ENV` environment variable
-- Supports ERB in configuration files
-
-**3. Separate Database**
-- HTM uses its own PostgreSQL database connection
-- Won't interfere with your Rails app's database
-- Configured with `prepared_statements: false` and `advisory_locks: false` for compatibility
-- Thread-safe for Rails multi-threading
-
-**4. Connection Management**
-- Separate connection pool from your Rails app
-- Configurable pool size and timeouts
-- Proper cleanup and disconnection support
-
-### Integration Steps
-
-#### 1. Add to Gemfile
-
-```ruby
-gem 'htm'
-```
-
-Then run:
-```bash
-bundle install
-```
-
-#### 2. Configure Environment Variables
-
-Add HTM database configuration to your Rails environment. You can use `.env` files (with `dotenv-rails`) or set them directly:
+  <p>
+    <a href="https://rubygems.org/gems/htm"><img src="https://img.shields.io/gem/v/htm.svg" alt="Gem Version"></a>
+    <a href="https://github.com/madbomber/htm/blob/main/LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License"></a>
+  </p>
+</div>
 
-```ruby
-# .env or config/application.rb
-HTM_DBURL=postgresql://user:pass@localhost:5432/htm_production
-```
+<br/>
 
-Or use individual variables:
-```ruby
-HTM_DBHOST=localhost
-HTM_DBNAME=htm_production
-HTM_DBUSER=postgres
-HTM_DBPASS=password
-HTM_DBPORT=5432
-```
+## The Problem
 
-#### 3. Create Initializer
+Your AI assistants are brilliant but forgetful. Claude Desktop doesn't remember what Claude Code learned. Your custom chatbot can't recall what happened yesterday. Every tool starts fresh, every session.
 
-Create `config/initializers/htm.rb`:
+**HTM fixes this.**
 
-```ruby
-# config/initializers/htm.rb
+Connect all your MCP-enabled tools to the same long-term memory. What one robot learns, all robots remember. Context persists across sessions, tools, and time.
 
-# Configure HTM
-HTM.configure do |config|
-  # Use Rails logger
-  config.logger = Rails.logger
+## What is HTM?
 
-  # Embedding configuration (optional - uses Ollama defaults if not set)
-  config.embedding_provider = :ollama
-  config.embedding_model = 'nomic-embed-text'
+- **Hierarchical**: Organizes information from simple concepts to detailed relationships
+- **Temporal**: Retrieves memories across time—from moments ago to months past
+- **Memory**: Encodes, stores, and recalls information with semantic understanding
 
-  # Tag extraction configuration (optional - uses Ollama defaults if not set)
-  config.tag_provider = :ollama
-  config.tag_model = 'llama3'
+HTM is a Ruby gem that provides durable, shared memory for LLM-powered applications.
 
-  # Custom providers (optional)
-  # config.embedding_generator = ->(text) { YourEmbeddingService.call(text) }
-  # config.tag_extractor = ->(text, ontology) { YourTagService.call(text, ontology) }
-end
+## Key Capabilities
 
-# Establish HTM database connection
-HTM::ActiveRecordConfig.establish_connection!
+### MCP Server — Connect Any AI Tool
 
-# Verify required PostgreSQL extensions are installed
-HTM::ActiveRecordConfig.verify_extensions!
+HTM includes a Model Context Protocol server with 23 tools for memory management. Connect Claude Desktop, Claude Code, AIA, or any MCP-compatible client:
 
-Rails.logger.info "HTM initialized with database: #{HTM::Database.default_config[:database]}"
+```json
+{
+  "mcpServers": {
+    "htm": {
+      "command": "htm_mcp",
+      "env": { "HTM_DBURL": "postgresql://localhost/htm" }
+    }
+  }
+}
 ```
 
-#### 4. Set Up Database
-
-Run the HTM database setup:
+The `htm_mcp` executable includes CLI commands for database management:
 
 ```bash
-# Using HTM rake tasks
-rake htm:db:setup
-
-# Or manually in Rails console
-rails c
-> HTM::Database.setup
-```
-
-#### 5. Use in Your Rails App
-
-**In Controllers:**
-
-```ruby
-class ChatsController < ApplicationController
-  before_action :initialize_memory
-
-  def create
-    # Add user message to memory (embeddings + tags auto-extracted asynchronously)
-    @memory.remember(
-      params[:message],
-      source: "user-#{current_user.id}"
-    )
-
-    # Recall relevant context for the conversation
-    context = @memory.create_context(strategy: :balanced)
-
-    # Use context with your LLM to generate response
-    response = generate_llm_response(context, params[:message])
-
-    # Store assistant's response
-    @memory.remember(
-      response,
-      source: "assistant"
-    )
-
-    render json: { response: response }
-  end
-
-  private
-
-  def initialize_memory
-    @memory = HTM.new(
-      robot_name: "ChatBot-#{current_user.id}",
-      working_memory_size: 128_000
-    )
-  end
-end
+htm_mcp setup    # Initialize database schema
+htm_mcp verify   # Check connection and migrations
+htm_mcp stats    # Show memory statistics
+htm_mcp help     # Full help with environment variables
+htm_mcp          # Start MCP server (default)
 ```
 
-**In Background Jobs:**
-
-```ruby
-class ProcessDocumentJob < ApplicationJob
-  queue_as :default
-
-  def perform(document_id)
-    document = Document.find(document_id)
-
-    # Initialize HTM for this job
-    memory = HTM.new(robot_name: "DocumentProcessor")
-
-    # Store document content in memory (embeddings + tags auto-extracted asynchronously)
-    memory.remember(
-      document.content,
-      source: "document-#{document.id}"
-    )
-
-    # Recall related documents (uses vector similarity)
-    related = memory.recall(
-      document.category,
-      timeframe: "last month",
-      strategy: :vector
-    )
+Now your AI assistant can remember and recall across sessions:
+- Store decisions, preferences, and context
+- Search memories with natural language
+- Build knowledge that compounds over time
 
-    # Process with context...
-  end
-end
-```
+### Hive Mind — Shared Knowledge Across Robots
 
-**In Models (as a concern):**
+All robots share the same global memory. What one learns, all can access:
 
 ```ruby
-# app/models/concerns/memorable.rb
-module Memorable
-  extend ActiveSupport::Concern
-
-  included do
-    after_create :store_in_memory
-    after_update :update_memory
-  end
-
-  def store_in_memory
-    memory = HTM.new(robot_name: "Rails-#{Rails.env}")
-
-    memory.remember(
-      memory_content,
-      source: "#{self.class.name}-#{id}"
-    )
-  end
-
-  def update_memory
-    # Update existing memory node
-    store_in_memory
-  end
-
-  private
-
-  def memory_content
-    # Override in model to customize what gets stored
-    attributes.to_json
-  end
-end
-
-# Then in your model:
-class Article < ApplicationRecord
-  include Memorable
+# Claude Desktop remembers a user preference
+claude_desktop.remember("User prefers dark mode and concise responses")
 
-  private
-
-  def memory_content
-    "Article: #{title}\n\n#{body}\n\nCategory: #{category}"
-  end
-end
+# Later, Claude Code can recall it
+claude_code.recall("user preferences")
+# => "User prefers dark mode and concise responses"
 ```
 
-### Multi-Database Configuration (Rails 6+)
+Every tool builds on what came before. No more repeating yourself.
 
-If you want to use Rails' built-in multi-database support, you can configure HTM in your `config/database.yml`:
+### Long-Term Memory — Never Forget
 
-```yaml
-# config/database.yml
-production:
-  primary:
-    <<: *default
-    database: myapp_production
+Two-tier architecture ensures nothing is lost:
 
-  htm:
-    adapter: postgresql
-    encoding: unicode
-    database: <%= ENV['HTM_DBNAME'] || 'htm_production' %>
-    host: <%= ENV['HTM_DBHOST'] || 'localhost' %>
-    port: <%= ENV['HTM_DBPORT'] || 5432 %>
-    username: <%= ENV['HTM_DBUSER'] || 'postgres' %>
-    password: <%= ENV['HTM_DBPASS'] %>
-    pool: <%= ENV['HTM_DB_POOL_SIZE'] || 10 %>
-```
+- **Working Memory**: Token-limited context for immediate use
+- **Long-Term Memory**: Durable PostgreSQL storage with vector search
 
-Then update your HTM initializer:
+Memories persist forever unless explicitly deleted. Working memory evicts to long-term storage—it never deletes.
 
 ```ruby
-# config/initializers/htm.rb
-HTM::ActiveRecordConfig.establish_connection!
+htm.remember("PostgreSQL with pgvector handles our vector search")
 
-# Or use Rails' connection
-# ActiveRecord::Base.connected_to(role: :writing, shard: :htm) do
-#   HTM::ActiveRecordConfig.verify_extensions!
-# end
+# Months later...
+htm.recall("database decisions", timeframe: "last year")
+# => Still there
 ```
 
-### Testing with Rails
-
-**In RSpec:**
-
-```ruby
-# spec/rails_helper.rb
-RSpec.configure do |config|
-  config.before(:suite) do
-    HTM::ActiveRecordConfig.establish_connection!
-  end
-
-  config.after(:suite) do
-    HTM::ActiveRecordConfig.disconnect!
-  end
-end
-
-# spec/features/chat_spec.rb
-RSpec.describe "Chat with memory", type: :feature do
-  let(:memory) { HTM.new(robot_name: "TestBot") }
-
-  before do
-    memory.remember("User prefers Ruby over Python", source: "user")
-  end
+### Robot Groups — High Availability
 
-  it "recalls user preferences" do
-    context = memory.create_context(strategy: :balanced)
-    expect(context).to include("Ruby")
-  end
-end
-```
-
-**In Minitest:**
+Coordinate multiple robots with shared working memory and instant failover:
 
 ```ruby
-# test/test_helper.rb
-class ActiveSupport::TestCase
-  setup do
-    HTM::ActiveRecordConfig.establish_connection! unless HTM::ActiveRecordConfig.connected?
-  end
-end
-
-# test/integration/chat_test.rb
-class ChatTest < ActionDispatch::IntegrationTest
-  test "stores chat messages in memory" do
-    memory = HTM.new(robot_name: "TestBot")
-
-    post chats_path, params: { message: "Hello!" }
-
-    assert_response :success
-
-    # Verify message was stored
-    nodes = memory.recall("Hello", timeframe: "last hour")
-    assert_not_empty nodes
-  end
-end
-```
-
-### Production Considerations
-
-1. **Connection Pooling**: Set appropriate pool size based on your Rails app's concurrency:
-   ```ruby
-   ENV['HTM_DB_POOL_SIZE'] = '20'  # Match or exceed Rails pool
-   ```
-
-2. **Separate Database Server**: For production, use a dedicated PostgreSQL instance for HTM
-
-3. **Required Extensions**: Ensure `pgvector` and `pg_trgm` extensions are installed on your PostgreSQL server
-
-4. **Memory Management**: HTM's working memory is per-instance. Consider using a singleton pattern or Rails cache for shared instances
-
-5. **Background Processing**: Use Sidekiq or similar for embedding generation if processing large amounts of data
-
-### Example Rails App Structure
-
-```
-app/
-├── controllers/
-│   └── chats_controller.rb        # Uses HTM for conversation memory
-├── jobs/
-│   └── process_document_job.rb    # Background HTM processing
-├── models/
-│   ├── concerns/
-│   │   └── memorable.rb           # HTM integration concern
-│   └── article.rb                 # Includes Memorable
-└── services/
-    └── memory_service.rb          # Centralized HTM access
-
-config/
-├── initializers/
-│   └── htm.rb                     # HTM setup
-└── database.yml                   # Optional: multi-database config
-
-.env
-  HTM_DBURL=postgresql://...       # HTM database connection
-  OPENAI_API_KEY=sk-...            # If using OpenAI embeddings
-```
-
-## Environment Variables
-
-HTM uses environment variables for database and service configuration. These can be set in your shell profile, a `.env` file, or exported directly.
-
-### Database Configuration
-
-#### HTM_DBURL (Recommended)
-
-The preferred method for database configuration is a single connection URL:
-
-```bash
-export HTM_DBURL="postgresql://username:password@host:port/dbname?sslmode=require"
-```
-
-**Format:** `postgresql://USER:PASSWORD@HOST:PORT/DATABASE?sslmode=MODE`
-
-**Example for Local PostgreSQL:**
-```bash
-export HTM_DBURL="postgresql://postgres:postgres@localhost:5432/htm_dev?sslmode=disable"
-```
-
-#### Individual Database Parameters (Alternative)
-
-If `HTM_DBURL` is not set, HTM will use individual parameters:
-
-| Variable | Description | Default | Required |
-|----------|-------------|---------|----------|
-| `HTM_DBNAME` | Database name | None | Yes |
-| `HTM_DBUSER` | Database username | None | Yes |
-| `HTM_DBPASS` | Database password | None | Yes |
-| `HTM_DBHOST` | Database hostname or IP | `localhost` | No |
-| `HTM_DBPORT` | Database port | `5432` | No |
-| `HTM_SERVICE_NAME` | Service identifier (informational only) | None | No |
+group = HTM::RobotGroup.new(
+  name: 'support-ha',
+  active: ['primary'],
+  passive: ['standby']
+)
 
-**Example:**
-```bash
-export HTM_DBNAME="htm_production"
-export HTM_DBUSER="postgres"
-export HTM_DBPASS="secure_password_here"
-export HTM_DBHOST="localhost"
-export HTM_DBPORT="5432"
-export HTM_SERVICE_NAME="production-db"
+# Primary fails? Standby takes over instantly
+group.failover!
 ```
 
-**Configuration Priority:**
-1. `HTM_DBURL` (if set, this takes precedence)
-2. Individual parameters (`HTM_DBNAME`, `HTM_DBUSER`, etc.)
-3. Direct configuration passed to `HTM.new(db_config: {...})`
-
-### LLM Provider Configuration
-
-HTM configuration is done programmatically via `HTM.configure` (see [Configuration](#configuration) section). However, a few environment variables are still used:
-
-#### OPENAI_API_KEY
+Real-time synchronization via PostgreSQL LISTEN/NOTIFY. Add or remove robots dynamically.
 
-Required when using OpenAI as your embedding or tag extraction provider:
+## Quick Start
 
 ```bash
-export OPENAI_API_KEY="sk-proj-your-api-key-here"
+gem install htm
 ```
 
-This is required when you configure:
 ```ruby
-HTM.configure do |config|
-  config.embedding_provider = :openai
-  # or
-  config.tag_provider = :openai
-end
-```
-
-#### OLLAMA_URL
-
-Custom Ollama server URL (optional):
-
-```bash
-export OLLAMA_URL="http://localhost:11434"  # Default
-export OLLAMA_URL="http://ollama-server:11434"  # Custom
-```
-
-HTM defaults to `http://localhost:11434` if not set.
-
-#### HTM_LOG_LEVEL
-
-Control logging verbosity:
-
-```bash
-export HTM_LOG_LEVEL="DEBUG"  # DEBUG, INFO, WARN, ERROR, FATAL
-export HTM_LOG_LEVEL="INFO"   # Default
-```
-
-This is used by the default logger when `HTM.configure` is called without a custom logger.
-
-#### HTM_TELEMETRY_ENABLED
-
-Enable OpenTelemetry metrics collection:
-
-```bash
-export HTM_TELEMETRY_ENABLED="true"   # Enable telemetry
-export HTM_TELEMETRY_ENABLED="false"  # Disable (default)
-```
-
-When enabled, HTM emits metrics to configured OpenTelemetry collectors. See [Telemetry](#telemetry-opentelemetry) for details.
-
-### Quick Setup Examples
-
-#### Local Development (PostgreSQL)
-
-```bash
-# Simple local setup
-export HTM_DBURL="postgresql://postgres:postgres@localhost:5432/htm_dev?sslmode=disable"
-
-# With custom user
-export HTM_DBURL="postgresql://myuser:mypass@localhost:5432/htm_dev"
-```
-
-#### With OpenAI Embeddings
-
-```bash
-# Database + OpenAI
-export HTM_DBURL="postgresql://user:pass@host:port/db?sslmode=require"
-export OPENAI_API_KEY="sk-proj-your-api-key-here"
-```
-
-### Persistent Configuration
-
-To make environment variables permanent, add them to your shell profile:
-
-**For Bash (~/.bashrc or ~/.bash_profile):**
-```bash
-echo 'export HTM_DBURL="postgresql://user:pass@host:port/db?sslmode=require"' >> ~/.bashrc
-source ~/.bashrc
-```
-
-**For Zsh (~/.zshrc):**
-```bash
-echo 'export HTM_DBURL="postgresql://user:pass@host:port/db?sslmode=require"' >> ~/.zshrc
-source ~/.zshrc
-```
-
-**Or create a dedicated file (~/.bashrc__htm):**
-```bash
-# ~/.bashrc__htm
-export HTM_DBURL="postgresql://user:pass@host:port/db?sslmode=require"
-export OPENAI_API_KEY="sk-proj-your-api-key"
-export HTM_SERVICE_NAME="my-service"
-
-# Then source it from your main profile
-echo 'source ~/.bashrc__htm' >> ~/.bashrc
-```
-
-### Verification
-
-Verify your configuration:
-
-```bash
-# Check database connection
-ruby test_connection.rb
-
-# Or in Ruby
-irb -r ./lib/htm
-> HTM::Database.default_config
-> # Should return a hash with your connection parameters
-```
-
-## Development
-
-After checking out the repo, run:
-
-```bash
-# Install dependencies
-bundle install
-
-# Enable direnv (loads .envrc environment variables)
-direnv allow
-
-# Run tests
-rake test
-
-# Run example
-ruby examples/basic_usage.rb
-```
-
-**Important**: Make sure `direnv allow` has been run once in the project directory to load database environment variables from `.envrc`. Alternatively, you can manually export the environment variables:
-
-```bash
-export HTM_DBURL="postgresql://user:password@host:port/dbname?sslmode=require"
-```
-
-### HTM Rake Tasks Reference
-
-HTM provides comprehensive rake tasks for database management, documentation, file loading, job processing, and tag management. Use `rake -T htm` to see all available tasks.
-
-#### Database Tasks (`htm:db:*`)
-
-| Task | Description |
-|------|-------------|
-| `rake htm:db:setup` | Set up HTM database schema and run migrations (sets up from scratch) |
-| `rake htm:db:create` | Create database if it doesn't exist (respects RAILS_ENV) |
-| `rake htm:db:migrate` | Run pending database migrations |
-| `rake htm:db:status` | Show migration status |
-| `rake htm:db:verify` | Verify database connection (respects RAILS_ENV) |
-| `rake htm:db:info` | Show database info (size, tables, extensions) |
-| `rake htm:db:stats` | Show record counts for all HTM tables |
-| `rake htm:db:console` | Open PostgreSQL console (respects RAILS_ENV) |
-| `rake htm:db:seed` | Seed database with sample data |
-| `rake htm:db:schema:dump` | Dump current schema to db/schema.sql |
-| `rake htm:db:schema:load` | Load schema from db/schema.sql |
-| `rake htm:db:rebuild:embeddings` | Rebuild embeddings for all nodes |
-| `rake htm:db:rebuild:propositions` | Rebuild propositions for all non-proposition nodes |
-| `rake htm:db:tags:cleanup` | Soft delete orphaned tags and stale node_tags entries |
-| `rake htm:db:drop` | Drop all HTM tables (WARNING: destructive!) |
-| `rake htm:db:reset` | Drop and recreate database (WARNING: destructive!) |
-
-#### Documentation Tasks (`htm:doc:*`)
-
-| Task | Description |
-|------|-------------|
-| `rake htm:doc:all` | Generate DB docs, YARD API docs, build site, and serve |
-| `rake htm:doc:yard` | Build YARD API documentation (markdown format for MkDocs) |
-| `rake htm:doc:db` | Generate/update database documentation in docs/database/ |
-| `rake htm:doc:build` | Build documentation site with MkDocs |
-| `rake htm:doc:serve` | Serve documentation site locally with MkDocs |
-| `rake htm:doc:server[port]` | Start YARD documentation server (live reload) |
-| `rake htm:doc:fix_anchors` | Fix YARD anchor links for MkDocs compatibility |
-| `rake htm:doc:stats` | Show documentation coverage statistics |
-| `rake htm:doc:clean` | Clean generated documentation |
-
-#### File Loading Tasks (`htm:files:*`)
-
-| Task | Description |
-|------|-------------|
-| `rake htm:files:load[path]` | Load a markdown file into long-term memory |
-| `rake htm:files:load_dir[path,pattern]` | Load all markdown files from a directory |
-| `rake htm:files:list` | List all loaded file sources |
-| `rake htm:files:info[path]` | Show details for a loaded file |
-| `rake htm:files:sync` | Sync all loaded files (reload changed files) |
-| `rake htm:files:unload[path]` | Unload a file from memory |
-| `rake htm:files:stats` | Show file loading statistics |
-
-#### Job Processing Tasks (`htm:jobs:*`)
-
-| Task | Description |
-|------|-------------|
-| `rake htm:jobs:stats` | Show statistics for nodes and async job processing |
-| `rake htm:jobs:process_embeddings` | Process pending embedding jobs for nodes without embeddings |
-| `rake htm:jobs:process_tags` | Process pending tag extraction jobs for nodes without tags |
-| `rake htm:jobs:process_all` | Process all pending jobs (embeddings and tags) |
-| `rake htm:jobs:reprocess_embeddings` | Reprocess embeddings for all nodes (force regeneration) |
-| `rake htm:jobs:failed` | Show nodes that failed async processing |
-| `rake htm:jobs:clear_all` | Clear all embeddings and tags (for testing/development) |
-
-#### Tag Management Tasks (`htm:tags:*`)
-
-| Task | Description |
-|------|-------------|
-| `rake htm:tags:tree[prefix]` | Display tags as a hierarchical tree (text format) |
-| `rake htm:tags:mermaid[prefix]` | Export tags as Mermaid flowchart to tags.md |
-| `rake htm:tags:svg[prefix]` | Export tags as SVG visualization to tags.svg |
-| `rake htm:tags:export[prefix]` | Export tags in all formats (tags.txt, tags.md, tags.svg) |
-| `rake htm:tags:rebuild` | Rebuild all tags from node content |
-
-### Common Workflows
-
-```bash
-# Initial setup (first time)
-direnv allow
-rake htm:db:setup        # Creates schema and runs migrations
-
-# After pulling new migrations
-rake htm:db:migrate      # Run pending migrations
-
-# Check database state
-rake htm:db:status       # See migration status
-rake htm:db:info         # See database details
-rake htm:db:stats        # See record counts
-
-# Monitor async processing
-rake htm:jobs:stats
-rake htm:jobs:failed     # Show stuck jobs
-
-# Process pending jobs manually
-rake htm:jobs:process_all
-
-# Generate documentation
-rake htm:doc:all         # Full documentation build
-rake htm:doc:serve       # Local preview
-
-# Load files into memory
-rake 'htm:files:load[docs/guide.md]'
-rake 'htm:files:load_dir[docs/,**/*.md]'
-
-# View tag hierarchy
-rake htm:tags:tree
-rake 'htm:tags:svg[database]'  # Export filtered tags
-
-# Reset database (development only!)
-rake htm:db:reset        # Drops and recreates everything
-
-# Open database console for debugging
-rake htm:db:console      # Opens psql
-```
-
-See the [Async Job Processing](#async-job-processing) section for more details on background jobs.
-
-## Testing
+require 'htm'
 
-HTM uses Minitest:
+# Initialize
+htm = HTM.new(robot_name: "MyAssistant")
 
-```bash
-# Run all tests
-rake test
+# Remember
+htm.remember("User's project uses Rails 7 with PostgreSQL")
 
-# Run specific test file
-ruby test/htm_test.rb
+# Recall
+memories = htm.recall("tech stack", timeframe: "last month")
 ```
 
-## Architecture
-
-See [htm_teamwork.md](htm_teamwork.md) for detailed design documentation and planning notes.
-
-### Key Components
-
-- **HTM**: Main API class that coordinates all components and provides the primary interface
-- **Configuration**: Dependency injection for LLM providers (embedding_generator, tag_extractor, token_counter, logger)
-- **WorkingMemory**: In-memory, token-limited active context for immediate LLM use
-- **LongTermMemory**: PostgreSQL-backed permanent storage with connection pooling
-- **EmbeddingService**: Processing and validation layer for vector embeddings (calls configured `embedding_generator`)
-- **TagService**: Processing and validation layer for hierarchical tags (calls configured `tag_extractor`)
-- **Database**: Schema setup, migrations, and management
-- **Background Jobs**: Async processing for embedding generation and tag extraction
+For MCP integration, database setup, and configuration options, see the [full documentation](https://madbomber.github.io/htm).
 
-### Database Schema
+## Features at a Glance
 
-- `robots`: Robot registry for all LLM agents using HTM
-- `nodes`: Main memory storage with vector embeddings (pgvector), full-text search (tsvector), JSONB metadata
-- `tags`: Hierarchical tag ontology (format: `root:level1:level2:level3`)
-- `node_tags`: Join table implementing many-to-many relationship between nodes and tags
+| Feature | Description |
+|---------|-------------|
+| **MCP Server** | 23 tools for AI assistant integration |
+| **Hive Mind** | Shared memory across all robots |
+| **Vector Search** | Semantic retrieval with pgvector |
+| **Hybrid Search** | Combines vector + full-text matching |
+| **Temporal Queries** | "last week", "yesterday", date ranges |
+| **Auto-Tagging** | LLM extracts hierarchical tags automatically |
+| **Robot Groups** | High-availability with failover |
+| **Rails Integration** | Auto-configures via Railtie, uses ActiveJob |
+| **Telemetry** | Optional OpenTelemetry metrics |
 
-**Nodes Table Key Columns:**
-- `content`: The memory content
-- `embedding`: Vector embedding for semantic search (up to 2000 dimensions)
-- `metadata`: JSONB column for arbitrary key-value data (filterable via `@>` containment operator)
-- `content_hash`: SHA-256 hash for deduplication
+## Requirements
 
-### Service Architecture
+- Ruby 3.0+
+- PostgreSQL with pgvector and pg_trgm extensions
+- Ollama (default) or any local/private provider for embeddings and classifications
 
-HTM uses a layered architecture for LLM integration:
+## Documentation
 
-1. **Configuration Layer** (`HTM.configure`): Provides raw LLM access via `embedding_generator` and `tag_extractor` callables
-2. **Service Layer** (`EmbeddingService`, `TagService`): Processes and validates LLM responses, handles padding/truncation, formats for storage
-3. **Consumer Layer** (`GenerateEmbeddingJob`, `GenerateTagsJob`, rake tasks): Uses services for async or synchronous processing
-
-This separation allows you to provide any LLM implementation while HTM handles response processing, validation, and storage.
-
-## Roadmap
-
-- [x] Phase 1: Foundation (basic two-tier memory)
-- [x] Phase 2: RAG retrieval (semantic search)
-- [x] Phase 3: Relationships & tags
-- [x] Phase 4: Working memory management
-- [x] Phase 5: Hive mind features
-- [x] Phase 6: Operations & observability
-- [x] Phase 7: Advanced features
-- [x] Phase 8: Production-ready gem
+**[https://madbomber.github.io/htm](https://madbomber.github.io/htm)**
 
+- [Installation & Setup](https://madbomber.github.io/htm/getting-started/)
+- [MCP Server Guide](https://madbomber.github.io/htm/guides/mcp-server/)
+- [Rails Integration](https://madbomber.github.io/htm/guides/rails/)
+- [API Reference](https://madbomber.github.io/htm/api/)
 
 ## Why "Robots" Instead of "Agents"?
 
@@ -1677,7 +183,6 @@ HTM uses "robots" rather than the fashionable "agents" for several reasons:
 
 Honest terminology leads to clearer thinking. These are robots: tireless workers with perfect memory, executing instructions on our behalf. That's exactly what HTM helps them do better.
 
-
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/madbomber/htm.