htm 0.0.18 → 0.0.20

data/docs/examples/rails-integration.md

@@ -0,0 +1,173 @@
+# Rails Integration Example
+
+A full-stack Rails application demonstrating HTM integration with a compelling UI/UX for exploring and managing semantic memories.
+
+**Source:** [`examples/rails_app/`](https://github.com/madbomber/htm/tree/main/examples/rails_app)
+
+## Overview
+
+The HTM Memory Explorer is a Rails 7.1 application that demonstrates:
+
+- **Dashboard** - Overview of memory statistics, recent memories, top tags, and active robots
+- **Memories Browser** - Full CRUD for memories with search, filtering, soft delete, and restore
+- **Tag Visualization** - View tags as a list, text tree, or SVG hierarchy diagram
+- **Robots Management** - Manage LLM agents in the hive mind architecture
+- **Semantic Search** - Compare vector, full-text, and hybrid search strategies
+- **File Loading** - Load markdown files with automatic chunking and re-sync
+
+## Tech Stack
+
+- Rails 7.1 with Hotwire (Turbo + Stimulus)
+- Tailwind CSS for dark-themed UI
+- PostgreSQL with pgvector
+- HTM gem for semantic memory management
+
+## Setup
+
+```bash
+cd examples/rails_app
+
+# Install dependencies
+bundle install
+
+# Ensure HTM database is set up
+export HTM_DATABASE__URL="postgresql://localhost/htm_development"
+
+# Install frontend dependencies
+rails tailwindcss:install
+rails importmap:install
+rails turbo:install
+rails stimulus:install
+```
+
+## Running
+
+```bash
+# Start the Rails server with Tailwind CSS watching
+./bin/dev
+
+# Or start manually
+bundle exec rails server
+```
+
+Then open http://localhost:3000 in your browser.
+
+## Pages
+
+| Route | Description |
+|-------|-------------|
+| `/` | Dashboard with stats and overview |
+| `/memories` | Browse, search, and filter memories |
+| `/memories/new` | Add a new memory |
+| `/memories/deleted` | View and restore deleted memories |
+| `/tags` | Browse tags (list, tree, or diagram view) |
+| `/robots` | Manage robots and switch active robot |
+| `/search` | Semantic search playground |
+| `/files` | File loading and management |
+
+## How HTM Integration Works
+
+### Automatic Rails Integration
+
+The HTM gem includes a Rails Railtie (`lib/htm/railtie.rb`) that automatically:
+
+1. Configures HTM to use `Rails.logger`
+2. Sets job backend to `:active_job` (or `:inline` in test env)
+3. Loads HTM rake tasks
+4. Verifies database connection in development
+
+### Controller Helper
+
+The `ApplicationController` provides a `htm` helper method:
+
+```ruby
+class ApplicationController < ActionController::Base
+  private
+
+  def htm
+    @htm ||= HTM.new(
+      robot_name: session[:robot_name] || "web_user_#{session.id}"
+    )
+  end
+  helper_method :htm
+end
+```
+
+### Using HTM in Controllers
+
+```ruby
+class MemoriesController < ApplicationController
+  def create
+    node_id = htm.remember(
+      params[:content],
+      tags: params[:tags]&.split(','),
+      metadata: { category: params[:category] }
+    )
+    redirect_to memories_path, notice: "Memory created (ID: #{node_id})"
+  end
+
+  def index
+    @memories = if params[:search].present?
+      htm.recall(params[:search], strategy: :hybrid, limit: 50, raw: true)
+    else
+      HTM::Models::Node.order(created_at: :desc).limit(50)
+    end
+  end
+
+  def destroy
+    htm.forget(params[:id])  # Soft delete by default
+    redirect_to memories_path, notice: "Memory deleted"
+  end
+
+  def restore
+    htm.restore(params[:id])
+    redirect_to memories_path, notice: "Memory restored"
+  end
+end
+```
+
+### Tag Visualization
+
+```ruby
+class TagsController < ApplicationController
+  def index
+    @tags = HTM::Models::Tag.all
+
+    respond_to do |format|
+      format.html
+      format.text { render plain: @tags.tree_string }
+      format.svg { render plain: @tags.tree_svg }
+    end
+  end
+end
+```
+
+## Features Demonstrated
+
+| HTM Feature | UI Component |
+|-------------|--------------|
+| `htm.remember()` | Memories > New form |
+| `htm.recall()` | Search page with strategy selector |
+| `htm.forget()` | Delete button (soft delete) |
+| `htm.restore()` | Restore button in deleted view |
+| `htm.load_file()` | Files page with file browser |
+| Tag hierarchy | Tags page with tree/SVG views |
+| Robot management | Robots page with activity stats |
+
+## Development Notes
+
+The app uses:
+
+- `propshaft` for asset pipeline
+- `importmap-rails` for JavaScript
+- `tailwindcss-rails` for styling
+- `kaminari` for pagination
+- `turbo-rails` for SPA-like navigation
+
+No JavaScript build step required.
+
+## See Also
+
+- [Getting Started Guide](../guides/getting-started.md)
+- [Multi-Robot Systems](../robots/multi-robot.md)
+- [Search Strategies](../guides/search-strategies.md)

data/docs/examples/robot-groups.md

@@ -0,0 +1,210 @@
+# Robot Groups Example
+
+This example demonstrates coordinating multiple robots with shared working memory and automatic failover capabilities.
+
+**Source:** [`examples/robot_groups/same_process.rb`](https://github.com/madbomber/htm/blob/main/examples/robot_groups/same_process.rb)
+
+## Overview
+
+Robot Groups enable:
+
+- **Shared Working Memory**: Multiple robots share the same in-memory context
+- **Active/Passive Roles**: Active robots handle requests; passive robots maintain synchronized context
+- **Instant Failover**: When an active robot fails, passive robots take over with full context
+- **Real-time Sync**: PostgreSQL LISTEN/NOTIFY enables real-time synchronization
+- **Dynamic Scaling**: Add or remove robots on demand
+
+## Running the Example
+
+```bash
+export HTM_DATABASE__URL="postgresql://user@localhost:5432/htm_development"
+ruby examples/robot_groups/same_process.rb
+```
+
+## Code Walkthrough
+
+### Create a Robot Group
+
+```ruby
+group = HTM::RobotGroup.new(
+  name: 'customer-support-ha',
+  active: ['support-primary'],      # Active robot names
+  passive: ['support-standby'],     # Standby robot names
+  max_tokens: 8000                  # Token limit for shared memory
+)
+```
+
+### Add Shared Memories
+
+```ruby
+# Memories are automatically shared across all robots in the group
+group.remember(
+  'Customer account #12345 prefers email communication.',
+  originator: 'support-primary'
+)
+
+group.remember(
+  'Open ticket #789: Billing discrepancy reported.',
+  originator: 'support-primary'
+)
+```
+
+### Check Synchronization Status
+
+```ruby
+status = group.status
+# => {
+#   name: 'customer-support-ha',
+#   active: ['support-primary'],
+#   passive: ['support-standby'],
+#   working_memory_nodes: 3,
+#   token_utilization: 0.15,
+#   in_sync: true
+# }
+```
+
+### Simulate Failover
+
+```ruby
+# Primary robot fails
+puts "Primary robot stopped responding!"
+
+# Failover to standby
+group.failover!
+
+status = group.status
+# Active robots now: ['support-standby']
+# Passive robots now: []
+```
+
+### Recall with Full Context
+
+```ruby
+# Standby has full context after failover
+memories = group.recall('customer', limit: 5, strategy: :fulltext)
+# => All memories are available immediately
+```
+
+### Scale Up (Add Robots)
+
+```ruby
+# Add a second active robot
+group.add_active('support-secondary')
+group.sync_robot('support-secondary')
+
+# Both robots now share the same working memory
+```
+
+### Collaborative Memory
+
+```ruby
+# Either robot can add memories that sync to all
+group.remember(
+  'Issue escalated to billing department.',
+  originator: 'support-secondary'
+)
+# Automatically synced to all group members
+```
+
+## Real-Time Sync
+
+Robot Groups use PostgreSQL LISTEN/NOTIFY for real-time synchronization:
+
+```ruby
+sync_stats = group.sync_stats
+# => {
+#   nodes_synced: 5,
+#   evictions_synced: 0,
+#   notifications_received: 8
+# }
+
+# Check listener status
+group.channel.listening?  # => true
+```
+
+## High Availability Pattern
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    RobotGroup                           │
+│                'customer-support-ha'                    │
+├─────────────────────────────────────────────────────────┤
+│                                                         │
+│  ┌─────────────────┐     ┌─────────────────┐           │
+│  │  support-primary│     │ support-standby │           │
+│  │     (ACTIVE)    │     │    (PASSIVE)    │           │
+│  └────────┬────────┘     └────────┬────────┘           │
+│           │                       │                     │
+│           └───────────┬───────────┘                     │
+│                       │                                 │
+│            ┌──────────▼──────────┐                     │
+│            │  Shared Working     │                     │
+│            │      Memory         │                     │
+│            └──────────┬──────────┘                     │
+│                       │                                 │
+│            ┌──────────▼──────────┐                     │
+│            │ PostgreSQL          │                     │
+│            │ LISTEN/NOTIFY       │                     │
+│            └─────────────────────┘                     │
+└─────────────────────────────────────────────────────────┘
+```
+
+## Use Cases
+
+### Customer Support
+
+Multiple support agents share context about ongoing conversations:
+
+```ruby
+group = HTM::RobotGroup.new(
+  name: 'support-team',
+  active: ['agent-1', 'agent-2', 'agent-3'],
+  passive: ['backup-agent'],
+  max_tokens: 16000
+)
+```
+
+### Load Balancing
+
+Distribute queries across multiple active robots:
+
+```ruby
+group = HTM::RobotGroup.new(
+  name: 'query-handlers',
+  active: ['handler-1', 'handler-2'],
+  passive: [],
+  max_tokens: 8000
+)
+```
+
+### Disaster Recovery
+
+Keep a warm standby ready to take over:
+
+```ruby
+group = HTM::RobotGroup.new(
+  name: 'primary-with-dr',
+  active: ['primary'],
+  passive: ['dr-standby'],
+  max_tokens: 32000
+)
+
+# On primary failure
+group.failover!
+```
+
+## Cleanup
+
+```ruby
+# Clear shared working memory
+group.clear_working_memory
+
+# Stop the LISTEN/NOTIFY listener
+group.shutdown
+```
+
+## See Also
+
+- [Multi-Robot Usage Guide](../robots/multi-robot.md)
+- [Hive Mind Architecture](../robots/hive-mind.md)
+- [Working Memory API](../api/working-memory.md)

data/docs/examples/sinatra-integration.md

@@ -0,0 +1,218 @@
+# Sinatra Integration Example
+
+A Sinatra web application demonstrating HTM integration with Sidekiq for background job processing.
+
+**Source:** [`examples/sinatra_app/`](https://github.com/madbomber/htm/tree/main/examples/sinatra_app)
+
+## Overview
+
+The Sinatra example demonstrates:
+
+- HTM integration with Sinatra using `register_htm` helper
+- Session-based robot identification
+- RESTful API endpoints for memory operations
+- Sidekiq for background embedding and tag generation
+- Thread-safe concurrent request handling
+
+## Prerequisites
+
+```bash
+# PostgreSQL with pgvector
+export HTM_DATABASE__URL="postgresql://user@localhost:5432/htm_development"
+
+# Redis for Sidekiq (optional)
+export REDIS_URL="redis://localhost:6379/0"
+
+# Ollama for embeddings and tags
+ollama pull nomic-embed-text
+ollama pull gemma3:latest
+```
+
+## Setup
+
+```bash
+cd examples/sinatra_app
+
+# Install dependencies
+bundle install
+```
+
+## Running
+
+```bash
+# Start the Sinatra app
+bundle exec ruby app.rb
+
+# In a separate terminal, start Sidekiq (optional, for async processing)
+bundle exec sidekiq
+```
+
+Then open http://localhost:4567 in your browser.
+
+## Code Walkthrough
+
+### HTM Registration
+
+```ruby
+require 'sinatra'
+require_relative '../../lib/htm'
+require_relative '../../lib/htm/integrations/sinatra'
+
+class HTMApp < Sinatra::Base
+  # Register HTM with automatic configuration
+  register_htm
+
+  # Enable sessions for robot identification
+  enable :sessions
+  set :session_secret, ENV.fetch('SESSION_SECRET', SecureRandom.hex(64))
+end
+```
+
+### Request Initialization
+
+```ruby
+before do
+  # Use session ID as robot identifier
+  robot_name = session[:robot_id] ||= SecureRandom.uuid[0..7]
+  init_htm(robot_name: "web_user_#{robot_name}")
+
+  # Set content type for API responses
+  content_type :json if request.path.start_with?('/api/')
+end
+```
+
+### API Endpoints
+
+```ruby
+# Remember information
+post '/api/remember' do
+  content = params[:content]
+
+  unless content && !content.empty?
+    halt 400, json(error: 'Content is required')
+  end
+
+  node_id = remember(content)
+
+  json(
+    status: 'ok',
+    node_id: node_id,
+    message: 'Memory stored. Embedding and tags are being generated in background.'
+  )
+end
+
+# Recall memories
+get '/api/recall' do
+  topic = params[:topic]
+  limit = (params[:limit] || 10).to_i
+  strategy = (params[:strategy] || 'hybrid').to_sym
+
+  unless topic && !topic.empty?
+    halt 400, json(error: 'Topic is required')
+  end
+
+  memories = recall(topic, strategy: strategy, limit: limit)
+
+  json(memories: memories)
+end
+```
+
+## API Reference
+
+### POST /api/remember
+
+Store information in memory.
+
+**Parameters:**
+- `content` (required) - The text content to remember
+
+**Response:**
+```json
+{
+  "status": "ok",
+  "node_id": 42,
+  "message": "Memory stored. Embedding and tags are being generated in background."
+}
+```
+
+### GET /api/recall
+
+Recall memories matching a topic.
+
+**Parameters:**
+- `topic` (required) - Search topic
+- `limit` (optional) - Max results (default: 10)
+- `strategy` (optional) - Search strategy: `vector`, `fulltext`, or `hybrid` (default: hybrid)
+- `timeframe` (optional) - Time range in seconds
+
+**Response:**
+```json
+{
+  "memories": [
+    "PostgreSQL supports vector search via pgvector...",
+    "The HTM gem provides intelligent memory management..."
+  ]
+}
+```
+
+### GET /api/stats
+
+Get memory statistics.
+
+**Response:**
+```json
+{
+  "total_nodes": 150,
+  "total_robots": 5,
+  "total_tags": 42,
+  "working_memory_nodes": 12
+}
+```
+
+## Sidekiq Configuration
+
+```ruby
+require 'sidekiq'
+
+Sidekiq.configure_server do |config|
+  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0') }
+end
+
+Sidekiq.configure_client do |config|
+  config.redis = { url: ENV.fetch('REDIS_URL', 'redis://localhost:6379/0') }
+end
+```
+
+HTM automatically uses Sidekiq for background jobs when available:
+
+- `GenerateEmbeddingJob` - Generates vector embeddings
+- `GenerateTagsJob` - Extracts hierarchical tags via LLM
+
+## Environment Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `HTM_DATABASE__URL` | (required) | PostgreSQL connection URL |
+| `REDIS_URL` | `redis://localhost:6379/0` | Redis URL for Sidekiq |
+| `OLLAMA_URL` | `http://localhost:11434` | Ollama server URL |
+| `SESSION_SECRET` | (random) | Session encryption secret |
+
+## Testing with curl
+
+```bash
+# Remember something
+curl -X POST http://localhost:4567/api/remember \
+  -d "content=PostgreSQL supports vector similarity search"
+
+# Recall memories
+curl "http://localhost:4567/api/recall?topic=database&strategy=hybrid&limit=5"
+
+# Get stats
+curl http://localhost:4567/api/stats
+```
+
+## See Also
+
+- [Getting Started Guide](../guides/getting-started.md)
+- [MCP Server Guide](../guides/mcp-server.md)
+- [Configuration Guide](../guides/configuration.md)