htm 0.0.18 → 0.0.30

data/examples/README.md

@@ -2,20 +2,35 @@
 
 This directory contains example applications demonstrating various ways to use the HTM (Hierarchical Temporal Memory) gem.
 
-## Prerequisites
+## Quick Start
+
+All examples use the `htm_examples` database, isolated from development and production data.
+
+```bash
+# One-time setup: create and configure examples database
+rake examples:setup
+
+# Run the basic example
+rake examples:basic
+
+# Run all standalone examples
+rake examples:all
+
+# Check examples database status
+rake examples:status
+```
 
-All examples require:
+## Prerequisites
 
 1. **PostgreSQL Database** with pgvector extension:
    ```bash
-   export HTM_DATABASE__URL="postgresql://user@localhost:5432/htm_development"
+   # Create the examples database (done automatically by rake examples:setup)
+   createdb htm_examples
+   psql htm_examples -c "CREATE EXTENSION IF NOT EXISTS vector; CREATE EXTENSION IF NOT EXISTS pg_trgm;"
    ```
 
-   > **Note**: Database selection now respects `RAILS_ENV`. If `RAILS_ENV` is set,
-   > HTM extracts the base name from `HTM_DATABASE__URL` and appends the environment suffix.
-   > For example, with `HTM_DATABASE__URL=...htm_development` and `HTM_ENV=test`, HTM
-   > connects to `htm_test`. When `RAILS_ENV` is unset (typical for examples),
-   > behavior is unchanged.
+   > **Note**: All examples use `HTM_ENV=examples` which connects to `htm_examples` database.
+   > This isolation prevents examples from polluting development or production data.
 
 2. **Ollama** (recommended for local LLM):
    ```bash
@@ -28,18 +43,62 @@ All examples require:
    bundle install
    ```
 
+## Available Rake Tasks
+
+```bash
+rake examples:setup    # Set up examples database (create + setup schema)
+rake examples:reset    # Reset examples database (drop + create + setup)
+rake examples:basic    # Run basic_usage example
+rake examples:all      # Run all standalone examples
+rake examples:status   # Show examples database status
+rake example           # Alias for examples:basic
+```
+
 ---
 
-## Standalone Scripts
+## Example Progression
+
+Examples are numbered to indicate recommended learning order, from basic concepts to advanced features.
+
+### 00_create_examples_db.rb
+
+**Database setup for examples.**
+
+Creates and configures the `htm_examples` database. Run this first before other examples.
+
+```bash
+ruby examples/00_create_examples_db.rb
+```
+
+---
+
+### 01_basic_usage.rb
+
+**Getting started with HTM fundamentals.**
+
+Demonstrates the core API: configuring HTM, registering a robot, and using the three primary methods (`remember`, `recall`, `forget`).
+
+```bash
+ruby examples/01_basic_usage.rb
+```
+
+**Features:**
+- HTM configuration with Ollama provider
+- Robot initialization
+- Storing memories with `remember()`
+- Retrieving memories with `recall()` using timeframes
+- Understanding async embedding/tag generation
+
+---
 
-### config_file_example/
+### 02_config_file_example/
 
 **Configuration management with source tracing.**
 
 Demonstrates how HTM uses `anyway_config` for layered configuration from multiple sources, with source tracing to show where each value originated.
 
 ```bash
-cd examples/config_file_example
+cd examples/02_config_file_example
 ruby show_config.rb
 ```
 
@@ -65,37 +124,18 @@ HTM_EMBEDDING__MODEL=mxbai-embed-large ruby show_config.rb
 HTM_ENV=production ruby show_config.rb
 ```
 
-See [config_file_example/README.md](config_file_example/README.md) for detailed documentation.
-
----
-
-### basic_usage.rb
-
-**Getting started with HTM fundamentals.**
-
-Demonstrates the core API: configuring HTM, registering a robot, and using the three primary methods (`remember`, `recall`, `forget`).
-
-```bash
-ruby examples/basic_usage.rb
-```
-
-**Features:**
-- HTM configuration with Ollama provider
-- Robot initialization
-- Storing memories with `remember()`
-- Retrieving memories with `recall()` using timeframes
-- Understanding async embedding/tag generation
+See [02_config_file_example/README.md](02_config_file_example/README.md) for detailed documentation.
 
 ---
 
-### custom_llm_configuration.rb
+### 03_custom_llm_configuration.rb
 
 **Flexible LLM integration patterns.**
 
 Shows how to configure HTM with custom embedding and tag generation methods, supporting multiple LLM providers or custom infrastructure.
 
 ```bash
-ruby examples/custom_llm_configuration.rb
+ruby examples/03_custom_llm_configuration.rb
 ```
 
 **Features:**
@@ -108,14 +148,14 @@ ruby examples/custom_llm_configuration.rb
 
 ---
 
-### file_loader_usage.rb
+### 04_file_loader_usage.rb
 
 **Loading documents into long-term memory.**
 
 Demonstrates loading markdown files with automatic paragraph chunking, YAML frontmatter extraction, and source tracking for re-sync.
 
 ```bash
-ruby examples/file_loader_usage.rb
+ruby examples/04_file_loader_usage.rb
 ```
 
 **Features:**
@@ -129,14 +169,14 @@ ruby examples/file_loader_usage.rb
 
 ---
 
-### timeframe_demo.rb
+### 05_timeframe_demo.rb
 
 **Flexible time-based filtering for recall.**
 
 Comprehensive demonstration of all timeframe options supported by `recall()`, including natural language parsing.
 
 ```bash
-ruby examples/timeframe_demo.rb
+ruby examples/05_timeframe_demo.rb
 ```
 
 **Features:**
@@ -150,97 +190,101 @@ ruby examples/timeframe_demo.rb
 
 ---
 
-### telemetry/
+### 06_example_app/
 
-**Live Grafana dashboard for HTM metrics.**
+**Full-featured HTM demonstration with RubyLLM integration.**
 
-A complete telemetry demo using Homebrew-installed Prometheus and Grafana to visualize HTM metrics in real-time graphs.
+A standalone application showing complete HTM workflow with database connection, Ollama integration, memory operations, and multiple search strategies.
 
 ```bash
-cd examples/telemetry
-ruby demo.rb
+ruby examples/06_example_app/app.rb
 ```
 
-The demo automatically:
-- Checks/installs Prometheus and Grafana via Homebrew
-- Starts both services if not running
-- Configures Prometheus to scrape the demo's metrics
-- Cleans up previous demo data
-- Opens Grafana in your browser
-
 **Features:**
-- Live updating dashboard with job counts, latencies, cache hit rates
-- Pre-built Grafana dashboard JSON
-- No Docker required - uses native macOS services
+- Database connection verification
+- RubyLLM configuration for embeddings and tags
+- Async embedding/tag generation with wait
+- Comparison of search strategies (:fulltext, :vector, :hybrid)
+- Detailed output of generated tags and embeddings
+
+---
+
+### 07_cli_app/
+
+**Interactive command-line application.**
+
+A REPL-style CLI demonstrating synchronous job execution with the `:inline` backend, ideal for CLI tools and scripts.
 
-**Dependencies:**
 ```bash
-brew install prometheus grafana
-gem install prometheus-client webrick
+ruby examples/07_cli_app/htm_cli.rb
 ```
 
-See [telemetry/README.md](telemetry/README.md) for detailed setup instructions.
+**Commands:**
+- `remember <text>` - Store information (waits for embedding/tags)
+- `recall <topic>` - Hybrid search with LLM-powered response generation
+- `tags [prefix]` - List all tags with linked node content
+- `stats` - Memory statistics
+- `help` - Show help
+- `exit` - Quit
 
----
+**Features:**
+- Synchronous job execution (`:inline` backend)
+- Real-time progress feedback
+- Tag extraction visibility during search
+- Hybrid search with scoring (similarity, tag_boost, combined)
+- RubyLLM chat integration for context-aware responses
+- Response storage in long-term memory
 
-### mcp_server.rb & mcp_client.rb
+See [07_cli_app/README.md](07_cli_app/README.md) for detailed documentation.
 
-**Model Context Protocol (MCP) integration for AI assistants.**
+---
 
-A pair of examples demonstrating how to expose HTM as an MCP server and connect to it from a chat client. This enables AI assistants like Claude Desktop to use HTM's memory capabilities.
+### 08_sinatra_app/
 
-#### mcp_server.rb
+**Web application with Sidekiq background processing.**
 
-An MCP server that exposes HTM's memory operations as tools:
+A Sinatra-based web application demonstrating HTM in a multi-user web context with async job processing.
 
 ```bash
-ruby examples/mcp_server.rb
+cd examples/08_sinatra_app
+bundle install
+bundle exec ruby app.rb
 ```
 
-**Tools exposed:**
-- `SetRobotTool` - Set the robot identity for this session (call first)
-- `GetRobotTool` - Get current robot information
-- `GetWorkingMemoryTool` - Get working memory contents for session restore
-- `RememberTool` - Store information with optional tags and metadata
-- `RecallTool` - Search memories using vector, fulltext, or hybrid strategies
-- `ForgetTool` - Soft-delete a memory (recoverable)
-- `RestoreTool` - Restore a soft-deleted memory
-- `ListTagsTool` - List tags with optional prefix filtering
-- `StatsTool` - Get memory usage statistics
-
-**Resources exposed:**
-- `htm://statistics` - Memory statistics as JSON
-- `htm://tags/hierarchy` - Tag hierarchy as text tree
-- `htm://memories/recent` - Last 20 memories
-
-**Claude Desktop configuration** (`~/.config/claude/claude_desktop_config.json`):
-```json
-{
-  "mcpServers": {
-    "htm-memory": {
-      "command": "ruby",
-      "args": ["/path/to/htm/examples/mcp_server.rb"],
-      "env": {
-        "HTM_DATABASE__URL": "postgresql://user@localhost:5432/htm_development"
-      }
-    }
-  }
-}
-```
+**Features:**
+- Sidekiq integration for background jobs
+- Session-based robot identification
+- RESTful API endpoints:
+  - `POST /api/remember` - Store information
+  - `GET /api/recall` - Search memories with timeframe filtering
+  - `GET /api/stats` - Memory statistics
+  - `GET /api/tags` - Tag tree structure
+  - `GET /api/health` - Health check
+- Interactive HTML UI with hybrid search scoring display
+- Tag tree visualization
 
-#### mcp_client.rb
+**Environment Variables:**
+- `HTM_DATABASE__URL` - PostgreSQL connection (required)
+- `REDIS_URL` - Redis for Sidekiq (default: redis://localhost:6379/0)
+- `SESSION_SECRET` - Session encryption key
 
-An interactive chat client that connects to the MCP server via STDIO and uses a local Ollama model (gpt-oss) for conversation:
+---
+
+### 09_mcp_client.rb
+
+**Model Context Protocol (MCP) client for AI assistants.**
+
+An interactive chat client that connects to an MCP server via STDIO and uses a local Ollama model for conversation. This enables AI assistants like Claude Desktop to use HTM's memory capabilities.
 
 ```bash
-ruby examples/mcp_client.rb
+ruby examples/09_mcp_client.rb
 ```
 
 **Features:**
 - Prompts for robot name on startup (or uses `HTM_ROBOT_NAME` env var)
 - Calls `SetRobotTool` to establish robot identity with the server
 - Offers to restore previous session from working memory
-- Connects to `mcp_server.rb` automatically via STDIO transport
+- Connects to MCP server automatically via STDIO transport
 - Interactive chat loop with tool calling
 - LLM decides when to remember/recall information
 - Logs tool calls and results for visibility
@@ -254,7 +298,7 @@ ruby examples/mcp_client.rb
 
 **Example startup and conversation:**
 ```
-$ ruby examples/mcp_client.rb
+$ ruby examples/09_mcp_client.rb
 Connecting to HTM MCP server...
 [✓] Connected to HTM MCP server
 [✓] Found 9 tools:
@@ -304,89 +348,40 @@ ollama pull gpt-oss  # Or your preferred model
 
 ---
 
-## Application Examples
+### 10_telemetry/
 
-### example_app/
-
-**Full-featured HTM demonstration with RubyLLM integration.**
+**Live Grafana dashboard for HTM metrics.**
 
-A standalone application showing complete HTM workflow with database connection, Ollama integration, memory operations, and multiple search strategies.
+A complete telemetry demo using Homebrew-installed Prometheus and Grafana to visualize HTM metrics in real-time graphs.
 
 ```bash
-ruby examples/example_app/app.rb
+cd examples/10_telemetry
+ruby demo.rb
 ```
 
-**Features:**
-- Database connection verification
-- RubyLLM configuration for embeddings and tags
-- Async embedding/tag generation with wait
-- Comparison of search strategies (:fulltext, :vector, :hybrid)
-- Detailed output of generated tags and embeddings
-
----
-
-### sinatra_app/
-
-**Web application with Sidekiq background processing.**
-
-A Sinatra-based web application demonstrating HTM in a multi-user web context with async job processing.
-
-```bash
-cd examples/sinatra_app
-bundle install
-bundle exec ruby app.rb
-```
+The demo automatically:
+- Checks/installs Prometheus and Grafana via Homebrew
+- Starts both services if not running
+- Configures Prometheus to scrape the demo's metrics
+- Cleans up previous demo data
+- Opens Grafana in your browser
 
 **Features:**
-- Sidekiq integration for background jobs
-- Session-based robot identification
-- RESTful API endpoints:
-  - `POST /api/remember` - Store information
-  - `GET /api/recall` - Search memories with timeframe filtering
-  - `GET /api/stats` - Memory statistics
-  - `GET /api/tags` - Tag tree structure
-  - `GET /api/health` - Health check
-- Interactive HTML UI with hybrid search scoring display
-- Tag tree visualization
-
-**Environment Variables:**
-- `HTM_DATABASE__URL` - PostgreSQL connection (required)
-- `REDIS_URL` - Redis for Sidekiq (default: redis://localhost:6379/0)
-- `SESSION_SECRET` - Session encryption key
-
----
-
-### cli_app/
-
-**Interactive command-line application.**
-
-A REPL-style CLI demonstrating synchronous job execution with the `:inline` backend, ideal for CLI tools and scripts.
+- Live updating dashboard with job counts, latencies, cache hit rates
+- Pre-built Grafana dashboard JSON
+- No Docker required - uses native macOS services
 
+**Dependencies:**
 ```bash
-ruby examples/cli_app/htm_cli.rb
+brew install prometheus grafana
+gem install prometheus-client webrick
 ```
 
-**Commands:**
-- `remember <text>` - Store information (waits for embedding/tags)
-- `recall <topic>` - Hybrid search with LLM-powered response generation
-- `tags [prefix]` - List all tags with linked node content
-- `stats` - Memory statistics
-- `help` - Show help
-- `exit` - Quit
-
-**Features:**
-- Synchronous job execution (`:inline` backend)
-- Real-time progress feedback
-- Tag extraction visibility during search
-- Hybrid search with scoring (similarity, tag_boost, combined)
-- RubyLLM chat integration for context-aware responses
-- Response storage in long-term memory
-
-See [cli_app/README.md](cli_app/README.md) for detailed documentation.
+See [10_telemetry/README.md](10_telemetry/README.md) for detailed setup instructions.
 
 ---
 
-### robot_groups/
+### 11_robot_groups/
 
 **Multi-robot coordination with shared working memory.**
 
@@ -397,7 +392,7 @@ Demonstrates high-availability patterns with shared working memory, failover, an
 Single-process demonstration of robot groups:
 
 ```bash
-ruby examples/robot_groups/same_process.rb
+ruby examples/11_robot_groups/same_process.rb
 ```
 
 **Scenarios demonstrated:**
@@ -415,7 +410,7 @@ ruby examples/robot_groups/same_process.rb
 Cross-process demonstration with separate Ruby processes:
 
 ```bash
-ruby examples/robot_groups/multi_process.rb
+ruby examples/11_robot_groups/multi_process.rb
 ```
 
 **Scenarios demonstrated:**
@@ -433,45 +428,68 @@ ruby examples/robot_groups/multi_process.rb
 
 ---
 
+### 12_rails_app/
+
+**Rails integration example.**
+
+A complete Rails application demonstrating HTM integration with Rails controllers, views, and background jobs.
+
+```bash
+cd examples/12_rails_app
+bundle install
+bin/rails db:create db:migrate
+bin/dev
+```
+
+See [12_rails_app/README.md](12_rails_app/README.md) for detailed documentation.
+
+---
+
 ## Directory Structure
 
 ```
 examples/
-├── README.md                      # This file
-├── basic_usage.rb                 # Core API demonstration
-├── custom_llm_configuration.rb    # LLM integration patterns
-├── file_loader_usage.rb           # Document loading
-├── timeframe_demo.rb              # Time-based filtering
-├── config_file_example/
-│   ├── show_config.rb             # Config source tracing demo
-│   ├── custom_config.yml          # Example for HTM_CONF
-│   ├── README.md                  # Configuration documentation
+├── README.md                           # This file
+├── examples_helper.rb                  # Shared setup for all examples
+├── 00_create_examples_db.rb            # Database setup
+├── 01_basic_usage.rb                   # Core API demonstration
+├── 02_config_file_example/
+│   ├── show_config.rb                  # Config source tracing demo
+│   ├── custom_config.yml               # Example for HTM_CONF
+│   ├── README.md                       # Configuration documentation
 │   └── config/
-│       └── htm.local.yml          # Auto-loaded local overrides
-├── telemetry/
-│   ├── demo.rb                    # Live Grafana metrics dashboard
-│   ├── README.md
-│   ├── SETUP_README.md
-│   └── grafana/dashboards/htm-metrics.json
-├── mcp_server.rb                  # MCP server exposing HTM tools
-├── mcp_client.rb                  # MCP client with chat interface
-├── example_app/
-│   ├── app.rb                     # Full-featured demo app
+│       └── htm.local.yml               # Auto-loaded local overrides
+├── 03_custom_llm_configuration.rb      # LLM integration patterns
+├── 04_file_loader_usage.rb             # Document loading
+├── 05_timeframe_demo.rb                # Time-based filtering
+├── 06_example_app/
+│   ├── app.rb                          # Full-featured demo app
 │   └── Rakefile
-├── sinatra_app/
-│   ├── app.rb                     # Sinatra web application
+├── 07_cli_app/
+│   ├── htm_cli.rb                      # Interactive CLI
+│   └── README.md                       # Detailed CLI documentation
+├── 08_sinatra_app/
+│   ├── app.rb                          # Sinatra web application
 │   ├── Gemfile
 │   └── Gemfile.lock
-├── cli_app/
-│   ├── htm_cli.rb                 # Interactive CLI
-│   └── README.md                  # Detailed CLI documentation
-└── robot_groups/
-    ├── same_process.rb            # Single-process robot groups
-    ├── multi_process.rb           # Multi-process coordination
-    ├── robot_worker.rb            # Worker process for multi_process.rb
-    └── lib/
-        ├── robot_group.rb         # RobotGroup coordination class
-        └── working_memory_channel.rb  # PostgreSQL pub/sub
+├── 09_mcp_client.rb                    # MCP client with chat interface
+├── 10_telemetry/
+│   ├── demo.rb                         # Live Grafana metrics dashboard
+│   ├── README.md
+│   ├── SETUP_README.md
+│   └── grafana/dashboards/htm-metrics.json
+├── 11_robot_groups/
+│   ├── same_process.rb                 # Single-process robot groups
+│   ├── multi_process.rb                # Multi-process coordination
+│   ├── robot_worker.rb                 # Worker process for multi_process.rb
+│   └── lib/
+│       ├── robot_group.rb              # RobotGroup coordination class
+│       └── working_memory_channel.rb   # PostgreSQL pub/sub
+└── 12_rails_app/
+    ├── app/                            # Rails application files
+    ├── config/                         # Rails configuration
+    ├── Gemfile
+    └── README.md
 ```
 
 ---
@@ -480,15 +498,16 @@ examples/
 
 | Use Case | Example |
 |----------|---------|
-| Learning HTM basics | `basic_usage.rb` |
-| Configuration management | `config_file_example/` |
-| Custom LLM integration | `custom_llm_configuration.rb` |
-| Loading documents/files | `file_loader_usage.rb` |
-| Time-based queries | `timeframe_demo.rb` |
-| Production observability | `telemetry/` |
-| MCP server for AI assistants | `mcp_server.rb` |
-| MCP client with chat interface | `mcp_client.rb` |
-| Web application | `sinatra_app/` |
-| CLI tool | `cli_app/` |
-| Multi-robot coordination | `robot_groups/` |
-| High availability | `robot_groups/` |
+| Database setup | `00_create_examples_db.rb` |
+| Learning HTM basics | `01_basic_usage.rb` |
+| Configuration management | `02_config_file_example/` |
+| Custom LLM integration | `03_custom_llm_configuration.rb` |
+| Loading documents/files | `04_file_loader_usage.rb` |
+| Time-based queries | `05_timeframe_demo.rb` |
+| Full-featured demo | `06_example_app/` |
+| CLI tool | `07_cli_app/` |
+| Web application | `08_sinatra_app/` |
+| MCP client with chat | `09_mcp_client.rb` |
+| Production observability | `10_telemetry/` |
+| Multi-robot coordination | `11_robot_groups/` |
+| Rails integration | `12_rails_app/` |