htm 0.0.10 → 0.0.14

data/docs/telemetry.md

@@ -0,0 +1,391 @@
+# Telemetry (OpenTelemetry Metrics)
+
+HTM includes optional OpenTelemetry-based metrics for production observability. This document provides detailed configuration and usage information.
+
+## Overview
+
+HTM uses [OpenTelemetry](https://opentelemetry.io/) for metrics collection, the industry-standard observability framework. This provides:
+
+- **Universal compatibility**: Works with 50+ observability backends
+- **Zero vendor lock-in**: Emit standard OTLP, route anywhere
+- **Zero overhead when disabled**: Null object pattern ensures no performance impact
+
+## Quick Start
+
+### 1. Install Dependencies
+
+Add the OpenTelemetry gems to your Gemfile:
+
+```ruby
+gem 'opentelemetry-sdk'
+gem 'opentelemetry-metrics-sdk'
+gem 'opentelemetry-exporter-otlp'  # For OTLP export
+```
+
+### 2. Enable Telemetry
+
+```ruby
+HTM.configure do |config|
+  config.telemetry_enabled = true
+end
+```
+
+Or via environment variable:
+
+```bash
+export HTM_TELEMETRY_ENABLED="true"
+```
+
+### 3. Configure Destination
+
+Set standard OpenTelemetry environment variables:
+
+```bash
+export OTEL_METRICS_EXPORTER="otlp"
+export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"
+```
+
+## Configuration Options
+
+### HTM Configuration
+
+| Option | Type | Default | Description |
+|--------|------|---------|-------------|
+| `telemetry_enabled` | Boolean | `false` | Enable/disable telemetry |
+
+```ruby
+HTM.configure do |config|
+  config.telemetry_enabled = true
+end
+```
+
+### Environment Variables
+
+#### HTM-specific
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `HTM_TELEMETRY_ENABLED` | `false` | Enable telemetry (`true`/`false`) |
+
+#### OpenTelemetry Standard
+
+These are standard OpenTelemetry environment variables (not HTM-specific):
+
+| Variable | Example | Description |
+|----------|---------|-------------|
+| `OTEL_SERVICE_NAME` | `htm` | Service name in telemetry data |
+| `OTEL_METRICS_EXPORTER` | `otlp` | Metrics exporter type |
+| `OTEL_EXPORTER_OTLP_ENDPOINT` | `http://localhost:4318` | OTLP collector endpoint |
+| `OTEL_EXPORTER_OTLP_HEADERS` | `Authorization=Bearer token` | Headers for OTLP requests |
+
+## Available Metrics
+
+### htm.jobs (Counter)
+
+Counts job executions by type and outcome.
+
+**Attributes:**
+- `job`: Job type (`embedding`, `tags`)
+- `status`: Outcome (`success`, `error`, `circuit_open`)
+
+**Example queries (PromQL):**
+```promql
+# Jobs per minute by type
+rate(htm_jobs_total[1m])
+
+# Error rate for embedding jobs
+rate(htm_jobs_total{job="embedding",status="error"}[5m])
+  / rate(htm_jobs_total{job="embedding"}[5m])
+```
+
+### htm.embedding.latency (Histogram)
+
+Measures embedding generation time in milliseconds.
+
+**Attributes:**
+- `provider`: LLM provider (`ollama`, `openai`, etc.)
+- `status`: Outcome (`success`, `error`)
+
+**Example queries (PromQL):**
+```promql
+# p95 embedding latency
+histogram_quantile(0.95, rate(htm_embedding_latency_bucket[5m]))
+
+# Average latency by provider
+rate(htm_embedding_latency_sum[5m]) / rate(htm_embedding_latency_count[5m])
+```
+
+### htm.tag.latency (Histogram)
+
+Measures tag extraction time in milliseconds.
+
+**Attributes:**
+- `provider`: LLM provider (`ollama`, `openai`, etc.)
+- `status`: Outcome (`success`, `error`)
+
+**Example queries (PromQL):**
+```promql
+# p99 tag extraction latency
+histogram_quantile(0.99, rate(htm_tag_latency_bucket[5m]))
+```
+
+### htm.search.latency (Histogram)
+
+Measures search operation time in milliseconds.
+
+**Attributes:**
+- `strategy`: Search strategy (`vector`, `fulltext`, `hybrid`)
+
+**Example queries (PromQL):**
+```promql
+# Search latency by strategy
+histogram_quantile(0.95, rate(htm_search_latency_bucket[5m])) by (strategy)
+
+# Hybrid search avg latency
+rate(htm_search_latency_sum{strategy="hybrid"}[5m])
+  / rate(htm_search_latency_count{strategy="hybrid"}[5m])
+```
+
+### htm.cache.operations (Counter)
+
+Counts cache hits and misses.
+
+**Attributes:**
+- `operation`: Operation type (`hit`, `miss`)
+
+**Example queries (PromQL):**
+```promql
+# Cache hit rate
+rate(htm_cache_operations_total{operation="hit"}[5m])
+  / (rate(htm_cache_operations_total{operation="hit"}[5m])
+     + rate(htm_cache_operations_total{operation="miss"}[5m]))
+```
+
+## Compatible Backends
+
+HTM metrics work with any OTLP-compatible platform:
+
+### Open Source / Self-Hosted
+
+| Platform | OTLP Support | Notes |
+|----------|--------------|-------|
+| **Jaeger** | Native (v1.35+) | Distributed tracing with metrics |
+| **Prometheus** | Via receiver | Popular metrics platform |
+| **Grafana Tempo** | Native | Traces with metrics correlation |
+| **Grafana Mimir** | Native | Scalable Prometheus-compatible |
+| **SigNoz** | Native | Full-stack observability |
+| **Uptrace** | Native | Open source APM |
+
+### Commercial / SaaS
+
+| Platform | OTLP Support | Notes |
+|----------|--------------|-------|
+| **Datadog** | Native | Full-stack observability |
+| **New Relic** | Native | APM and infrastructure |
+| **Honeycomb** | Native (preferred) | High-cardinality observability |
+| **Splunk APM** | Native | Enterprise observability |
+| **Dynatrace** | Native | AI-powered monitoring |
+| **AWS X-Ray** | Via ADOT | AWS native tracing |
+| **Google Cloud Trace** | Native | GCP native tracing |
+| **Azure Monitor** | Native | Azure native monitoring |
+| **Grafana Cloud** | Native | Managed Grafana stack |
+
+## Setup Examples
+
+### Jaeger (Local Development)
+
+```bash
+# Start Jaeger with OTLP support
+docker run -d --name jaeger \
+  -p 4318:4318 \
+  -p 16686:16686 \
+  jaegertracing/all-in-one:latest
+
+# Configure HTM
+export HTM_TELEMETRY_ENABLED="true"
+export OTEL_METRICS_EXPORTER="otlp"
+export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"
+```
+
+### Prometheus (via OpenTelemetry Collector)
+
+```yaml
+# otel-collector-config.yaml
+receivers:
+  otlp:
+    protocols:
+      grpc:
+      http:
+
+exporters:
+  prometheus:
+    endpoint: "0.0.0.0:8889"
+
+service:
+  pipelines:
+    metrics:
+      receivers: [otlp]
+      exporters: [prometheus]
+```
+
+```bash
+docker run -d \
+  -p 4317:4317 \
+  -p 4318:4318 \
+  -p 8889:8889 \
+  -v $(pwd)/otel-collector-config.yaml:/etc/otel-collector-config.yaml \
+  otel/opentelemetry-collector:latest \
+  --config=/etc/otel-collector-config.yaml
+```
+
+### Datadog
+
+```bash
+export HTM_TELEMETRY_ENABLED="true"
+export OTEL_METRICS_EXPORTER="otlp"
+export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp.datadoghq.com"
+export OTEL_EXPORTER_OTLP_HEADERS="DD-API-KEY=your-api-key"
+```
+
+### Honeycomb
+
+```bash
+export HTM_TELEMETRY_ENABLED="true"
+export OTEL_METRICS_EXPORTER="otlp"
+export OTEL_EXPORTER_OTLP_ENDPOINT="https://api.honeycomb.io"
+export OTEL_EXPORTER_OTLP_HEADERS="x-honeycomb-team=your-api-key"
+```
+
+### New Relic
+
+```bash
+export HTM_TELEMETRY_ENABLED="true"
+export OTEL_METRICS_EXPORTER="otlp"
+export OTEL_EXPORTER_OTLP_ENDPOINT="https://otlp.nr-data.net"
+export OTEL_EXPORTER_OTLP_HEADERS="api-key=your-license-key"
+```
+
+## Architecture
+
+### Null Object Pattern
+
+HTM uses a null object pattern for telemetry. When disabled:
+
+```ruby
+# This code runs identically whether telemetry is enabled or not
+HTM::Telemetry.job_counter.add(1, attributes: { 'job' => 'embedding' })
+```
+
+When telemetry is disabled or the SDK is not installed:
+- `HTM::Telemetry.meter` returns a `NullMeter`
+- All instruments return `NullInstrument` instances
+- All operations (`add`, `record`) are no-ops returning `nil`
+- Zero memory allocation, zero CPU overhead
+
+### Code Flow
+
+![HTM Telemetry Architecture](images/telemetry-architecture.svg)
+
+### Instrumentation Points
+
+HTM instruments these key operations:
+
+| Component | Metrics Recorded |
+|-----------|-----------------|
+| `GenerateEmbeddingJob` | `htm.jobs`, `htm.embedding.latency` |
+| `GenerateTagsJob` | `htm.jobs`, `htm.tag.latency` |
+| `VectorSearch#search` | `htm.search.latency` (strategy: vector) |
+| `FulltextSearch#search_fulltext` | `htm.search.latency` (strategy: fulltext) |
+| `HybridSearch#search_hybrid` | `htm.search.latency` (strategy: hybrid) |
+| `QueryCache#fetch` | `htm.cache.operations` |
+
+## Testing
+
+When writing tests, reset telemetry state between tests:
+
+```ruby
+def setup
+  HTM::Telemetry.reset!
+  HTM.configuration.telemetry_enabled = false
+end
+
+def teardown
+  HTM::Telemetry.reset!
+end
+```
+
+### Verifying Metrics in Tests
+
+```ruby
+def test_records_embedding_latency
+  # Create a mock histogram
+  recorded = []
+  mock_histogram = Object.new
+  mock_histogram.define_singleton_method(:record) do |value, **kwargs|
+    recorded << { value: value, attributes: kwargs[:attributes] }
+  end
+
+  # Inject mock (requires test helper)
+  HTM::Telemetry.instance_variable_set(:@embedding_latency, mock_histogram)
+
+  # Run operation
+  HTM::Jobs::GenerateEmbeddingJob.perform(node_id: node.id)
+
+  # Verify
+  assert recorded.any? { |r| r[:attributes]['status'] == 'success' }
+end
+```
+
+## Troubleshooting
+
+### Metrics Not Appearing
+
+1. **Check telemetry is enabled:**
+   ```ruby
+   puts HTM.configuration.telemetry_enabled  # Should be true
+   puts HTM::Telemetry.enabled?              # Should be true
+   ```
+
+2. **Check SDK is installed:**
+   ```ruby
+   puts HTM::Telemetry.sdk_available?  # Should be true
+   ```
+
+3. **Check environment variables:**
+   ```bash
+   echo $OTEL_METRICS_EXPORTER
+   echo $OTEL_EXPORTER_OTLP_ENDPOINT
+   ```
+
+4. **Check collector is running:**
+   ```bash
+   curl http://localhost:4318/v1/metrics  # Should not 404
+   ```
+
+### High Cardinality Issues
+
+Avoid high-cardinality attributes in metrics. HTM uses low-cardinality attributes by design:
+
+- `job`: 2-3 values (embedding, tags, propositions)
+- `status`: 3 values (success, error, circuit_open)
+- `provider`: ~10 values (ollama, openai, etc.)
+- `strategy`: 3 values (vector, fulltext, hybrid)
+- `operation`: 2 values (hit, miss)
+
+### Performance Concerns
+
+With telemetry disabled (default), there is zero overhead:
+- No gem loading
+- No object allocation
+- No method calls beyond the null check
+
+With telemetry enabled, overhead is minimal:
+- ~1-5μs per metric recording
+- Memory for histogram buckets (~1KB per histogram)
+
+## References
+
+- [OpenTelemetry Ruby SDK](https://github.com/open-telemetry/opentelemetry-ruby)
+- [OpenTelemetry Metrics SDK](https://github.com/open-telemetry/opentelemetry-ruby/tree/main/metrics_sdk)
+- [OpenTelemetry Specification](https://opentelemetry.io/docs/specs/otel/)
+- [OTLP Protocol](https://opentelemetry.io/docs/specs/otlp/)

data/examples/README.md

@@ -1,6 +1,6 @@
 # HTM Examples
 
-This directory contains example applications demonstrating various ways to use the HTM (Hierarchical Temporary Memory) gem.
+This directory contains example applications demonstrating various ways to use the HTM (Hierarchical Temporal Memory) gem.
 
 ## Prerequisites
 
@@ -11,6 +11,12 @@ All examples require:
    export HTM_DBURL="postgresql://user@localhost:5432/htm_development"
    ```
 
+   > **Note**: Database selection now respects `RAILS_ENV`. If `RAILS_ENV` is set,
+   > HTM extracts the base name from `HTM_DBURL` and appends the environment suffix.
+   > For example, with `HTM_DBURL=...htm_development` and `RAILS_ENV=test`, HTM
+   > connects to `htm_test`. When `RAILS_ENV` is unset (typical for examples),
+   > behavior is unchanged.
+
 2. **Ollama** (recommended for local LLM):
    ```bash
    ollama pull nomic-embed-text  # For embeddings
@@ -107,6 +113,160 @@ ruby examples/timeframe_demo.rb
 
 ---
 
+### telemetry/
+
+**Live Grafana dashboard for HTM metrics.**
+
+A complete telemetry demo using Homebrew-installed Prometheus and Grafana to visualize HTM metrics in real-time graphs.
+
+```bash
+cd examples/telemetry
+ruby demo.rb
+```
+
+The demo automatically:
+- Checks/installs Prometheus and Grafana via Homebrew
+- Starts both services if not running
+- Configures Prometheus to scrape the demo's metrics
+- Cleans up previous demo data
+- Opens Grafana in your browser
+
+**Features:**
+- Live updating dashboard with job counts, latencies, cache hit rates
+- Pre-built Grafana dashboard JSON
+- No Docker required - uses native macOS services
+
+**Dependencies:**
+```bash
+brew install prometheus grafana
+gem install prometheus-client webrick
+```
+
+See [telemetry/README.md](telemetry/README.md) for detailed setup instructions.
+
+---
+
+### mcp_server.rb & mcp_client.rb
+
+**Model Context Protocol (MCP) integration for AI assistants.**
+
+A pair of examples demonstrating how to expose HTM as an MCP server and connect to it from a chat client. This enables AI assistants like Claude Desktop to use HTM's memory capabilities.
+
+#### mcp_server.rb
+
+An MCP server that exposes HTM's memory operations as tools:
+
+```bash
+ruby examples/mcp_server.rb
+```
+
+**Tools exposed:**
+- `SetRobotTool` - Set the robot identity for this session (call first)
+- `GetRobotTool` - Get current robot information
+- `GetWorkingMemoryTool` - Get working memory contents for session restore
+- `RememberTool` - Store information with optional tags and metadata
+- `RecallTool` - Search memories using vector, fulltext, or hybrid strategies
+- `ForgetTool` - Soft-delete a memory (recoverable)
+- `RestoreTool` - Restore a soft-deleted memory
+- `ListTagsTool` - List tags with optional prefix filtering
+- `StatsTool` - Get memory usage statistics
+
+**Resources exposed:**
+- `htm://statistics` - Memory statistics as JSON
+- `htm://tags/hierarchy` - Tag hierarchy as text tree
+- `htm://memories/recent` - Last 20 memories
+
+**Claude Desktop configuration** (`~/.config/claude/claude_desktop_config.json`):
+```json
+{
+  "mcpServers": {
+    "htm-memory": {
+      "command": "ruby",
+      "args": ["/path/to/htm/examples/mcp_server.rb"],
+      "env": {
+        "HTM_DBURL": "postgresql://user@localhost:5432/htm_development"
+      }
+    }
+  }
+}
+```
+
+#### mcp_client.rb
+
+An interactive chat client that connects to the MCP server via STDIO and uses a local Ollama model (gpt-oss) for conversation:
+
+```bash
+ruby examples/mcp_client.rb
+```
+
+**Features:**
+- Prompts for robot name on startup (or uses `HTM_ROBOT_NAME` env var)
+- Calls `SetRobotTool` to establish robot identity with the server
+- Offers to restore previous session from working memory
+- Connects to `mcp_server.rb` automatically via STDIO transport
+- Interactive chat loop with tool calling
+- LLM decides when to remember/recall information
+- Logs tool calls and results for visibility
+
+**Commands:**
+- `/tools` - List available MCP tools
+- `/resources` - List available MCP resources
+- `/clear` - Clear chat history
+- `/help` - Show help
+- `/exit` - Quit
+
+**Example startup and conversation:**
+```
+$ ruby examples/mcp_client.rb
+Connecting to HTM MCP server...
+[✓] Connected to HTM MCP server
+[✓] Found 9 tools:
+    - SetRobotTool: Set the robot identity for this session...
+    - GetRobotTool: Get information about the current robot...
+    - GetWorkingMemoryTool: Get all working memory contents...
+    ...
+
+Enter your robot name (or press Enter for default): alice-assistant
+[✓] Robot name: alice-assistant
+Setting robot identity on MCP server...
+[✓] Robot identity set: alice-assistant (id=5, nodes=12)
+
+Found 3 memories in working memory from previous session.
+Restore previous session? (y/N): y
+[✓] Will restore 3 memories after chat setup
+
+Initializing chat with gpt-oss:latest...
+[✓] Chat initialized with tools attached
+Restoring 3 memories to chat context...
+[✓] Restored 3 memories to chat context
+
+======================================================================
+HTM MCP Client - AI Chat with Memory Tools
+======================================================================
+
+Robot: alice-assistant
+Model: gpt-oss:latest (via Ollama)
+...
+
+you> What's the API rate limit?
+
+assistant> The API rate limit is 1000 requests per minute.
+```
+
+**Additional dependencies:**
+```bash
+gem install fast-mcp ruby_llm-mcp
+ollama pull gpt-oss  # Or your preferred model
+```
+
+**Environment Variables:**
+- `HTM_DBURL` - PostgreSQL connection (required)
+- `OLLAMA_URL` - Ollama server URL (default: http://localhost:11434)
+- `OLLAMA_MODEL` - Model to use (default: gpt-oss:latest)
+- `HTM_ROBOT_NAME` - Robot name (optional, prompts if not set)
+
+---
+
 ## Application Examples
 
 ### example_app/
@@ -245,6 +405,13 @@ examples/
 ├── custom_llm_configuration.rb    # LLM integration patterns
 ├── file_loader_usage.rb           # Document loading
 ├── timeframe_demo.rb              # Time-based filtering
+├── telemetry/
+│   ├── demo.rb                    # Live Grafana metrics dashboard
+│   ├── README.md
+│   ├── SETUP_README.md
+│   └── grafana/dashboards/htm-metrics.json
+├── mcp_server.rb                  # MCP server exposing HTM tools
+├── mcp_client.rb                  # MCP client with chat interface
 ├── example_app/
 │   ├── app.rb                     # Full-featured demo app
 │   └── Rakefile
@@ -274,6 +441,9 @@ examples/
 | Custom LLM integration | `custom_llm_configuration.rb` |
 | Loading documents/files | `file_loader_usage.rb` |
 | Time-based queries | `timeframe_demo.rb` |
+| Production observability | `telemetry/` |
+| MCP server for AI assistants | `mcp_server.rb` |
+| MCP client with chat interface | `mcp_client.rb` |
 | Web application | `sinatra_app/` |
 | CLI tool | `cli_app/` |
 | Multi-robot coordination | `robot_groups/` |

data/examples/cli_app/README.md

@@ -111,7 +111,7 @@ This means:
 $ ruby htm_cli.rb
 
 ============================================================
-HTM CLI - Hierarchical Temporary Memory Assistant
+HTM CLI - Hierarchical Temporal Memory Assistant
 ============================================================
 
 Job Backend: inline (synchronous execution)

data/examples/cli_app/htm_cli.rb

@@ -67,7 +67,7 @@ class HTMCli
   def run
     puts
     puts "=" * 60
-    puts "HTM CLI - Hierarchical Temporary Memory Assistant"
+    puts "HTM CLI - Hierarchical Temporal Memory Assistant"
     puts "=" * 60
     puts
     puts "Job Backend: #{HTM.configuration.job_backend} (synchronous execution)"