htm 0.0.18 → 0.0.20

data/docs/examples/standalone-app.md

@@ -0,0 +1,216 @@
+# Standalone Application Example
+
+A simple Ruby application demonstrating HTM's core features with full RubyLLM integration.
+
+**Source:** [`examples/example_app/`](https://github.com/madbomber/htm/tree/main/examples/example_app)
+
+## Overview
+
+The standalone example demonstrates:
+
+- HTM initialization and configuration
+- RubyLLM integration for embeddings and tags
+- Remembering information with automatic processing
+- Comparing search strategies (fulltext, vector, hybrid)
+- Background job processing for embeddings and tags
+
+## Prerequisites
+
+```bash
+# PostgreSQL with pgvector
+export HTM_DATABASE__URL="postgresql://user@localhost:5432/htm_development"
+
+# Ollama for embeddings and tags
+ollama pull nomic-embed-text
+ollama pull gemma3:latest
+```
+
+## Running
+
+```bash
+cd examples/example_app
+ruby app.rb
+```
+
+## Code Walkthrough
+
+### Configuration
+
+```ruby
+require_relative '../../lib/htm'
+require 'ruby_llm'
+
+HTM.configure do |c|
+  # Logger
+  c.logger = Logger.new($stdout)
+  c.logger.level = Logger::INFO
+
+  # Embedding generation (using Ollama)
+  c.embedding.provider = :ollama
+  c.embedding.model = 'nomic-embed-text'
+  c.embedding.dimensions = 768
+  c.providers.ollama.url = ENV['OLLAMA_URL'] || 'http://localhost:11434'
+
+  # Tag extraction (using Ollama)
+  c.tag.provider = :ollama
+  c.tag.model = 'gemma3'
+
+  # Apply default implementations
+  c.reset_to_defaults
+end
+```
+
+### Creating an HTM Instance
+
+```ruby
+htm = HTM.new(robot_name: "Example App Robot")
+```
+
+### Remembering Information
+
+```ruby
+# Store conversation messages
+node_1 = htm.remember(
+  "HTM provides intelligent memory management for LLM-based applications"
+)
+
+node_2 = htm.remember(
+  "The two-tier architecture includes working memory and long-term storage"
+)
+
+node_3 = htm.remember(
+  "Can you explain how the working memory eviction algorithm works?"
+)
+
+puts "Remembered 3 messages (nodes #{node_1}, #{node_2}, #{node_3})"
+puts "Embeddings and tags are being generated asynchronously..."
+```
+
+### Waiting for Background Jobs
+
+```ruby
+# Tag generation with LLM can take 10-15 seconds
+puts "Waiting for background jobs to complete..."
+sleep 15
+
+# Check generated tags
+[node_1, node_2, node_3].each do |node_id|
+  node = HTM::Models::Node.includes(:tags).find(node_id)
+  puts "Node #{node_id}: #{node.tags.map(&:name).join(', ')}"
+end
+```
+
+### Comparing Search Strategies
+
+```ruby
+# 1. Full-text search (doesn't require embeddings)
+fulltext_memories = htm.recall(
+  "memory",
+  timeframe: (Time.now - 3600)..Time.now,
+  strategy: :fulltext,
+  limit: 3
+)
+
+# 2. Vector search (requires embeddings)
+vector_memories = htm.recall(
+  "intelligent memory system",
+  timeframe: (Time.now - 3600)..Time.now,
+  strategy: :vector,
+  limit: 3
+)
+
+# 3. Hybrid search (combines both)
+hybrid_memories = htm.recall(
+  "working memory architecture",
+  timeframe: (Time.now - 3600)..Time.now,
+  strategy: :hybrid,
+  limit: 3
+)
+```
+
+## Example Output
+
+```
+=== HTM Full-Featured Example Application ===
+
+Checking database connection...
+✓ Database configured: htm_development @ localhost
+
+Configuring HTM with RubyLLM...
+✓ Configured with Ollama:
+  - Embeddings: nomic-embed-text
+  - Tags: gemma3
+  - Ollama URL: http://localhost:11434
+
+Checking Ollama connection...
+✓ Ollama is running
+
+Initializing HTM...
+
+Remembering example conversation...
+✓ Remembered 3 conversation messages (nodes 1, 2, 3)
+
+Waiting for background jobs to complete (15 seconds)...
+
+--- Generated Tags ---
+Node 1:
+  - ai:memory-management
+  - software:architecture
+Node 2:
+  - architecture:two-tier
+  - memory:working
+  - memory:long-term
+Node 3:
+  - algorithm:eviction
+  - memory:working
+
+--- Embedding Status ---
+Node 1: ✓ Generated (768 dimensions)
+Node 2: ✓ Generated (768 dimensions)
+Node 3: ✓ Generated (768 dimensions)
+
+--- Recall Strategies Comparison ---
+
+1. Full-text Search for 'memory':
+Found 3 memories:
+  - HTM provides intelligent memory management for LLM-based...
+  - The two-tier architecture includes working memory and...
+  - Can you explain how the working memory eviction algorit...
+
+2. Vector Search for 'intelligent memory system':
+Found 3 memories:
+  - HTM provides intelligent memory management for LLM-based...
+  - The two-tier architecture includes working memory and...
+  - Can you explain how the working memory eviction algorit...
+
+3. Hybrid Search for 'working memory architecture':
+Found 3 memories:
+  - The two-tier architecture includes working memory and...
+  - Can you explain how the working memory eviction algorit...
+  - HTM provides intelligent memory management for LLM-based...
+```
+
+## HTM Core API Summary
+
+```ruby
+# 1. Remember - Store information
+htm.remember(content, tags: [], metadata: {})
+# - Stores in long-term memory
+# - Adds to working memory for immediate use
+# - Generates embeddings and tags in background
+
+# 2. Recall - Retrieve relevant memories
+htm.recall(topic, timeframe:, strategy:, limit:)
+# - Strategies: :fulltext, :vector, :hybrid
+# - Results added to working memory
+
+# 3. Forget - Delete a memory
+htm.forget(node_id)                           # Soft delete (default)
+htm.forget(node_id, soft: false, confirm: :confirmed)  # Permanent
+```
+
+## See Also
+
+- [Basic Usage Example](basic-usage.md)
+- [LLM Configuration Example](llm-configuration.md)
+- [Search Strategies Guide](../guides/search-strategies.md)

data/docs/examples/telemetry.md

@@ -0,0 +1,224 @@
+# Telemetry Demo
+
+This example demonstrates HTM's OpenTelemetry-based metrics with live Prometheus and Grafana visualization.
+
+**Source:** [`examples/telemetry/demo.rb`](https://github.com/madbomber/htm/blob/main/examples/telemetry/demo.rb)
+
+## Overview
+
+The telemetry demo shows:
+
+- Setting up Prometheus metrics collection
+- Visualizing metrics in Grafana dashboards
+- Available HTM metrics (jobs, latency, cache)
+- Real-time monitoring during HTM operations
+
+## Prerequisites
+
+### macOS (Homebrew)
+
+```bash
+# Install Prometheus and Grafana
+brew install prometheus grafana
+
+# Install Ruby gems (system gems, not bundled)
+gem install prometheus-client webrick
+
+# Set database connection
+export HTM_DATABASE__URL="postgresql://user@localhost:5432/htm_development"
+```
+
+## Running the Demo
+
+```bash
+cd examples/telemetry
+ruby demo.rb
+```
+
+The demo will:
+
+1. Check and start Prometheus/Grafana services
+2. Configure Prometheus to scrape HTM metrics
+3. Start a metrics server on port 9394
+4. Run HTM operations in a loop
+5. Open Grafana in your browser
+
+## Available Metrics
+
+| Metric | Type | Labels | Description |
+|--------|------|--------|-------------|
+| `htm_jobs_total` | Counter | job, status | Job execution counts |
+| `htm_embedding_latency_milliseconds` | Histogram | provider, status | Embedding generation time |
+| `htm_tag_latency_milliseconds` | Histogram | provider, status | Tag extraction time |
+| `htm_search_latency_milliseconds` | Histogram | strategy | Search operation time |
+| `htm_cache_operations_total` | Counter | operation | Cache hit/miss counts |
+
+## Enabling Telemetry in Your App
+
+### Configuration
+
+```ruby
+HTM.configure do |config|
+  config.telemetry_enabled = true
+end
+
+# Or via environment variable
+# HTM_TELEMETRY_ENABLED=true
+```
+
+### Required Gems
+
+```ruby
+# Add to your Gemfile
+gem 'opentelemetry-sdk'
+gem 'opentelemetry-metrics-sdk'
+gem 'opentelemetry-exporter-otlp'  # For OTLP export
+```
+
+### OpenTelemetry Configuration
+
+```bash
+# Export to OTLP-compatible backend
+export OTEL_METRICS_EXPORTER="otlp"
+export OTEL_EXPORTER_OTLP_ENDPOINT="http://localhost:4318"
+```
+
+## Grafana Dashboard
+
+The demo includes a pre-configured Grafana dashboard:
+
+**Location:** `examples/telemetry/grafana/dashboards/htm-metrics.json`
+
+### Import Dashboard
+
+1. Open Grafana at http://localhost:3000
+2. Default login: admin / admin
+3. Go to Dashboards > Import
+4. Upload the JSON file
+
+### Dashboard Panels
+
+- **Job Success Rate**: Percentage of successful jobs
+- **Embedding Latency**: P50, P95, P99 latencies
+- **Tag Latency**: Tag extraction performance
+- **Search Performance**: Query latency by strategy
+- **Cache Hit Rate**: Cache effectiveness
+
+## Architecture
+
+```
+┌─────────────────────────────────────────────────────────┐
+│                    HTM Application                      │
+│                                                         │
+│  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐     │
+│  │   remember  │  │    recall   │  │    jobs     │     │
+│  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘     │
+│         │                │                │             │
+│         └────────────────┼────────────────┘             │
+│                          │                              │
+│              ┌───────────▼───────────┐                  │
+│              │  Telemetry Collector  │                  │
+│              └───────────┬───────────┘                  │
+└──────────────────────────┼──────────────────────────────┘
+                           │
+              ┌────────────▼────────────┐
+              │   /metrics endpoint     │
+              │     (port 9394)         │
+              └────────────┬────────────┘
+                           │
+              ┌────────────▼────────────┐
+              │      Prometheus         │
+              │   (scrapes /metrics)    │
+              └────────────┬────────────┘
+                           │
+              ┌────────────▼────────────┐
+              │        Grafana          │
+              │    (visualization)      │
+              └─────────────────────────┘
+```
+
+## Compatible Backends
+
+HTM telemetry is OTLP-compatible and works with:
+
+### Open Source
+
+- Jaeger
+- Prometheus + Grafana
+- Grafana Tempo/Mimir
+- SigNoz
+- Uptrace
+
+### Commercial
+
+- Datadog
+- New Relic
+- Honeycomb
+- Splunk
+- Dynatrace
+- AWS X-Ray
+- Google Cloud Trace
+- Azure Monitor
+
+## Demo Output
+
+```
+╔══════════════════════════════════════════════════════════════════╗
+║         HTM Telemetry Demo - Live Grafana Visualization          ║
+╚══════════════════════════════════════════════════════════════════╝
+
+Checking Ruby dependencies...
+  [OK] prometheus-client gem
+  [OK] webrick gem
+
+Loading HTM...
+  [OK] HTM 0.1.0
+
+Starting services...
+  prometheus: already running
+  grafana: already running
+
+Starting metrics server on port 9394...
+  [OK] Metrics available at http://localhost:9394/metrics
+
+Opening Grafana...
+  Default login: admin / admin
+
+============================================================
+Starting demo loop...
+  Metrics: http://localhost:9394/metrics
+  Grafana: http://localhost:3000
+============================================================
+
+Press Ctrl+C to stop
+
+[10:30:15] Iteration 1
+  > Remember: PostgreSQL supports vector similarity search... node 42
+  > Recall (fulltext): 'database    ' -> 3 results (12ms)
+  > Recall (vector  ): 'database    ' -> 3 results (45ms)
+  > Recall (hybrid  ): 'database    ' -> 3 results (52ms)
+  Metrics exported to Prometheus
+```
+
+## Cleanup
+
+The demo prompts to stop services on exit:
+
+```
+Stop Prometheus and Grafana services? (y/N): y
+  Stopping prometheus... done
+  Stopping grafana... done
+  Services stopped.
+```
+
+Or stop manually:
+
+```bash
+brew services stop prometheus grafana
+```
+
+## See Also
+
+- [Telemetry Guide](../guides/telemetry.md)
+- [Observability API](../api/yard/HTM/Observability.md)
+- [OpenTelemetry Ruby](https://opentelemetry.io/docs/languages/ruby/)

data/docs/examples/timeframes.md

@@ -0,0 +1,143 @@
+# Timeframe Demo
+
+This example demonstrates the flexible timeframe options for filtering memories by time in HTM recall queries.
+
+**Source:** [`examples/timeframe_demo.rb`](https://github.com/madbomber/htm/blob/main/examples/timeframe_demo.rb)
+
+## Overview
+
+HTM supports natural language time expressions for filtering memories. This example shows:
+
+- Date and Time object filtering
+- Range-based precise filtering
+- Natural language expressions ("yesterday", "last week")
+- Automatic timeframe extraction from queries (`:auto`)
+- Multiple time windows
+
+## Running the Example
+
+```bash
+export HTM_DATABASE__URL="postgresql://user@localhost:5432/htm_development"
+ruby examples/timeframe_demo.rb
+```
+
+## Timeframe Options
+
+### No Filter (nil)
+
+```ruby
+# Search all memories regardless of time
+htm.recall("PostgreSQL", timeframe: nil)
+```
+
+### Date Object
+
+A Date is expanded to cover the entire day (00:00:00 to 23:59:59):
+
+```ruby
+htm.recall("meetings", timeframe: Date.today)
+htm.recall("notes", timeframe: Date.new(2025, 11, 15))
+```
+
+### Range (Precise Control)
+
+```ruby
+start_time = Time.now - (7 * 24 * 60 * 60)  # 7 days ago
+end_time = Time.now
+
+htm.recall("updates", timeframe: start_time..end_time)
+```
+
+### Natural Language
+
+HTM uses the Chronic gem for parsing natural language time expressions:
+
+```ruby
+htm.recall("notes", timeframe: "yesterday")
+htm.recall("discussions", timeframe: "last week")
+htm.recall("decisions", timeframe: "last month")
+htm.recall("tasks", timeframe: "this morning")
+```
+
+#### Supported Expressions
+
+| Expression | Meaning |
+|------------|---------|
+| `"yesterday"` | Previous day |
+| `"last week"` | Previous 7 days |
+| `"last month"` | Previous 30 days |
+| `"today"` | Current day |
+| `"this morning"` | Today before noon |
+| `"few days ago"` | 3 days ago |
+| `"last weekend"` | Previous Saturday-Monday |
+| `"2 weekends ago"` | Two weekends back |
+
+### Automatic Extraction (`:auto`)
+
+Extract timeframe from the query text automatically:
+
+```ruby
+# The timeframe is extracted and removed from the search query
+htm.recall("what did we discuss last week about databases", timeframe: :auto)
+# Searches for: "what did we discuss about databases"
+# Timeframe: last week's date range
+
+htm.recall("show me notes from yesterday about PostgreSQL", timeframe: :auto)
+# Searches for: "show me notes about PostgreSQL"
+# Timeframe: yesterday's date range
+```
+
+### Multiple Time Windows
+
+Search across multiple date ranges (OR'd together):
+
+```ruby
+today = Date.today
+last_friday = today - ((today.wday + 2) % 7)
+two_fridays_ago = last_friday - 7
+
+htm.recall("standup notes", timeframe: [last_friday, two_fridays_ago])
+```
+
+SQL equivalent:
+```sql
+WHERE (created_at BETWEEN '...' AND '...')
+   OR (created_at BETWEEN '...' AND '...')
+```
+
+## Configuration
+
+```ruby
+HTM.configure do |config|
+  # Configure week start for "last weekend" calculations
+  config.week_start = :sunday  # or :monday
+end
+```
+
+## Summary Table
+
+| Input Type | Behavior |
+|------------|----------|
+| `nil` | No time filter |
+| `Date` | Entire day (00:00:00 to 23:59:59) |
+| `DateTime` | Entire day (same as Date) |
+| `Time` | Entire day (same as Date) |
+| `Range` | Exact time window |
+| `String` | Natural language parsing via Chronic |
+| `:auto` | Extract from query, return cleaned query |
+| `Array<Range>` | Multiple time windows OR'd together |
+
+## Special Keywords
+
+| Keyword | Meaning |
+|---------|---------|
+| `few`, `a few`, `several` | Maps to 3 |
+| `recently`, `recent` | Last 3 days |
+| `weekend before last` | 2 weekends ago (Sat-Mon) |
+| `N weekends ago` | N weekends back (Sat-Mon range) |
+
+## See Also
+
+- [Recalling Memories Guide](../guides/recalling-memories.md)
+- [Search Strategies Guide](../guides/search-strategies.md)
+- [Timeframe API Reference](../api/yard/HTM/Timeframe.md)