RubyGems - htm - Versions diffs - 0.0.18 → 0.0.30 - Mend

htm 0.0.18 → 0.0.30

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (216) hide show

checksums.yaml +4 -4
data/CHANGELOG.md +119 -1
data/README.md +12 -0
data/Rakefile +104 -18
data/db/migrate/00001_enable_extensions.rb +9 -5
data/db/migrate/00002_create_robots.rb +18 -6
data/db/migrate/00003_create_file_sources.rb +30 -17
data/db/migrate/00004_create_nodes.rb +60 -48
data/db/migrate/00005_create_tags.rb +24 -12
data/db/migrate/00006_create_node_tags.rb +28 -13
data/db/migrate/00007_create_robot_nodes.rb +40 -26
data/db/schema.sql +17 -1
data/db/seeds.rb +34 -34
data/docs/api/embedding-service.md +140 -110
data/docs/api/yard/HTM/ActiveRecordConfig.md +6 -0
data/docs/api/yard/HTM/Config.md +173 -0
data/docs/api/yard/HTM/ConfigSection.md +28 -0
data/docs/api/yard/HTM/Database.md +1 -1
data/docs/api/yard/HTM/Railtie.md +2 -2
data/docs/api/yard/HTM.md +0 -57
data/docs/api/yard/index.csv +76 -61
data/docs/api/yard-reference.md +2 -1
data/docs/architecture/adrs/003-ollama-embeddings.md +45 -36
data/docs/architecture/adrs/004-hive-mind.md +1 -1
data/docs/architecture/adrs/008-robot-identification.md +1 -1
data/docs/architecture/index.md +11 -9
data/docs/architecture/overview.md +11 -7
data/docs/assets/images/balanced-strategy-decay.svg +41 -0
data/docs/assets/images/class-hierarchy.svg +1 -1
data/docs/assets/images/eviction-priority.svg +43 -0
data/docs/assets/images/exception-hierarchy.svg +2 -2
data/docs/assets/images/hive-mind-shared-memory.svg +52 -0
data/docs/assets/images/htm-architecture-overview.svg +3 -3
data/docs/assets/images/htm-core-components.svg +4 -4
data/docs/assets/images/htm-layered-architecture.svg +1 -1
data/docs/assets/images/htm-memory-addition-flow.svg +2 -2
data/docs/assets/images/htm-memory-recall-flow.svg +2 -2
data/docs/assets/images/memory-topology.svg +53 -0
data/docs/assets/images/two-tier-memory-architecture.svg +55 -0
data/docs/database/naming-convention.md +244 -0
data/docs/database_rake_tasks.md +31 -0
data/docs/development/rake-tasks.md +80 -35
data/docs/development/setup.md +76 -44
data/docs/examples/basic-usage.md +133 -0
data/docs/examples/config-files.md +170 -0
data/docs/examples/file-loading.md +208 -0
data/docs/examples/index.md +116 -0
data/docs/examples/llm-configuration.md +168 -0
data/docs/examples/mcp-client.md +172 -0
data/docs/examples/rails-integration.md +173 -0
data/docs/examples/robot-groups.md +210 -0
data/docs/examples/sinatra-integration.md +218 -0
data/docs/examples/standalone-app.md +216 -0
data/docs/examples/telemetry.md +224 -0
data/docs/examples/timeframes.md +143 -0
data/docs/getting-started/installation.md +97 -40
data/docs/getting-started/quick-start.md +28 -11
data/docs/guides/configuration.md +515 -0
data/docs/guides/file-loading.md +322 -0
data/docs/guides/getting-started.md +40 -9
data/docs/guides/index.md +3 -3
data/docs/guides/mcp-server.md +100 -13
data/docs/guides/propositions.md +264 -0
data/docs/guides/recalling-memories.md +4 -4
data/docs/guides/search-strategies.md +3 -3
data/docs/guides/tags.md +318 -0
data/docs/guides/telemetry.md +229 -0
data/docs/index.md +8 -16
data/docs/{architecture → robots}/hive-mind.md +8 -111
data/docs/robots/index.md +73 -0
data/docs/{guides → robots}/multi-robot.md +3 -3
data/docs/{guides → robots}/robot-groups.md +8 -7
data/docs/{architecture → robots}/two-tier-memory.md +13 -149
data/docs/robots/why-robots.md +85 -0
data/examples/.envrc +6 -0
data/examples/.gitignore +2 -0
data/examples/00_create_examples_db.rb +94 -0
data/examples/{basic_usage.rb → 01_basic_usage.rb} +12 -16
data/examples/{custom_llm_configuration.rb → 03_custom_llm_configuration.rb} +13 -3
data/examples/{file_loader_usage.rb → 04_file_loader_usage.rb} +11 -14
data/examples/{timeframe_demo.rb → 05_timeframe_demo.rb} +10 -3
data/examples/{example_app → 06_example_app}/app.rb +15 -15
data/examples/{cli_app → 07_cli_app}/htm_cli.rb +15 -22
data/examples/08_sinatra_app/Gemfile.lock +241 -0
data/examples/{sinatra_app → 08_sinatra_app}/app.rb +19 -18
data/examples/{mcp_client.rb → 09_mcp_client.rb} +5 -8
data/examples/{telemetry → 10_telemetry}/SETUP_README.md +1 -1
data/examples/{telemetry → 10_telemetry}/demo.rb +14 -10
data/examples/11_robot_groups/README.md +335 -0
data/examples/{robot_groups → 11_robot_groups/lib}/robot_worker.rb +17 -3
data/examples/{robot_groups → 11_robot_groups}/multi_process.rb +9 -9
data/examples/{robot_groups → 11_robot_groups}/same_process.rb +9 -12
data/examples/{rails_app → 12_rails_app}/Gemfile +3 -0
data/examples/{rails_app → 12_rails_app}/Gemfile.lock +87 -58
data/examples/{rails_app → 12_rails_app}/app/controllers/dashboard_controller.rb +10 -6
data/examples/{rails_app → 12_rails_app}/app/controllers/files_controller.rb +5 -5
data/examples/{rails_app → 12_rails_app}/app/controllers/memories_controller.rb +11 -7
data/examples/{rails_app → 12_rails_app}/app/controllers/robots_controller.rb +8 -8
data/examples/12_rails_app/app/controllers/tags_controller.rb +36 -0
data/examples/{rails_app → 12_rails_app}/app/views/dashboard/index.html.erb +2 -2
data/examples/{rails_app → 12_rails_app}/app/views/files/new.html.erb +5 -2
data/examples/{rails_app → 12_rails_app}/app/views/memories/_memory_card.html.erb +3 -3
data/examples/{rails_app → 12_rails_app}/app/views/memories/deleted.html.erb +3 -3
data/examples/{rails_app → 12_rails_app}/app/views/memories/edit.html.erb +3 -3
data/examples/{rails_app → 12_rails_app}/app/views/memories/show.html.erb +4 -4
data/examples/{rails_app → 12_rails_app}/app/views/robots/index.html.erb +2 -2
data/examples/{rails_app → 12_rails_app}/app/views/robots/show.html.erb +4 -4
data/examples/{rails_app → 12_rails_app}/app/views/search/index.html.erb +1 -1
data/examples/{rails_app → 12_rails_app}/app/views/tags/index.html.erb +2 -2
data/examples/{rails_app → 12_rails_app}/app/views/tags/show.html.erb +1 -1
data/examples/12_rails_app/config/initializers/htm.rb +7 -0
data/examples/12_rails_app/config/initializers/rack.rb +5 -0
data/examples/README.md +230 -211
data/examples/examples_helper.rb +138 -0
data/lib/htm/config/builder.rb +167 -0
data/lib/htm/config/database.rb +317 -0
data/lib/htm/config/defaults.yml +41 -13
data/lib/htm/config/section.rb +74 -0
data/lib/htm/config/validator.rb +83 -0
data/lib/htm/config.rb +65 -361
data/lib/htm/database.rb +85 -127
data/lib/htm/errors.rb +14 -0
data/lib/htm/integrations/sinatra.rb +13 -44
data/lib/htm/job_adapter.rb +75 -1
data/lib/htm/jobs/generate_embedding_job.rb +3 -4
data/lib/htm/jobs/generate_propositions_job.rb +4 -5
data/lib/htm/jobs/generate_tags_job.rb +16 -15
data/lib/htm/loaders/defaults_loader.rb +23 -0
data/lib/htm/loaders/markdown_loader.rb +17 -15
data/lib/htm/loaders/xdg_config_loader.rb +9 -9
data/lib/htm/long_term_memory/fulltext_search.rb +14 -14
data/lib/htm/long_term_memory/hybrid_search.rb +396 -229
data/lib/htm/long_term_memory/node_operations.rb +24 -23
data/lib/htm/long_term_memory/relevance_scorer.rb +23 -20
data/lib/htm/long_term_memory/robot_operations.rb +4 -4
data/lib/htm/long_term_memory/tag_operations.rb +91 -77
data/lib/htm/long_term_memory/vector_search.rb +4 -5
data/lib/htm/long_term_memory.rb +13 -13
data/lib/htm/mcp/cli.rb +115 -8
data/lib/htm/mcp/resources.rb +4 -3
data/lib/htm/mcp/server.rb +5 -4
data/lib/htm/mcp/tools.rb +37 -28
data/lib/htm/migration.rb +72 -0
data/lib/htm/models/file_source.rb +52 -31
data/lib/htm/models/node.rb +224 -108
data/lib/htm/models/node_tag.rb +49 -28
data/lib/htm/models/robot.rb +38 -27
data/lib/htm/models/robot_node.rb +63 -35
data/lib/htm/models/tag.rb +126 -123
data/lib/htm/observability.rb +45 -41
data/lib/htm/proposition_service.rb +76 -7
data/lib/htm/railtie.rb +2 -2
data/lib/htm/robot_group.rb +30 -18
data/lib/htm/sequel_config.rb +215 -0
data/lib/htm/sql_builder.rb +14 -16
data/lib/htm/tag_service.rb +78 -0
data/lib/htm/tasks.rb +3 -0
data/lib/htm/version.rb +1 -1
data/lib/htm/workflows/remember_workflow.rb +213 -0
data/lib/htm.rb +27 -22
data/lib/tasks/db.rake +0 -2
data/lib/tasks/doc.rake +2 -2
data/lib/tasks/files.rake +11 -18
data/lib/tasks/htm.rake +190 -62
data/lib/tasks/jobs.rake +179 -54
data/lib/tasks/tags.rake +8 -13
data/mkdocs.yml +33 -8
data/scripts/backfill_parent_tags.rb +376 -0
data/scripts/normalize_plural_tags.rb +335 -0
metadata +168 -86
data/docs/api/yard/HTM/Configuration.md +0 -240
data/docs/telemetry.md +0 -391
data/examples/rails_app/app/controllers/tags_controller.rb +0 -30
data/examples/sinatra_app/Gemfile.lock +0 -166
data/lib/htm/active_record_config.rb +0 -104
/data/examples/{config_file_example → 02_config_file_example}/README.md +0 -0
/data/examples/{config_file_example → 02_config_file_example}/config/htm.local.yml +0 -0
/data/examples/{config_file_example → 02_config_file_example}/custom_config.yml +0 -0
/data/examples/{config_file_example → 02_config_file_example}/show_config.rb +0 -0
/data/examples/{example_app → 06_example_app}/Rakefile +0 -0
/data/examples/{cli_app → 07_cli_app}/README.md +0 -0
/data/examples/{sinatra_app → 08_sinatra_app}/Gemfile +0 -0
/data/examples/{telemetry → 10_telemetry}/README.md +0 -0
/data/examples/{telemetry → 10_telemetry}/grafana/dashboards/htm-metrics.json +0 -0
/data/examples/{rails_app → 12_rails_app}/.gitignore +0 -0
/data/examples/{rails_app → 12_rails_app}/Procfile.dev +0 -0
/data/examples/{rails_app → 12_rails_app}/README.md +0 -0
/data/examples/{rails_app → 12_rails_app}/Rakefile +0 -0
/data/examples/{rails_app → 12_rails_app}/app/assets/stylesheets/application.css +0 -0
/data/examples/{rails_app → 12_rails_app}/app/assets/stylesheets/inter-font.css +0 -0
/data/examples/{rails_app → 12_rails_app}/app/controllers/application_controller.rb +0 -0
/data/examples/{rails_app → 12_rails_app}/app/controllers/search_controller.rb +0 -0
/data/examples/{rails_app → 12_rails_app}/app/javascript/application.js +0 -0
/data/examples/{rails_app → 12_rails_app}/app/javascript/controllers/application.js +0 -0
/data/examples/{rails_app → 12_rails_app}/app/javascript/controllers/index.js +0 -0
/data/examples/{rails_app → 12_rails_app}/app/views/files/index.html.erb +0 -0
/data/examples/{rails_app → 12_rails_app}/app/views/files/show.html.erb +0 -0
/data/examples/{rails_app → 12_rails_app}/app/views/layouts/application.html.erb +0 -0
/data/examples/{rails_app → 12_rails_app}/app/views/memories/index.html.erb +0 -0
/data/examples/{rails_app → 12_rails_app}/app/views/memories/new.html.erb +0 -0
/data/examples/{rails_app → 12_rails_app}/app/views/robots/new.html.erb +0 -0
/data/examples/{rails_app → 12_rails_app}/app/views/shared/_navbar.html.erb +0 -0
/data/examples/{rails_app → 12_rails_app}/app/views/shared/_stat_card.html.erb +0 -0
/data/examples/{rails_app → 12_rails_app}/bin/dev +0 -0
/data/examples/{rails_app → 12_rails_app}/bin/rails +0 -0
/data/examples/{rails_app → 12_rails_app}/bin/rake +0 -0
/data/examples/{rails_app → 12_rails_app}/config/application.rb +0 -0
/data/examples/{rails_app → 12_rails_app}/config/boot.rb +0 -0
/data/examples/{rails_app → 12_rails_app}/config/database.yml +0 -0
/data/examples/{rails_app → 12_rails_app}/config/environment.rb +0 -0
/data/examples/{rails_app → 12_rails_app}/config/importmap.rb +0 -0
/data/examples/{rails_app → 12_rails_app}/config/routes.rb +0 -0
/data/examples/{rails_app → 12_rails_app}/config/tailwind.config.js +0 -0
/data/examples/{rails_app → 12_rails_app}/config.ru +0 -0
/data/examples/{rails_app → 12_rails_app}/log/.keep +0 -0
/data/examples/{rails_app → 12_rails_app}/tmp/local_secret.txt +0 -0

data/docs/assets/images/hive-mind-shared-memory.svg ADDED Viewed

@@ -0,0 +1,52 @@
+<svg viewBox="0 0 900 600" xmlns="http://www.w3.org/2000/svg" style="background: transparent;">
+  <!-- Title -->
+  <text x="450" y="30" text-anchor="middle" fill="#E0E0E0" font-size="18" font-weight="bold">Hive Mind: Shared Long-Term Memory</text>
+  <!-- Central Database -->
+  <ellipse cx="450" cy="300" rx="180" ry="120" fill="rgba(156, 39, 176, 0.2)" stroke="#9C27B0" stroke-width="3"/>
+  <text x="450" y="280" text-anchor="middle" fill="#E0E0E0" font-size="16" font-weight="bold">Long-Term Memory</text>
+  <text x="450" y="305" text-anchor="middle" fill="#B0B0B0" font-size="12">PostgreSQL</text>
+  <text x="450" y="325" text-anchor="middle" fill="#B0B0B0" font-size="12">Shared Global Database</text>
+  <text x="450" y="345" text-anchor="middle" fill="#4CAF50" font-size="13" font-weight="bold">All Robots Access Here</text>
+  <!-- Robot 1: Code Helper -->
+  <rect x="50" y="80" width="200" height="100" fill="rgba(33, 150, 243, 0.2)" stroke="#2196F3" stroke-width="2" rx="5"/>
+  <text x="150" y="110" text-anchor="middle" fill="#E0E0E0" font-size="14" font-weight="bold">Robot 1: Code Helper</text>
+  <text x="150" y="135" text-anchor="middle" fill="#B0B0B0" font-size="11">ID: robot-abc123</text>
+  <text x="150" y="155" text-anchor="middle" fill="#B0B0B0" font-size="11">Own Working Memory</text>
+  <!-- Robot 2: Research Assistant -->
+  <rect x="650" y="80" width="200" height="100" fill="rgba(76, 175, 80, 0.2)" stroke="#4CAF50" stroke-width="2" rx="5"/>
+  <text x="750" y="110" text-anchor="middle" fill="#E0E0E0" font-size="14" font-weight="bold">Robot 2: Research Bot</text>
+  <text x="750" y="135" text-anchor="middle" fill="#B0B0B0" font-size="11">ID: robot-xyz789</text>
+  <text x="750" y="155" text-anchor="middle" fill="#B0B0B0" font-size="11">Own Working Memory</text>
+  <!-- Robot 3: Chat Companion -->
+  <rect x="50" y="450" width="200" height="100" fill="rgba(255, 152, 0, 0.2)" stroke="#FF9800" stroke-width="2" rx="5"/>
+  <text x="150" y="480" text-anchor="middle" fill="#E0E0E0" font-size="14" font-weight="bold">Robot 3: Chat Bot</text>
+  <text x="150" y="505" text-anchor="middle" fill="#B0B0B0" font-size="11">ID: robot-def456</text>
+  <text x="150" y="525" text-anchor="middle" fill="#B0B0B0" font-size="11">Own Working Memory</text>
+  <!-- Robot 4: Design Assistant -->
+  <rect x="650" y="450" width="200" height="100" fill="rgba(244, 67, 54, 0.2)" stroke="#F44336" stroke-width="2" rx="5"/>
+  <text x="750" y="480" text-anchor="middle" fill="#E0E0E0" font-size="14" font-weight="bold">Robot 4: Designer</text>
+  <text x="750" y="505" text-anchor="middle" fill="#B0B0B0" font-size="11">ID: robot-ghi012</text>
+  <text x="750" y="525" text-anchor="middle" fill="#B0B0B0" font-size="11">Own Working Memory</text>
+  <!-- Connections to central database -->
+  <line x1="150" y1="180" x2="320" y2="240" stroke="#2196F3" stroke-width="3"/>
+  <line x1="750" y1="180" x2="580" y2="240" stroke="#4CAF50" stroke-width="3"/>
+  <line x1="150" y1="450" x2="320" y2="360" stroke="#FF9800" stroke-width="3"/>
+  <line x1="750" y1="450" x2="580" y2="360" stroke="#F44336" stroke-width="3"/>
+  <!-- Labels on connections -->
+  <text x="235" y="210" fill="#2196F3" font-size="10">read/write</text>
+  <text x="650" y="210" fill="#4CAF50" font-size="10">read/write</text>
+  <text x="235" y="410" fill="#FF9800" font-size="10">read/write</text>
+  <text x="650" y="410" fill="#F44336" font-size="10">read/write</text>
+  <!-- Key benefit -->
+  <rect x="300" y="520" width="300" height="60" fill="rgba(76, 175, 80, 0.1)" stroke="#4CAF50" stroke-width="2" rx="5"/>
+  <text x="450" y="545" text-anchor="middle" fill="#4CAF50" font-size="13" font-weight="bold">Knowledge Sharing:</text>
+  <text x="450" y="565" text-anchor="middle" fill="#B0B0B0" font-size="11">All robots see all memories</text>
+</svg>

data/docs/assets/images/htm-architecture-overview.svg CHANGED Viewed

@@ -46,8 +46,8 @@
   <!-- Embedding Service -->
   <rect x="500" y="520" width="280" height="100" fill="rgba(255, 152, 0, 0.3)" stroke="#FF9800" stroke-width="3" rx="8"/>
   <text x="640" y="555" text-anchor="middle" fill="#FFFFFF" font-size="16" font-weight="bold">Embedding Service</text>
-  <text x="640" y="580" text-anchor="middle" fill="#B0B0B0" font-size="13">(Ollama/RubyLLM)</text>
-  <text x="640" y="603" text-anchor="middle" fill="#B0B0B0" font-size="13">gpt-oss model</text>
+  <text x="640" y="580" text-anchor="middle" fill="#B0B0B0" font-size="13">(RubyLLM Multi-Provider)</text>
+  <text x="640" y="603" text-anchor="middle" fill="#B0B0B0" font-size="13">Configurable model</text>
   <!-- Notes -->
   <rect x="30" y="500" width="360" height="160" fill="rgba(33, 150, 243, 0.1)" stroke="#2196F3" stroke-width="2" rx="5" stroke-dasharray="5,5"/>
@@ -56,7 +56,7 @@
   <text x="40" y="570" fill="#B0B0B0" font-size="11">• <tspan fill="#2196F3" font-weight="bold">HTM API</tspan>: Main interface for memory operations</text>
   <text x="40" y="590" fill="#B0B0B0" font-size="11">• <tspan fill="#2196F3" font-weight="bold">Working Memory</tspan>: Fast, token-limited cache</text>
   <text x="40" y="610" fill="#B0B0B0" font-size="11">• <tspan fill="#9C27B0" font-weight="bold">Long-term Memory</tspan>: Durable PostgreSQL storage</text>
-  <text x="40" y="630" fill="#B0B0B0" font-size="11">• <tspan fill="#FF9800" font-weight="bold">Embedding Service</tspan>: Vector generation via Ollama</text>
+  <text x="40" y="630" fill="#B0B0B0" font-size="11">• <tspan fill="#FF9800" font-weight="bold">Embedding Service</tspan>: Vector generation via RubyLLM</text>
   <text x="40" y="650" fill="#B0B0B0" font-size="11">• <tspan fill="#FFC107" font-weight="bold">Eviction/Recall</tspan>: Automatic memory management</text>
   <!-- Markers -->

data/docs/assets/images/htm-core-components.svg CHANGED Viewed

@@ -44,13 +44,13 @@
   <text x="630" y="310" fill="#B0B0B0" font-size="11">@provider: Symbol</text>
   <text x="630" y="330" fill="#B0B0B0" font-size="11">@model: String</text>
   <text x="630" y="350" fill="#B0B0B0" font-size="11">@dimensions: Integer</text>
-  <text x="630" y="370" fill="#B0B0B0" font-size="11">@ollama_url: String</text>
+  <text x="630" y="370" fill="#B0B0B0" font-size="11">@provider_url: String</text>
   <line x1="620" y1="380" x2="840" y2="380" stroke="#FF9800" stroke-width="1"/>
   <text x="630" y="400" fill="#4CAF50" font-size="11">+embed(text)</text>
   <text x="630" y="420" fill="#B0B0B0" font-size="11">+count_tokens(text)</text>
-  <text x="630" y="440" fill="#B0B0B0" font-size="11">+embed_with_ollama()</text>
-  <text x="630" y="460" fill="#B0B0B0" font-size="11">+embed_with_openai()</text>
-  <text x="630" y="480" fill="#FFC107" font-size="10" font-style="italic">// Client-side generation</text>
+  <text x="630" y="440" fill="#B0B0B0" font-size="11">+generate_embedding()</text>
+  <text x="630" y="460" fill="#B0B0B0" font-size="11">+validate_response()</text>
+  <text x="630" y="480" fill="#FFC107" font-size="10" font-style="italic">// RubyLLM multi-provider</text>
   <!-- Relationships -->
   <line x1="450" y1="150" x2="170" y2="250" stroke="#2196F3" stroke-width="2"/>

data/docs/assets/images/htm-layered-architecture.svg CHANGED Viewed

@@ -37,7 +37,7 @@
   <!-- Layer 4: Services Layer -->
   <rect x="100" y="440" width="290" height="100" fill="rgba(244, 67, 54, 0.2)" stroke="#F44336" stroke-width="2" rx="5"/>
   <text x="245" y="465" text-anchor="middle" fill="#E0E0E0" font-size="14" font-weight="bold">Embedding Service</text>
-  <text x="245" y="490" text-anchor="middle" fill="#B0B0B0" font-size="11">Ollama • OpenAI • Cohere</text>
+  <text x="245" y="490" text-anchor="middle" fill="#B0B0B0" font-size="11">RubyLLM (Multi-Provider)</text>
   <text x="245" y="510" text-anchor="middle" fill="#B0B0B0" font-size="11">Vector Generation</text>
   <text x="245" y="530" text-anchor="middle" fill="#B0B0B0" font-size="11">Token Counting</text>

data/docs/assets/images/htm-memory-addition-flow.svg CHANGED Viewed

@@ -14,9 +14,9 @@
   <rect x="470" y="80" width="190" height="60" fill="rgba(255, 152, 0, 0.3)" stroke="#FF9800" stroke-width="3" rx="8"/>
   <text x="565" y="115" text-anchor="middle" fill="#FFFFFF" font-size="14" font-weight="bold">EmbeddingService</text>
-  <!-- Ollama/OpenAI -->
+  <!-- LLM Provider -->
   <rect x="710" y="80" width="180" height="60" fill="rgba(255, 193, 7, 0.3)" stroke="#FFC107" stroke-width="3" rx="8"/>
-  <text x="800" y="115" text-anchor="middle" fill="#FFFFFF" font-size="14" font-weight="bold">Ollama/OpenAI</text>
+  <text x="800" y="115" text-anchor="middle" fill="#FFFFFF" font-size="14" font-weight="bold">LLM Provider</text>
   <!-- LongTermMemory -->
   <rect x="280" y="260" width="180" height="60" fill="rgba(156, 39, 176, 0.3)" stroke="#9C27B0" stroke-width="3" rx="8"/>

data/docs/assets/images/htm-memory-recall-flow.svg CHANGED Viewed

@@ -19,9 +19,9 @@
   <rect x="240" y="180" width="190" height="60" fill="rgba(255, 152, 0, 0.3)" stroke="#2196F3" stroke-width="3" rx="8"/>
   <text x="335" y="215" text-anchor="middle" fill="#FFFFFF" font-size="14" font-weight="bold">EmbeddingService</text>
-  <!-- Ollama/OpenAI -->
+  <!-- LLM Provider -->
   <rect x="480" y="180" width="180" height="60" fill="rgba(255, 193, 7, 0.3)" stroke="#FFC107" stroke-width="3" rx="8"/>
-  <text x="570" y="215" text-anchor="middle" fill="#FFFFFF" font-size="14" font-weight="bold">Ollama/OpenAI</text>
+  <text x="570" y="215" text-anchor="middle" fill="#FFFFFF" font-size="14" font-weight="bold">LLM Provider</text>
   <!-- LongTermMemory -->
   <rect x="240" y="300" width="180" height="60" fill="rgba(156, 39, 176, 0.3)" stroke="#9C27B0" stroke-width="3" rx="8"/>

data/docs/assets/images/memory-topology.svg ADDED Viewed

@@ -0,0 +1,53 @@
+<svg viewBox="0 0 800 500" xmlns="http://www.w3.org/2000/svg" style="background: transparent;">
+  <!-- Title -->
+  <text x="400" y="30" text-anchor="middle" fill="#E0E0E0" font-size="16" font-weight="bold">Memory Topology: Shared LTM + Local WM</text>
+  <!-- Legend -->
+  <rect x="50" y="50" width="20" height="20" fill="rgba(156, 39, 176, 0.3)" stroke="#9C27B0"/>
+  <text x="80" y="65" fill="#B0B0B0" font-size="12">Shared (Global)</text>
+  <rect x="200" y="50" width="20" height="20" fill="rgba(33, 150, 243, 0.3)" stroke="#2196F3"/>
+  <text x="230" y="65" fill="#B0B0B0" font-size="12">Per-Robot (Local)</text>
+  <!-- Robot 1 -->
+  <g transform="translate(0, 100)">
+    <text x="150" y="0" text-anchor="middle" fill="#E0E0E0" font-size="14" font-weight="bold">Robot 1 (Process 1)</text>
+    <rect x="50" y="20" width="200" height="80" fill="rgba(33, 150, 243, 0.2)" stroke="#2196F3" stroke-width="2" rx="5"/>
+    <text x="150" y="50" text-anchor="middle" fill="#E0E0E0" font-size="12">Working Memory</text>
+    <text x="150" y="70" text-anchor="middle" fill="#B0B0B0" font-size="10">In-memory, token-limited</text>
+    <text x="150" y="85" text-anchor="middle" fill="#B0B0B0" font-size="10">Independent</text>
+  </g>
+  <!-- Robot 2 -->
+  <g transform="translate(300, 100)">
+    <text x="150" y="0" text-anchor="middle" fill="#E0E0E0" font-size="14" font-weight="bold">Robot 2 (Process 2)</text>
+    <rect x="50" y="20" width="200" height="80" fill="rgba(33, 150, 243, 0.2)" stroke="#2196F3" stroke-width="2" rx="5"/>
+    <text x="150" y="50" text-anchor="middle" fill="#E0E0E0" font-size="12">Working Memory</text>
+    <text x="150" y="70" text-anchor="middle" fill="#B0B0B0" font-size="10">In-memory, token-limited</text>
+    <text x="150" y="85" text-anchor="middle" fill="#B0B0B0" font-size="10">Independent</text>
+  </g>
+  <!-- Shared Long-Term Memory -->
+  <rect x="150" y="280" width="500" height="150" fill="rgba(156, 39, 176, 0.2)" stroke="#9C27B0" stroke-width="3" rx="5"/>
+  <text x="400" y="310" text-anchor="middle" fill="#E0E0E0" font-size="16" font-weight="bold">Long-Term Memory (Shared)</text>
+  <text x="400" y="340" text-anchor="middle" fill="#B0B0B0" font-size="12">PostgreSQL</text>
+  <text x="400" y="365" text-anchor="middle" fill="#B0B0B0" font-size="12">All robots read/write here</text>
+  <text x="400" y="390" text-anchor="middle" fill="#B0B0B0" font-size="12">Memories attributed with robot_id</text>
+  <text x="400" y="410" text-anchor="middle" fill="#4CAF50" font-size="12" font-weight="bold">Single Source of Truth</text>
+  <!-- Connections -->
+  <line x1="150" y1="200" x2="300" y2="280" stroke="#9C27B0" stroke-width="2" marker-end="url(#arrow-purple)"/>
+  <line x1="450" y1="200" x2="400" y2="280" stroke="#9C27B0" stroke-width="2" marker-end="url(#arrow-purple)"/>
+  <text x="225" y="240" fill="#9C27B0" font-size="10">read/write</text>
+  <text x="425" y="240" fill="#9C27B0" font-size="10">read/write</text>
+  <defs>
+    <marker id="arrow-purple" markerWidth="10" markerHeight="10" refX="9" refY="3" orient="auto">
+      <polygon points="0 0, 10 3, 0 6" fill="#9C27B0"/>
+    </marker>
+  </defs>
+  <!-- Key Point -->
+  <rect x="100" y="460" width="600" height="30" fill="rgba(76, 175, 80, 0.1)" stroke="#4CAF50" stroke-width="1" rx="3"/>
+  <text x="400" y="480" text-anchor="middle" fill="#4CAF50" font-size="12">Each robot has fast local cache (WM) + access to global knowledge (LTM)</text>
+</svg>

data/docs/assets/images/two-tier-memory-architecture.svg ADDED Viewed

@@ -0,0 +1,55 @@
+<svg viewBox="0 0 800 500" xmlns="http://www.w3.org/2000/svg" style="background: transparent;">
+  <!-- Title -->
+  <text x="400" y="30" text-anchor="middle" fill="#E0E0E0" font-size="18" font-weight="bold">Two-Tier Memory Architecture</text>
+  <!-- Working Memory (Hot Tier) -->
+  <rect x="50" y="80" width="300" height="180" fill="rgba(33, 150, 243, 0.2)" stroke="#2196F3" stroke-width="3" rx="5"/>
+  <text x="200" y="110" text-anchor="middle" fill="#E0E0E0" font-size="16" font-weight="bold">Working Memory (Hot)</text>
+  <text x="80" y="140" fill="#B0B0B0" font-size="12">Capacity: Token-limited (128K)</text>
+  <text x="80" y="160" fill="#B0B0B0" font-size="12">Storage: In-memory Ruby Hash</text>
+  <text x="80" y="180" fill="#B0B0B0" font-size="12">Speed: O(1) lookups</text>
+  <text x="80" y="200" fill="#B0B0B0" font-size="12">Lifetime: Process lifetime</text>
+  <text x="80" y="220" fill="#B0B0B0" font-size="12">Eviction: Importance + Recency</text>
+  <text x="80" y="240" fill="#4CAF50" font-size="12" font-weight="bold">Fast, Token-Aware, Volatile</text>
+  <!-- Long-Term Memory (Cold Tier) -->
+  <rect x="450" y="80" width="300" height="180" fill="rgba(156, 39, 176, 0.2)" stroke="#9C27B0" stroke-width="3" rx="5"/>
+  <text x="600" y="110" text-anchor="middle" fill="#E0E0E0" font-size="16" font-weight="bold">Long-Term Memory (Cold)</text>
+  <text x="480" y="140" fill="#B0B0B0" font-size="12">Capacity: Unlimited</text>
+  <text x="480" y="160" fill="#B0B0B0" font-size="12">Storage: PostgreSQL + TimescaleDB</text>
+  <text x="480" y="180" fill="#B0B0B0" font-size="12">Speed: O(log n) with indexes</text>
+  <text x="480" y="200" fill="#B0B0B0" font-size="12">Lifetime: Permanent</text>
+  <text x="480" y="220" fill="#B0B0B0" font-size="12">Retrieval: RAG (semantic + temporal)</text>
+  <text x="480" y="240" fill="#4CAF50" font-size="12" font-weight="bold">Durable, Searchable, Persistent</text>
+  <!-- Data Flow: Add Memory -->
+  <path d="M 200 280 L 200 320 L 400 320 L 400 280" stroke="#4CAF50" stroke-width="3" fill="none" marker-end="url(#arrow-green)"/>
+  <text x="300" y="310" text-anchor="middle" fill="#4CAF50" font-size="12" font-weight="bold">Add Memory</text>
+  <text x="300" y="330" text-anchor="middle" fill="#B0B0B0" font-size="10">(Stored in both tiers)</text>
+  <!-- Data Flow: Eviction -->
+  <path d="M 350 360 L 600 360" stroke="#FF9800" stroke-width="3" marker-end="url(#arrow-orange)"/>
+  <text x="475" y="350" text-anchor="middle" fill="#FF9800" font-size="12" font-weight="bold">Eviction</text>
+  <text x="475" y="380" text-anchor="middle" fill="#B0B0B0" font-size="10">(Token limit → move to LTM only)</text>
+  <!-- Data Flow: Recall -->
+  <path d="M 600 400 L 200 400" stroke="#9C27B0" stroke-width="3" marker-end="url(#arrow-purple)"/>
+  <text x="400" y="390" text-anchor="middle" fill="#9C27B0" font-size="12" font-weight="bold">Recall</text>
+  <text x="400" y="420" text-anchor="middle" fill="#B0B0B0" font-size="10">(RAG search → load back to WM)</text>
+  <!-- Never Forget Note -->
+  <rect x="150" y="450" width="500" height="40" fill="rgba(76, 175, 80, 0.1)" stroke="#4CAF50" stroke-width="1" rx="3"/>
+  <text x="400" y="475" text-anchor="middle" fill="#4CAF50" font-size="13" font-weight="bold">Never Forget: Evicted memories stay in LTM forever (explicit deletion only)</text>
+  <defs>
+    <marker id="arrow-green" markerWidth="10" markerHeight="10" refX="9" refY="3" orient="auto">
+      <polygon points="0 0, 10 3, 0 6" fill="#4CAF50"/>
+    </marker>
+    <marker id="arrow-orange" markerWidth="10" markerHeight="10" refX="9" refY="3" orient="auto">
+      <polygon points="0 0, 10 3, 0 6" fill="#FF9800"/>
+    </marker>
+    <marker id="arrow-purple" markerWidth="10" markerHeight="10" refX="9" refY="3" orient="auto">
+      <polygon points="0 0, 10 3, 0 6" fill="#9C27B0"/>
+    </marker>
+  </defs>
+</svg>

data/docs/database/naming-convention.md ADDED Viewed

@@ -0,0 +1,244 @@
+# Database Naming Convention
+HTM enforces a strict database naming convention to prevent accidental data corruption or loss from operating on the wrong database.
+## The Convention
+Database names **must** follow this exact format:
+```
+{service_name}_{environment}
+```
+Where:
+- `service_name` is the value of `config.service.name` (default: `htm`)
+- `environment` is the value of `HTM_ENV` (or `RAILS_ENV` / `RACK_ENV` fallback)
+## Valid Examples
+| Service Name | Environment | Expected Database Name |
+|--------------|-------------|------------------------|
+| `htm` | `development` | `htm_development` |
+| `htm` | `test` | `htm_test` |
+| `htm` | `production` | `htm_production` |
+| `payroll` | `development` | `payroll_development` |
+| `payroll` | `production` | `payroll_production` |
+## Why This Matters
+Without strict enforcement, dangerous misconfigurations can go undetected:
+### Scenario 1: Environment Mismatch
+```bash
+# Developer thinks they're in test, but connected to production
+HTM_ENV=test
+HTM_DATABASE__URL="postgresql://user@host/htm_production"
+rake htm:db:drop  # DISASTER: Drops production database!
+```
+With the naming convention enforced, this command fails immediately:
+```
+Error: Database name does not follow naming convention!
+  Database names must be: {service_name}_{environment}
+  Service name: htm
+  Environment:  test
+  Expected:     htm_test
+  Actual:       htm_production
+```
+### Scenario 2: Service Mismatch
+```bash
+# HTM configured to use another application's database
+HTM_ENV=production
+# service.name = "htm" (default)
+HTM_DATABASE__URL="postgresql://user@host/payroll_production"
+rake htm:db:setup  # DISASTER: Corrupts payroll application's data!
+```
+With enforcement, this fails:
+```
+Error: Database name does not follow naming convention!
+  Service name: htm
+  Environment:  production
+  Expected:     htm_production
+  Actual:       payroll_production
+```
+## How Enforcement Works
+### Validation Points
+The naming convention is validated at these points:
+1. **All rake tasks** that depend on `htm:db:validate` (setup, migrate, drop, etc.)
+2. **Programmatic access** via `HTM.config.validate_database_name!`
+### No Bypass Option
+There is no way to skip this validation. If your database name doesn't match the convention, you must either:
+1. Rename your database to match the convention
+2. Change `HTM_ENV` to match the database suffix
+3. Change `config.service.name` to match the database prefix
+## Configuration
+### Setting the Service Name
+The service name defaults to `htm`. To use a different name:
+**Via environment variable:**
+```bash
+export HTM_SERVICE__NAME="myapp"
+```
+**Via configuration file (`config/htm.yml`):**
+```yaml
+service:
+  name: myapp
+```
+**Via Ruby configuration:**
+```ruby
+HTM.configure do |config|
+  config.service.name = "myapp"
+end
+```
+### Setting the Environment
+Environment is determined by (in priority order):
+1. `HTM_ENV`
+2. `RAILS_ENV`
+3. `RACK_ENV`
+4. Default: `development`
+**Valid environments:**
+- `development`
+- `test`
+- `production`
+These correspond to the top-level keys in `config/defaults.yml`.
+## Validation Methods
+### Check if Database Name is Valid
+```ruby
+config = HTM.config
+# Boolean check
+if config.valid_database_name?
+  puts "Database name is correct"
+else
+  puts "Expected: #{config.expected_database_name}"
+  puts "Actual: #{config.actual_database_name}"
+end
+```
+### Raise Error on Invalid Name
+```ruby
+config = HTM.config
+# Raises HTM::ConfigurationError if invalid
+config.validate_database_name!
+```
+### Get Expected and Actual Names
+```ruby
+config = HTM.config
+config.expected_database_name  # => "htm_test"
+config.actual_database_name    # => Extracted from URL or config
+```
+## Rake Task Validation
+All database-related rake tasks run validation automatically:
+```bash
+# These all validate the naming convention first:
+rake htm:db:setup
+rake htm:db:migrate
+rake htm:db:drop
+rake htm:db:reset
+rake htm:db:create
+```
+To validate manually without performing any operation:
+```bash
+rake htm:db:validate
+```
+## Migration Guide
+If you have existing databases that don't follow the convention:
+### Option 1: Rename the Database
+```bash
+# PostgreSQL
+psql -c "ALTER DATABASE old_name RENAME TO htm_development;"
+```
+### Option 2: Export and Import
+```bash
+# Export from old database
+pg_dump old_database > backup.sql
+# Create new database with correct name
+createdb htm_development
+# Import to new database
+psql htm_development < backup.sql
+```
+### Option 3: Change Your Service Name
+If your database is named `myapp_production`, set your service name to match:
+```bash
+export HTM_SERVICE__NAME="myapp"
+```
+## Error Messages
+When validation fails, you'll see a clear error message:
+```
+Error: Database name 'wrong_db' does not match expected 'htm_test'.
+Database names must follow the convention: {service_name}_{environment}
+  Service name: htm
+  Environment:  test
+  Expected:     htm_test
+  Actual:       wrong_db
+Either:
+  - Set HTM_DATABASE__URL to point to 'htm_test'
+  - Set HTM_DATABASE__NAME=htm_test
+  - Change HTM_ENV to match the database suffix
+```
+## Summary
+The strict database naming convention:
+- **Prevents** accidental operations on wrong environments
+- **Prevents** cross-application database corruption
+- **Requires** exact match of `{service_name}_{environment}`
+- **Has no bypass** - you must fix the configuration
+- **Validates automatically** on all database operations

data/docs/database_rake_tasks.md CHANGED Viewed

@@ -219,6 +219,37 @@ $ rake htm:db:reset
 # Runs drop (with confirmation) then setup
 ```
+#### `rake htm:db:purge_all`
+Permanently removes all soft-deleted records from all tables.
+**What it does:**
+- Removes soft-deleted nodes, node_tags, and robot_nodes
+- Removes orphaned join table entries (pointing to non-existent nodes)
+- Removes orphaned propositions (where source_node_id no longer exists)
+- Removes orphaned robots (with no associated memory nodes)
+- Deletes in correct order for referential integrity
+**Safety:** Prompts for confirmation before deletion
+```bash
+$ rake htm:db:purge_all
+HTM Purge All Soft-Deleted Records
+============================================================
+Records to permanently delete:
+--------------------------------------------------------------
+Soft-deleted nodes:              23
+Soft-deleted node_tags:          45
+Orphaned propositions:           5
+Orphaned robots (no nodes):      2
+--------------------------------------------------------------
+Total records to delete:         75
+Proceed with permanent deletion? (yes/no): yes
+✓ Purge complete!
+```
 ---
 ## Environment Variables