htm 0.0.18 → 0.0.20
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +59 -1
- data/README.md +12 -0
- data/db/seeds.rb +1 -1
- data/docs/api/embedding-service.md +140 -110
- data/docs/api/yard/HTM/ActiveRecordConfig.md +6 -0
- data/docs/api/yard/HTM/Config.md +173 -0
- data/docs/api/yard/HTM/ConfigSection.md +28 -0
- data/docs/api/yard/HTM/Database.md +1 -1
- data/docs/api/yard/HTM/Railtie.md +2 -2
- data/docs/api/yard/HTM.md +0 -57
- data/docs/api/yard/index.csv +76 -61
- data/docs/api/yard-reference.md +2 -1
- data/docs/architecture/adrs/003-ollama-embeddings.md +45 -36
- data/docs/architecture/adrs/004-hive-mind.md +1 -1
- data/docs/architecture/adrs/008-robot-identification.md +1 -1
- data/docs/architecture/index.md +11 -9
- data/docs/architecture/overview.md +11 -7
- data/docs/assets/images/balanced-strategy-decay.svg +41 -0
- data/docs/assets/images/class-hierarchy.svg +1 -1
- data/docs/assets/images/eviction-priority.svg +43 -0
- data/docs/assets/images/exception-hierarchy.svg +2 -2
- data/docs/assets/images/hive-mind-shared-memory.svg +52 -0
- data/docs/assets/images/htm-architecture-overview.svg +3 -3
- data/docs/assets/images/htm-core-components.svg +4 -4
- data/docs/assets/images/htm-layered-architecture.svg +1 -1
- data/docs/assets/images/htm-memory-addition-flow.svg +2 -2
- data/docs/assets/images/htm-memory-recall-flow.svg +2 -2
- data/docs/assets/images/memory-topology.svg +53 -0
- data/docs/assets/images/two-tier-memory-architecture.svg +55 -0
- data/docs/development/setup.md +76 -44
- data/docs/examples/basic-usage.md +133 -0
- data/docs/examples/config-files.md +170 -0
- data/docs/examples/file-loading.md +208 -0
- data/docs/examples/index.md +116 -0
- data/docs/examples/llm-configuration.md +168 -0
- data/docs/examples/mcp-client.md +172 -0
- data/docs/examples/rails-integration.md +173 -0
- data/docs/examples/robot-groups.md +210 -0
- data/docs/examples/sinatra-integration.md +218 -0
- data/docs/examples/standalone-app.md +216 -0
- data/docs/examples/telemetry.md +224 -0
- data/docs/examples/timeframes.md +143 -0
- data/docs/getting-started/installation.md +97 -40
- data/docs/getting-started/quick-start.md +28 -11
- data/docs/guides/configuration.md +515 -0
- data/docs/guides/file-loading.md +322 -0
- data/docs/guides/getting-started.md +40 -9
- data/docs/guides/index.md +3 -3
- data/docs/guides/mcp-server.md +30 -12
- data/docs/guides/propositions.md +264 -0
- data/docs/guides/recalling-memories.md +4 -4
- data/docs/guides/search-strategies.md +3 -3
- data/docs/guides/tags.md +318 -0
- data/docs/guides/telemetry.md +229 -0
- data/docs/index.md +8 -16
- data/docs/{architecture → robots}/hive-mind.md +8 -111
- data/docs/robots/index.md +73 -0
- data/docs/{guides → robots}/multi-robot.md +3 -3
- data/docs/{guides → robots}/robot-groups.md +8 -7
- data/docs/{architecture → robots}/two-tier-memory.md +13 -149
- data/docs/robots/why-robots.md +85 -0
- data/lib/htm/config/defaults.yml +4 -4
- data/lib/htm/config.rb +2 -2
- data/lib/htm/job_adapter.rb +75 -1
- data/lib/htm/version.rb +1 -1
- data/lib/htm/workflows/remember_workflow.rb +212 -0
- data/lib/htm.rb +1 -0
- data/mkdocs.yml +33 -8
- metadata +60 -7
- data/docs/api/yard/HTM/Configuration.md +0 -240
- data/docs/telemetry.md +0 -391
|
@@ -0,0 +1,173 @@
|
|
|
1
|
+
# Class: HTM::Config
|
|
2
|
+
**Inherits:** Anyway::Config
|
|
3
|
+
|
|
4
|
+
|
|
5
|
+
HTM Configuration using Anyway Config
|
|
6
|
+
|
|
7
|
+
Schema is defined in lib/htm/config/defaults.yml (single source of truth)
|
|
8
|
+
Configuration uses nested sections for better organization:
|
|
9
|
+
- HTM.config.database.host
|
|
10
|
+
- HTM.config.embedding.provider
|
|
11
|
+
- HTM.config.providers.openai.api_key
|
|
12
|
+
|
|
13
|
+
Configuration sources (lowest to highest priority):
|
|
14
|
+
1. Bundled defaults: lib/htm/config/defaults.yml (ships with gem)
|
|
15
|
+
2. XDG user config:
|
|
16
|
+
* ~/Library/Application Support/htm/htm.yml (macOS only)
|
|
17
|
+
* ~/.config/htm/htm.yml (XDG default)
|
|
18
|
+
* $XDG_CONFIG_HOME/htm/htm.yml (if XDG_CONFIG_HOME is set)
|
|
19
|
+
3. Project config: ./config/htm.yml (environment-specific)
|
|
20
|
+
4. Local overrides: ./config/htm.local.yml (gitignored)
|
|
21
|
+
5. Environment variables (HTM_*)
|
|
22
|
+
6. Explicit values passed to configure block
|
|
23
|
+
|
|
24
|
+
|
|
25
|
+
**`@example`**
|
|
26
|
+
```ruby
|
|
27
|
+
export HTM_EMBEDDING__PROVIDER=openai
|
|
28
|
+
export HTM_EMBEDDING__MODEL=text-embedding-3-small
|
|
29
|
+
export HTM_PROVIDERS__OPENAI__API_KEY=sk-xxx
|
|
30
|
+
```
|
|
31
|
+
**`@example`**
|
|
32
|
+
```ruby
|
|
33
|
+
embedding:
|
|
34
|
+
provider: ollama
|
|
35
|
+
model: nomic-embed-text:latest
|
|
36
|
+
providers:
|
|
37
|
+
ollama:
|
|
38
|
+
url: http://localhost:11434
|
|
39
|
+
```
|
|
40
|
+
**`@example`**
|
|
41
|
+
```ruby
|
|
42
|
+
HTM.configure do |config|
|
|
43
|
+
config.embedding.provider = :openai
|
|
44
|
+
config.embedding.model = 'text-embedding-3-small'
|
|
45
|
+
end
|
|
46
|
+
```
|
|
47
|
+
# Class Methods
|
|
48
|
+
## active_xdg_config_file() {: #method-c-active_xdg_config_file }
|
|
49
|
+
## config_section_with_defaults(section_key ) {: #method-c-config_section_with_defaults }
|
|
50
|
+
Create a coercion that merges incoming value with SCHEMA defaults for a
|
|
51
|
+
section. This ensures env vars like HTM_DATABASE__URL don't lose other
|
|
52
|
+
defaults.
|
|
53
|
+
## deep_merge_hashes(base , overlay ) {: #method-c-deep_merge_hashes }
|
|
54
|
+
Deep merge helper for coercion
|
|
55
|
+
## env() {: #method-c-env }
|
|
56
|
+
## xdg_config_file() {: #method-c-xdg_config_file }
|
|
57
|
+
## xdg_config_paths() {: #method-c-xdg_config_paths }
|
|
58
|
+
XDG Config Path Helpers
|
|
59
|
+
|
|
60
|
+
# Attributes
|
|
61
|
+
## embedding_generator[RW] {: #attribute-i-embedding_generator }
|
|
62
|
+
Callable Accessors (not loaded from config sources)
|
|
63
|
+
|
|
64
|
+
|
|
65
|
+
## logger[RW] {: #attribute-i-logger }
|
|
66
|
+
Returns the value of attribute logger.
|
|
67
|
+
|
|
68
|
+
## proposition_extractor[RW] {: #attribute-i-proposition_extractor }
|
|
69
|
+
Callable Accessors (not loaded from config sources)
|
|
70
|
+
|
|
71
|
+
|
|
72
|
+
## tag_extractor[RW] {: #attribute-i-tag_extractor }
|
|
73
|
+
Callable Accessors (not loaded from config sources)
|
|
74
|
+
|
|
75
|
+
|
|
76
|
+
## token_counter[RW] {: #attribute-i-token_counter }
|
|
77
|
+
Returns the value of attribute token_counter.
|
|
78
|
+
|
|
79
|
+
|
|
80
|
+
# Instance Methods
|
|
81
|
+
## anthropic_api_key() {: #method-i-anthropic_api_key }
|
|
82
|
+
## azure_api_key() {: #method-i-azure_api_key }
|
|
83
|
+
## azure_api_version() {: #method-i-azure_api_version }
|
|
84
|
+
## azure_endpoint() {: #method-i-azure_endpoint }
|
|
85
|
+
## bedrock_access_key() {: #method-i-bedrock_access_key }
|
|
86
|
+
## bedrock_region() {: #method-i-bedrock_region }
|
|
87
|
+
## bedrock_secret_key() {: #method-i-bedrock_secret_key }
|
|
88
|
+
## chunk_overlap() {: #method-i-chunk_overlap }
|
|
89
|
+
## chunk_size() {: #method-i-chunk_size }
|
|
90
|
+
Chunking convenience accessors
|
|
91
|
+
|
|
92
|
+
## circuit_breaker_failure_threshold() {: #method-i-circuit_breaker_failure_threshold }
|
|
93
|
+
Circuit breaker convenience accessors
|
|
94
|
+
|
|
95
|
+
## circuit_breaker_half_open_max_calls() {: #method-i-circuit_breaker_half_open_max_calls }
|
|
96
|
+
## circuit_breaker_reset_timeout() {: #method-i-circuit_breaker_reset_timeout }
|
|
97
|
+
## configure_ruby_llm(providernil) {: #method-i-configure_ruby_llm }
|
|
98
|
+
## database_config() {: #method-i-database_config }
|
|
99
|
+
## database_configured?() {: #method-i-database_configured? }
|
|
100
|
+
**`@return`** [Boolean]
|
|
101
|
+
|
|
102
|
+
## database_url() {: #method-i-database_url }
|
|
103
|
+
Database convenience methods
|
|
104
|
+
|
|
105
|
+
## deepseek_api_key() {: #method-i-deepseek_api_key }
|
|
106
|
+
## development?() {: #method-i-development? }
|
|
107
|
+
**`@return`** [Boolean]
|
|
108
|
+
|
|
109
|
+
## embedding_dimensions() {: #method-i-embedding_dimensions }
|
|
110
|
+
## embedding_model() {: #method-i-embedding_model }
|
|
111
|
+
## embedding_provider() {: #method-i-embedding_provider }
|
|
112
|
+
Embedding convenience accessors
|
|
113
|
+
|
|
114
|
+
## embedding_timeout() {: #method-i-embedding_timeout }
|
|
115
|
+
## environment() {: #method-i-environment }
|
|
116
|
+
## extract_propositions() {: #method-i-extract_propositions }
|
|
117
|
+
## gemini_api_key() {: #method-i-gemini_api_key }
|
|
118
|
+
## huggingface_api_key() {: #method-i-huggingface_api_key }
|
|
119
|
+
## initialize() {: #method-i-initialize }
|
|
120
|
+
Instance Methods
|
|
121
|
+
|
|
122
|
+
|
|
123
|
+
**`@return`** [Config] a new instance of Config
|
|
124
|
+
|
|
125
|
+
## job_backend() {: #method-i-job_backend }
|
|
126
|
+
Job backend convenience accessor
|
|
127
|
+
|
|
128
|
+
## max_embedding_dimension() {: #method-i-max_embedding_dimension }
|
|
129
|
+
## max_tag_depth() {: #method-i-max_tag_depth }
|
|
130
|
+
## normalize_ollama_model(model_name) {: #method-i-normalize_ollama_model }
|
|
131
|
+
Ollama Helpers
|
|
132
|
+
|
|
133
|
+
|
|
134
|
+
## ollama_url() {: #method-i-ollama_url }
|
|
135
|
+
## openai_api_key() {: #method-i-openai_api_key }
|
|
136
|
+
Provider credential convenience accessors
|
|
137
|
+
|
|
138
|
+
## openai_organization() {: #method-i-openai_organization }
|
|
139
|
+
## openai_project() {: #method-i-openai_project }
|
|
140
|
+
## openrouter_api_key() {: #method-i-openrouter_api_key }
|
|
141
|
+
## production?() {: #method-i-production? }
|
|
142
|
+
**`@return`** [Boolean]
|
|
143
|
+
|
|
144
|
+
## proposition_model() {: #method-i-proposition_model }
|
|
145
|
+
## proposition_provider() {: #method-i-proposition_provider }
|
|
146
|
+
Proposition convenience accessors
|
|
147
|
+
|
|
148
|
+
## proposition_timeout() {: #method-i-proposition_timeout }
|
|
149
|
+
## refresh_ollama_models!() {: #method-i-refresh_ollama_models! }
|
|
150
|
+
## relevance_access_weight() {: #method-i-relevance_access_weight }
|
|
151
|
+
## relevance_recency_half_life_hours() {: #method-i-relevance_recency_half_life_hours }
|
|
152
|
+
## relevance_recency_weight() {: #method-i-relevance_recency_weight }
|
|
153
|
+
## relevance_semantic_weight() {: #method-i-relevance_semantic_weight }
|
|
154
|
+
Relevance scoring convenience accessors
|
|
155
|
+
|
|
156
|
+
## relevance_tag_weight() {: #method-i-relevance_tag_weight }
|
|
157
|
+
## reset_to_defaults() {: #method-i-reset_to_defaults }
|
|
158
|
+
## service_name() {: #method-i-service_name }
|
|
159
|
+
Service name convenience accessor
|
|
160
|
+
|
|
161
|
+
## tag_model() {: #method-i-tag_model }
|
|
162
|
+
## tag_provider() {: #method-i-tag_provider }
|
|
163
|
+
Tag convenience accessors
|
|
164
|
+
|
|
165
|
+
## tag_timeout() {: #method-i-tag_timeout }
|
|
166
|
+
## test?() {: #method-i-test? }
|
|
167
|
+
Environment Helpers
|
|
168
|
+
|
|
169
|
+
|
|
170
|
+
**`@return`** [Boolean]
|
|
171
|
+
|
|
172
|
+
## validate!() {: #method-i-validate! }
|
|
173
|
+
## validate_settings!() {: #method-i-validate_settings! }
|
|
@@ -0,0 +1,28 @@
|
|
|
1
|
+
# Class: HTM::ConfigSection
|
|
2
|
+
**Inherits:** Object
|
|
3
|
+
|
|
4
|
+
|
|
5
|
+
ConfigSection provides method access to nested configuration hashes
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
**`@example`**
|
|
9
|
+
```ruby
|
|
10
|
+
section = ConfigSection.new(host: 'localhost', port: 5432)
|
|
11
|
+
section.host # => 'localhost'
|
|
12
|
+
section.port # => 5432
|
|
13
|
+
```
|
|
14
|
+
|
|
15
|
+
# Instance Methods
|
|
16
|
+
## `[](key)` {: #method-i-[] }
|
|
17
|
+
## `[]=(key, value)` {: #method-i-[]= }
|
|
18
|
+
## `each(&block)` {: #method-i-each }
|
|
19
|
+
## `initialize(hash = {})` {: #method-i-initialize }
|
|
20
|
+
**`@return`** [ConfigSection] a new instance of ConfigSection
|
|
21
|
+
|
|
22
|
+
## `keys()` {: #method-i-keys }
|
|
23
|
+
## `merge(other)` {: #method-i-merge }
|
|
24
|
+
## `method_missing(method, *args, &block)` {: #method-i-method_missing }
|
|
25
|
+
## `respond_to_missing?(method, include_private = false)` {: #method-i-respond_to_missing? }
|
|
26
|
+
**`@return`** [Boolean]
|
|
27
|
+
|
|
28
|
+
## `to_h()` {: #method-i-to_h }
|
|
@@ -8,7 +8,7 @@ initialization
|
|
|
8
8
|
|
|
9
9
|
# Class Methods
|
|
10
10
|
## default_config() {: #method-c-default_config }
|
|
11
|
-
Get default database configuration
|
|
11
|
+
Get default database configuration
|
|
12
12
|
|
|
13
13
|
Uses HTM::Config for database settings.
|
|
14
14
|
**`@return`** [Hash, nil] Connection configuration hash with PG-style keys
|
|
@@ -20,8 +20,8 @@ This railtie automatically configures HTM when Rails boots:
|
|
|
20
20
|
```ruby
|
|
21
21
|
# config/initializers/htm.rb
|
|
22
22
|
HTM.configure do |config|
|
|
23
|
-
config.
|
|
24
|
-
config.
|
|
23
|
+
config.embedding_model = 'custom-model'
|
|
24
|
+
config.tag_model = 'custom-tag-model'
|
|
25
25
|
end
|
|
26
26
|
```
|
|
27
27
|
|
data/docs/api/yard/HTM.md
CHANGED
|
@@ -6,61 +6,4 @@ examples/robot_groups/lib/htm/working_memory_channel.rb frozen_string_literal:
|
|
|
6
6
|
true
|
|
7
7
|
|
|
8
8
|
|
|
9
|
-
# Class Methods
|
|
10
|
-
## configure() {: #method-c-configure }
|
|
11
|
-
Configure HTM
|
|
12
|
-
**`@yield`** [config] Configuration object
|
|
13
|
-
|
|
14
|
-
**`@yieldparam`** [HTM::Configuration]
|
|
15
|
-
|
|
16
|
-
|
|
17
|
-
**`@example`**
|
|
18
|
-
```ruby
|
|
19
|
-
HTM.configure do |config|
|
|
20
|
-
config.embedding_generator = ->(text) { MyEmbedder.embed(text) }
|
|
21
|
-
config.tag_extractor = ->(text, ontology) { MyTagger.extract(text, ontology) }
|
|
22
|
-
end
|
|
23
|
-
```
|
|
24
|
-
**`@example`**
|
|
25
|
-
```ruby
|
|
26
|
-
HTM.configure # Uses RubyLLM defaults
|
|
27
|
-
```
|
|
28
|
-
## count_tokens(text ) {: #method-c-count_tokens }
|
|
29
|
-
Count tokens using configured counter
|
|
30
|
-
**`@param`** [String] Text to count tokens for
|
|
31
|
-
|
|
32
|
-
**`@return`** [Integer] Token count
|
|
33
|
-
|
|
34
|
-
## embed(text ) {: #method-c-embed }
|
|
35
|
-
Generate embedding using EmbeddingService
|
|
36
|
-
**`@param`** [String] Text to embed
|
|
37
|
-
|
|
38
|
-
**`@return`** [Array<Float>] Embedding vector (original, not padded)
|
|
39
|
-
|
|
40
|
-
## extract_propositions(text ) {: #method-c-extract_propositions }
|
|
41
|
-
Extract propositions using PropositionService
|
|
42
|
-
**`@param`** [String] Text to analyze
|
|
43
|
-
|
|
44
|
-
**`@return`** [Array<String>] Extracted atomic propositions
|
|
45
|
-
|
|
46
|
-
## extract_tags(text , existing_ontology: []) {: #method-c-extract_tags }
|
|
47
|
-
Extract tags using TagService
|
|
48
|
-
**`@param`** [String] Text to analyze
|
|
49
|
-
|
|
50
|
-
**`@param`** [Array<String>] Sample of existing tags for context
|
|
51
|
-
|
|
52
|
-
**`@return`** [Array<String>] Extracted and validated tag names
|
|
53
|
-
|
|
54
|
-
## logger() {: #method-c-logger }
|
|
55
|
-
Get configured logger
|
|
56
|
-
**`@return`** [Logger] Configured logger instance
|
|
57
|
-
|
|
58
|
-
## reset_configuration!() {: #method-c-reset_configuration! }
|
|
59
|
-
Reset configuration to defaults
|
|
60
|
-
# Attributes
|
|
61
|
-
## configuration[RW] {: #attribute-c-configuration }
|
|
62
|
-
Get current configuration
|
|
63
|
-
|
|
64
|
-
**`@return`** [HTM::Configuration]
|
|
65
|
-
|
|
66
9
|
|
data/docs/api/yard/index.csv
CHANGED
|
@@ -28,14 +28,6 @@ HTM::Telemetry.search_latency,Method,HTM/Telemetry.md#method-c-search_latency
|
|
|
28
28
|
HTM::Telemetry.setup,Method,HTM/Telemetry.md#method-c-setup
|
|
29
29
|
HTM::Telemetry.tag_latency,Method,HTM/Telemetry.md#method-c-tag_latency
|
|
30
30
|
HTM,Class,HTM.md
|
|
31
|
-
HTM.configure,Method,HTM.md#method-c-configure
|
|
32
|
-
HTM.count_tokens,Method,HTM.md#method-c-count_tokens
|
|
33
|
-
HTM.embed,Method,HTM.md#method-c-embed
|
|
34
|
-
HTM.extract_propositions,Method,HTM.md#method-c-extract_propositions
|
|
35
|
-
HTM.extract_tags,Method,HTM.md#method-c-extract_tags
|
|
36
|
-
HTM.logger,Method,HTM.md#method-c-logger
|
|
37
|
-
HTM.reset_configuration!,Method,HTM.md#method-c-reset_configuration!
|
|
38
|
-
configuration,Attribute,HTM.md#attribute-c-configuration
|
|
39
31
|
HTM::ActiveRecordConfig,Class,HTM/ActiveRecordConfig.md
|
|
40
32
|
HTM::ActiveRecordConfig.connected?,Method,HTM/ActiveRecordConfig.md#method-c-connected?
|
|
41
33
|
HTM::ActiveRecordConfig.connection_stats,Method,HTM/ActiveRecordConfig.md#method-c-connection_stats
|
|
@@ -55,59 +47,82 @@ failure_count,Attribute,HTM/CircuitBreaker.md#attribute-i-failure_count
|
|
|
55
47
|
last_failure_time,Attribute,HTM/CircuitBreaker.md#attribute-i-last_failure_time
|
|
56
48
|
name,Attribute,HTM/CircuitBreaker.md#attribute-i-name
|
|
57
49
|
state,Attribute,HTM/CircuitBreaker.md#attribute-i-state
|
|
58
|
-
HTM::
|
|
59
|
-
HTM::
|
|
60
|
-
HTM::
|
|
61
|
-
HTM::
|
|
62
|
-
HTM::
|
|
63
|
-
HTM::
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
106
|
-
|
|
107
|
-
|
|
108
|
-
|
|
109
|
-
|
|
110
|
-
|
|
50
|
+
HTM::ConfigSection,Class,HTM/ConfigSection.md
|
|
51
|
+
HTM::ConfigSection.[],Method,HTM/ConfigSection.md#method-i-[]
|
|
52
|
+
HTM::ConfigSection.[]=,Method,HTM/ConfigSection.md#method-i-[]=
|
|
53
|
+
HTM::ConfigSection.each,Method,HTM/ConfigSection.md#method-i-each
|
|
54
|
+
HTM::ConfigSection.initialize,Method,HTM/ConfigSection.md#method-i-initialize
|
|
55
|
+
HTM::ConfigSection.keys,Method,HTM/ConfigSection.md#method-i-keys
|
|
56
|
+
HTM::ConfigSection.merge,Method,HTM/ConfigSection.md#method-i-merge
|
|
57
|
+
HTM::ConfigSection.method_missing,Method,HTM/ConfigSection.md#method-i-method_missing
|
|
58
|
+
HTM::ConfigSection.respond_to_missing?,Method,HTM/ConfigSection.md#method-i-respond_to_missing?
|
|
59
|
+
HTM::ConfigSection.to_h,Method,HTM/ConfigSection.md#method-i-to_h
|
|
60
|
+
HTM::Config,Class,HTM/Config.md
|
|
61
|
+
HTM::Config.anthropic_api_key,Method,HTM/Config.md#method-i-anthropic_api_key
|
|
62
|
+
HTM::Config.azure_api_key,Method,HTM/Config.md#method-i-azure_api_key
|
|
63
|
+
HTM::Config.azure_api_version,Method,HTM/Config.md#method-i-azure_api_version
|
|
64
|
+
HTM::Config.azure_endpoint,Method,HTM/Config.md#method-i-azure_endpoint
|
|
65
|
+
HTM::Config.bedrock_access_key,Method,HTM/Config.md#method-i-bedrock_access_key
|
|
66
|
+
HTM::Config.bedrock_region,Method,HTM/Config.md#method-i-bedrock_region
|
|
67
|
+
HTM::Config.bedrock_secret_key,Method,HTM/Config.md#method-i-bedrock_secret_key
|
|
68
|
+
HTM::Config.chunk_overlap,Method,HTM/Config.md#method-i-chunk_overlap
|
|
69
|
+
HTM::Config.chunk_size,Method,HTM/Config.md#method-i-chunk_size
|
|
70
|
+
HTM::Config.circuit_breaker_failure_threshold,Method,HTM/Config.md#method-i-circuit_breaker_failure_threshold
|
|
71
|
+
HTM::Config.circuit_breaker_half_open_max_calls,Method,HTM/Config.md#method-i-circuit_breaker_half_open_max_calls
|
|
72
|
+
HTM::Config.circuit_breaker_reset_timeout,Method,HTM/Config.md#method-i-circuit_breaker_reset_timeout
|
|
73
|
+
HTM::Config.configure_ruby_llm,Method,HTM/Config.md#method-i-configure_ruby_llm
|
|
74
|
+
HTM::Config.database_config,Method,HTM/Config.md#method-i-database_config
|
|
75
|
+
HTM::Config.database_configured?,Method,HTM/Config.md#method-i-database_configured?
|
|
76
|
+
HTM::Config.database_url,Method,HTM/Config.md#method-i-database_url
|
|
77
|
+
HTM::Config.deepseek_api_key,Method,HTM/Config.md#method-i-deepseek_api_key
|
|
78
|
+
HTM::Config.development?,Method,HTM/Config.md#method-i-development?
|
|
79
|
+
HTM::Config.embedding_dimensions,Method,HTM/Config.md#method-i-embedding_dimensions
|
|
80
|
+
HTM::Config.embedding_model,Method,HTM/Config.md#method-i-embedding_model
|
|
81
|
+
HTM::Config.embedding_provider,Method,HTM/Config.md#method-i-embedding_provider
|
|
82
|
+
HTM::Config.embedding_timeout,Method,HTM/Config.md#method-i-embedding_timeout
|
|
83
|
+
HTM::Config.environment,Method,HTM/Config.md#method-i-environment
|
|
84
|
+
HTM::Config.extract_propositions,Method,HTM/Config.md#method-i-extract_propositions
|
|
85
|
+
HTM::Config.gemini_api_key,Method,HTM/Config.md#method-i-gemini_api_key
|
|
86
|
+
HTM::Config.huggingface_api_key,Method,HTM/Config.md#method-i-huggingface_api_key
|
|
87
|
+
HTM::Config.initialize,Method,HTM/Config.md#method-i-initialize
|
|
88
|
+
HTM::Config.job_backend,Method,HTM/Config.md#method-i-job_backend
|
|
89
|
+
HTM::Config.max_embedding_dimension,Method,HTM/Config.md#method-i-max_embedding_dimension
|
|
90
|
+
HTM::Config.max_tag_depth,Method,HTM/Config.md#method-i-max_tag_depth
|
|
91
|
+
HTM::Config.normalize_ollama_model,Method,HTM/Config.md#method-i-normalize_ollama_model
|
|
92
|
+
HTM::Config.ollama_url,Method,HTM/Config.md#method-i-ollama_url
|
|
93
|
+
HTM::Config.openai_api_key,Method,HTM/Config.md#method-i-openai_api_key
|
|
94
|
+
HTM::Config.openai_organization,Method,HTM/Config.md#method-i-openai_organization
|
|
95
|
+
HTM::Config.openai_project,Method,HTM/Config.md#method-i-openai_project
|
|
96
|
+
HTM::Config.openrouter_api_key,Method,HTM/Config.md#method-i-openrouter_api_key
|
|
97
|
+
HTM::Config.production?,Method,HTM/Config.md#method-i-production?
|
|
98
|
+
HTM::Config.proposition_model,Method,HTM/Config.md#method-i-proposition_model
|
|
99
|
+
HTM::Config.proposition_provider,Method,HTM/Config.md#method-i-proposition_provider
|
|
100
|
+
HTM::Config.proposition_timeout,Method,HTM/Config.md#method-i-proposition_timeout
|
|
101
|
+
HTM::Config.refresh_ollama_models!,Method,HTM/Config.md#method-i-refresh_ollama_models!
|
|
102
|
+
HTM::Config.relevance_access_weight,Method,HTM/Config.md#method-i-relevance_access_weight
|
|
103
|
+
HTM::Config.relevance_recency_half_life_hours,Method,HTM/Config.md#method-i-relevance_recency_half_life_hours
|
|
104
|
+
HTM::Config.relevance_recency_weight,Method,HTM/Config.md#method-i-relevance_recency_weight
|
|
105
|
+
HTM::Config.relevance_semantic_weight,Method,HTM/Config.md#method-i-relevance_semantic_weight
|
|
106
|
+
HTM::Config.relevance_tag_weight,Method,HTM/Config.md#method-i-relevance_tag_weight
|
|
107
|
+
HTM::Config.reset_to_defaults,Method,HTM/Config.md#method-i-reset_to_defaults
|
|
108
|
+
HTM::Config.service_name,Method,HTM/Config.md#method-i-service_name
|
|
109
|
+
HTM::Config.tag_model,Method,HTM/Config.md#method-i-tag_model
|
|
110
|
+
HTM::Config.tag_provider,Method,HTM/Config.md#method-i-tag_provider
|
|
111
|
+
HTM::Config.tag_timeout,Method,HTM/Config.md#method-i-tag_timeout
|
|
112
|
+
HTM::Config.test?,Method,HTM/Config.md#method-i-test?
|
|
113
|
+
HTM::Config.validate!,Method,HTM/Config.md#method-i-validate!
|
|
114
|
+
HTM::Config.validate_settings!,Method,HTM/Config.md#method-i-validate_settings!
|
|
115
|
+
HTM::Config.active_xdg_config_file,Method,HTM/Config.md#method-c-active_xdg_config_file
|
|
116
|
+
HTM::Config.config_section_with_defaults,Method,HTM/Config.md#method-c-config_section_with_defaults
|
|
117
|
+
HTM::Config.deep_merge_hashes,Method,HTM/Config.md#method-c-deep_merge_hashes
|
|
118
|
+
HTM::Config.env,Method,HTM/Config.md#method-c-env
|
|
119
|
+
HTM::Config.xdg_config_file,Method,HTM/Config.md#method-c-xdg_config_file
|
|
120
|
+
HTM::Config.xdg_config_paths,Method,HTM/Config.md#method-c-xdg_config_paths
|
|
121
|
+
embedding_generator,Attribute,HTM/Config.md#attribute-i-embedding_generator
|
|
122
|
+
logger,Attribute,HTM/Config.md#attribute-i-logger
|
|
123
|
+
proposition_extractor,Attribute,HTM/Config.md#attribute-i-proposition_extractor
|
|
124
|
+
tag_extractor,Attribute,HTM/Config.md#attribute-i-tag_extractor
|
|
125
|
+
token_counter,Attribute,HTM/Config.md#attribute-i-token_counter
|
|
111
126
|
HTM::Database,Class,HTM/Database.md
|
|
112
127
|
HTM::Database.default_config,Method,HTM/Database.md#method-c-default_config
|
|
113
128
|
HTM::Database.drop,Method,HTM/Database.md#method-c-drop
|
data/docs/api/yard-reference.md
CHANGED
|
@@ -24,7 +24,8 @@ Complete API documentation generated by [YARD](https://yardoc.org/) with [yard-m
|
|
|
24
24
|
|--------------|-------------|
|
|
25
25
|
| [HTM](yard/HTM.md) | Main API class for memory operations |
|
|
26
26
|
| [HTM::CircuitBreaker](yard/HTM/CircuitBreaker.md) | Fault tolerance for external services |
|
|
27
|
-
| [HTM::
|
|
27
|
+
| [HTM::Config](yard/HTM/Config.md) | HTM::Config class |
|
|
28
|
+
| [HTM::ConfigSection](yard/HTM/ConfigSection.md) | HTM::ConfigSection class |
|
|
28
29
|
| [HTM::Database](yard/HTM/Database.md) | Database schema and connection management |
|
|
29
30
|
| [HTM::EmbeddingService](yard/HTM/EmbeddingService.md) | Vector embedding generation service |
|
|
30
31
|
| [HTM::JobAdapter](yard/HTM/JobAdapter.md) | Background job abstraction layer |
|
|
@@ -1,23 +1,25 @@
|
|
|
1
|
-
# ADR-003:
|
|
1
|
+
# ADR-003: Default Embedding Provider Selection
|
|
2
2
|
|
|
3
|
-
**Status**: Accepted (
|
|
3
|
+
**Status**: Accepted (Updated for Multi-Provider Support)
|
|
4
4
|
|
|
5
|
-
**Date**: 2025-10-25 (Updated: 2025-
|
|
5
|
+
**Date**: 2025-10-25 (Updated: 2025-12-21)
|
|
6
6
|
|
|
7
7
|
**Decision Makers**: Dewayne VanHoozer, Claude (Anthropic)
|
|
8
8
|
|
|
9
9
|
---
|
|
10
10
|
|
|
11
|
-
!!! success "Architecture Status
|
|
12
|
-
|
|
11
|
+
!!! success "Architecture Status"
|
|
12
|
+
HTM uses RubyLLM for embedding generation, supporting multiple providers. **Ollama is the default** for local development, but users can configure any supported provider (OpenAI, Anthropic, Gemini, Azure, Bedrock, DeepSeek) based on their requirements.
|
|
13
13
|
|
|
14
14
|
## Quick Summary
|
|
15
15
|
|
|
16
|
-
HTM uses **Ollama with the nomic-embed-text model** as the default embedding provider, prioritizing local-first, privacy-preserving operation with zero API costs
|
|
16
|
+
HTM uses **Ollama with the nomic-embed-text model** as the **default** embedding provider for local development, prioritizing local-first, privacy-preserving operation with zero API costs. However, **HTM fully supports multiple providers** through RubyLLM, allowing users to choose OpenAI, Gemini, Azure, Bedrock, or other providers for production deployments.
|
|
17
17
|
|
|
18
|
-
**Why**: Local embeddings eliminate API costs, preserve privacy, and enable offline operation
|
|
18
|
+
**Why Ollama as default**: Local embeddings eliminate API costs, preserve privacy, and enable offline operation—ideal for development and privacy-sensitive applications.
|
|
19
19
|
|
|
20
|
-
**
|
|
20
|
+
**Why multi-provider support**: Production deployments may require higher-quality embeddings, managed services, or integration with existing cloud infrastructure.
|
|
21
|
+
|
|
22
|
+
**Impact**: Ollama is recommended for getting started quickly, but users can easily switch providers via configuration for their specific needs.
|
|
21
23
|
|
|
22
24
|
---
|
|
23
25
|
|
|
@@ -47,7 +49,16 @@ HTM requires vector embeddings for semantic search functionality. Embeddings con
|
|
|
47
49
|
|
|
48
50
|
## Decision
|
|
49
51
|
|
|
50
|
-
We will use **Ollama with the
|
|
52
|
+
We will use **Ollama with the nomic-embed-text model** as the **default** embedding provider for HTM, while providing **full support for multiple providers** through RubyLLM:
|
|
53
|
+
|
|
54
|
+
- **Ollama** (default) - Local, privacy-preserving, free
|
|
55
|
+
- **OpenAI** - High-quality, production-ready
|
|
56
|
+
- **Gemini** - Google Cloud integration
|
|
57
|
+
- **Azure** - Enterprise Azure deployments
|
|
58
|
+
- **Bedrock** - AWS integration
|
|
59
|
+
- **DeepSeek** - Cost-effective alternative
|
|
60
|
+
|
|
61
|
+
Users configure their preferred provider via `HTM.configure` or environment variables.
|
|
51
62
|
|
|
52
63
|
---
|
|
53
64
|
|
|
@@ -205,38 +216,36 @@ end
|
|
|
205
216
|
|
|
206
217
|
### User Configuration
|
|
207
218
|
|
|
208
|
-
|
|
209
|
-
With pgai, configuration sets database session variables. Embedding generation happens automatically via triggers.
|
|
219
|
+
Configure your preferred provider globally or per-instance:
|
|
210
220
|
|
|
211
221
|
```ruby
|
|
212
|
-
#
|
|
213
|
-
|
|
214
|
-
|
|
215
|
-
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
219
|
-
|
|
220
|
-
|
|
222
|
+
# Global configuration (recommended)
|
|
223
|
+
HTM.configure do |config|
|
|
224
|
+
# Ollama (default for development)
|
|
225
|
+
config.embedding.provider = :ollama
|
|
226
|
+
config.embedding.model = 'nomic-embed-text'
|
|
227
|
+
|
|
228
|
+
# Or OpenAI (recommended for production)
|
|
229
|
+
# config.embedding.provider = :openai
|
|
230
|
+
# config.embedding.model = 'text-embedding-3-small'
|
|
231
|
+
|
|
232
|
+
# Or Gemini
|
|
233
|
+
# config.embedding.provider = :gemini
|
|
234
|
+
# config.embedding.model = 'text-embedding-004'
|
|
235
|
+
end
|
|
221
236
|
|
|
222
|
-
#
|
|
223
|
-
htm = HTM.new(
|
|
224
|
-
robot_name: "My Robot",
|
|
225
|
-
embedding_provider: :ollama,
|
|
226
|
-
embedding_model: 'mxbai-embed-large', # 1024 dimensions
|
|
227
|
-
embedding_dimensions: 1024
|
|
228
|
-
)
|
|
237
|
+
# Initialize HTM
|
|
238
|
+
htm = HTM.new(robot_name: "My Robot")
|
|
229
239
|
|
|
230
|
-
#
|
|
231
|
-
htm
|
|
232
|
-
|
|
233
|
-
embedding_provider: :openai,
|
|
234
|
-
embedding_model: 'text-embedding-3-small' # 1536 dimensions
|
|
235
|
-
)
|
|
240
|
+
# Add memory - embedding generated via configured provider
|
|
241
|
+
htm.remember("PostgreSQL is awesome", tags: ["database"])
|
|
242
|
+
```
|
|
236
243
|
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
#
|
|
244
|
+
**Environment variables for cloud providers:**
|
|
245
|
+
```bash
|
|
246
|
+
export OPENAI_API_KEY="sk-..." # For OpenAI
|
|
247
|
+
export GEMINI_API_KEY="..." # For Gemini
|
|
248
|
+
export ANTHROPIC_API_KEY="sk-..." # For Anthropic
|
|
240
249
|
```
|
|
241
250
|
|
|
242
251
|
---
|
|
@@ -420,7 +420,7 @@ htm = HTM.new(
|
|
|
420
420
|
- [Multi-Agent Systems](https://en.wikipedia.org/wiki/Multi-agent_system)
|
|
421
421
|
- [ADR-002: Two-Tier Memory](002-two-tier-memory.md)
|
|
422
422
|
- [ADR-008: Robot Identification](008-robot-identification.md)
|
|
423
|
-
- [Multi-Robot Guide](../../
|
|
423
|
+
- [Multi-Robot Guide](../../robots/multi-robot.md)
|
|
424
424
|
|
|
425
425
|
---
|
|
426
426
|
|
|
@@ -608,7 +608,7 @@ htm = HTM.new(
|
|
|
608
608
|
- [ULID Specification](https://github.com/ulid/spec)
|
|
609
609
|
- [Robot Registry Pattern](https://martinfowler.com/eaaCatalog/registry.html)
|
|
610
610
|
- [ADR-004: Hive Mind Architecture](004-hive-mind.md)
|
|
611
|
-
- [Multi-Robot Guide](../../
|
|
611
|
+
- [Multi-Robot Guide](../../robots/multi-robot.md)
|
|
612
612
|
|
|
613
613
|
---
|
|
614
614
|
|
data/docs/architecture/index.md
CHANGED
|
@@ -29,7 +29,7 @@ HTM provides intelligent memory management through five core components that wor
|
|
|
29
29
|
<!-- Embedding Service -->
|
|
30
30
|
<rect x="550" y="200" width="200" height="120" fill="rgba(255, 152, 0, 0.2)" stroke="#FF9800" stroke-width="2" rx="5"/>
|
|
31
31
|
<text x="650" y="235" text-anchor="middle" fill="#E0E0E0" font-size="14" font-weight="bold">Embedding Service</text>
|
|
32
|
-
<text x="650" y="255" text-anchor="middle" fill="#B0B0B0" font-size="11">
|
|
32
|
+
<text x="650" y="255" text-anchor="middle" fill="#B0B0B0" font-size="11">Multi-Provider (RubyLLM)</text>
|
|
33
33
|
<text x="650" y="275" text-anchor="middle" fill="#B0B0B0" font-size="11">Vector Embeddings</text>
|
|
34
34
|
<text x="650" y="295" text-anchor="middle" fill="#B0B0B0" font-size="11">Semantic Search</text>
|
|
35
35
|
|
|
@@ -113,12 +113,14 @@ Durable PostgreSQL storage for permanent knowledge retention. All memories are s
|
|
|
113
113
|
|
|
114
114
|
Generates vector embeddings for semantic search and manages token counting for memory management.
|
|
115
115
|
|
|
116
|
-
**Supported Providers:**
|
|
116
|
+
**Supported Providers (via RubyLLM):**
|
|
117
117
|
|
|
118
|
-
- **Ollama** (default): Local embedding models (
|
|
118
|
+
- **Ollama** (default): Local embedding models (nomic-embed-text, mxbai-embed-large)
|
|
119
119
|
- **OpenAI**: text-embedding-3-small, text-embedding-3-large
|
|
120
|
-
- **
|
|
121
|
-
- **
|
|
120
|
+
- **Gemini**: text-embedding-004
|
|
121
|
+
- **Azure**: Azure OpenAI Service
|
|
122
|
+
- **Bedrock**: Amazon Titan, Cohere models
|
|
123
|
+
- **DeepSeek**: DeepSeek embeddings
|
|
122
124
|
|
|
123
125
|
**Related ADRs:** [ADR-003](adrs/003-ollama-embeddings.md)
|
|
124
126
|
|
|
@@ -272,8 +274,8 @@ stateDiagram-v2
|
|
|
272
274
|
Explore detailed architecture documentation:
|
|
273
275
|
|
|
274
276
|
- **[Detailed Architecture](overview.md)** - Deep dive into system architecture, data flows, and performance characteristics
|
|
275
|
-
- **[Two-Tier Memory System](two-tier-memory.md)** - Working memory and long-term memory design, eviction strategies, and context assembly
|
|
276
|
-
- **[Hive Mind Architecture](hive-mind.md)** - Multi-robot shared memory, robot identification, and cross-robot knowledge sharing
|
|
277
|
+
- **[Two-Tier Memory System](../robots/two-tier-memory.md)** - Working memory and long-term memory design, eviction strategies, and context assembly
|
|
278
|
+
- **[Hive Mind Architecture](../robots/hive-mind.md)** - Multi-robot shared memory, robot identification, and cross-robot knowledge sharing
|
|
277
279
|
|
|
278
280
|
## Technology Stack
|
|
279
281
|
|
|
@@ -284,7 +286,7 @@ Explore detailed architecture documentation:
|
|
|
284
286
|
| **Time-Series** | TimescaleDB | Hypertable partitioning, compression |
|
|
285
287
|
| **Vector Search** | pgvector | Semantic similarity (HNSW) |
|
|
286
288
|
| **Full-Text** | pg_trgm | Fuzzy text matching |
|
|
287
|
-
| **Embeddings** |
|
|
289
|
+
| **Embeddings** | RubyLLM (multi-provider) | Vector generation |
|
|
288
290
|
| **Connection Pool** | connection_pool gem | Database connection management |
|
|
289
291
|
| **Testing** | Minitest | Test framework |
|
|
290
292
|
|
|
@@ -318,7 +320,7 @@ Explore detailed architecture documentation:
|
|
|
318
320
|
|
|
319
321
|
- **Working Memory**: Limited by process RAM (~1-2GB for 128K tokens)
|
|
320
322
|
- **Database**: PostgreSQL scales to TBs with proper indexing
|
|
321
|
-
- **Embeddings**: Local models (Ollama) bounded by GPU/CPU
|
|
323
|
+
- **Embeddings**: Local models (Ollama) bounded by GPU/CPU; cloud providers scale independently
|
|
322
324
|
|
|
323
325
|
### Horizontal Scaling
|
|
324
326
|
|