htm 0.0.1

data/.architecture/decisions/adrs/006-context-assembly-strategies.md

@@ -0,0 +1,444 @@
+# ADR-006: Context Assembly Strategies
+
+**Status**: Accepted
+
+**Date**: 2025-10-25
+
+**Decision Makers**: Dewayne VanHoozer, Claude (Anthropic)
+
+---
+
+## ⚠️ UPDATE (2025-10-28)
+
+**No changes to this ADR's implementation.**
+
+After initial struggles with database configuration, the decision was made to drop the TimescaleDB extension as it was not providing sufficient value for the current proof-of-concept applications. Context assembly strategies are unaffected as they operate on working memory, not the database layer.
+
+See [ADR-001](001-use-postgresql-timescaledb-storage.md) for details on the TimescaleDB removal.
+
+---
+
+## Context
+
+When preparing context for an LLM, working memory may contain more information than can fit within token limits. HTM needs to intelligently select which memories to include in the assembled context string.
+
+Challenges in context assembly:
+
+- **Token limits**: LLMs have finite context windows (even with 128K working memory)
+- **Relevance**: Not all memories are equally important for current task
+- **Recency bias**: Recent context often more relevant, but not always
+- **Priority conflicts**: Important old memories vs. less important recent ones
+- **Performance**: Context assembly should be fast (< 10ms)
+
+Alternative approaches:
+
+1. **FIFO (First-In-First-Out)**: Always include oldest memories
+2. **LIFO (Last-In-First-Out)**: Always include newest memories
+3. **Importance-only**: Sort by importance score
+4. **Recency-only**: Sort by access time
+5. **Balanced (hybrid)**: Combine importance and recency with decay function
+
+## Decision
+
+We will implement **three context assembly strategies**: recent, important, and balanced, allowing users to choose based on their use case.
+
+### Strategy Definitions
+
+**1. Recent (`:recent`)**
+
+- Sort by access order, most recently accessed first
+- Best for: Conversational continuity, following current discussion thread
+- Use case: Chat interfaces, debugging sessions, iterative development
+
+**2. Important (`:important`)**
+
+- Sort by importance score, highest first
+- Best for: Critical information, architectural decisions, key facts
+- Use case: Decision-making, strategic planning, summarization
+
+**3. Balanced (`:balanced`)** - **Recommended Default**
+
+- Hybrid scoring: `importance * (1.0 / (1 + recency_hours))`
+- Importance weighted by time decay (1-hour half-life)
+- Best for: General-purpose context assembly
+- Use case: Most LLM interactions, mixed conversational + strategic context
+
+### Default Strategy
+
+**Balanced** is the recommended default as it provides the best general-purpose behavior, preserving both important long-term knowledge and recent conversational context.
+
+## Rationale
+
+### Why Multiple Strategies?
+
+**Different tasks need different context**:
+
+- Chat conversation: Recent context critical for coherence
+- Strategic planning: Important decisions matter more than recency
+- General assistance: Balance of both
+
+**User control over LLM context**:
+
+- Explicit strategy selection gives predictable behavior
+- No hidden heuristics or magic
+- Easy to debug context issues
+
+**Performance and simplicity**:
+
+- All strategies are simple sorts (O(n log n))
+- No ML models or complex algorithms
+- Fast context assembly (< 10ms for typical working memory)
+
+### Implementation Details
+
+```ruby
+def assemble_context(strategy:, max_tokens: nil)
+  max = max_tokens || @max_tokens
+
+  nodes = case strategy
+  when :recent
+    # Most recently accessed first
+    @access_order.reverse.map { |k| @nodes[k] }
+
+  when :important
+    # Highest importance first
+    @nodes.sort_by { |k, v| -v[:importance] }.map(&:last)
+
+  when :balanced
+    # Importance * recency decay (1-hour half-life)
+    @nodes.sort_by { |k, v|
+      recency_hours = (Time.now - v[:added_at]) / 3600.0
+      score = v[:importance] * (1.0 / (1 + recency_hours))
+      -score  # Negate for descending sort
+    }.map(&:last)
+
+  else
+    raise ArgumentError, "Unknown strategy: #{strategy}"
+  end
+
+  # Build context up to token limit
+  context_parts = []
+  current_tokens = 0
+
+  nodes.each do |node|
+    break if current_tokens + node[:token_count] > max
+    context_parts << node[:value]
+    current_tokens += node[:token_count]
+  end
+
+  context_parts.join("\n\n")
+end
+```
+
+### User API
+
+```ruby
+# Use balanced strategy (recommended default)
+context = htm.create_context(strategy: :balanced)
+
+# Use recent for conversational continuity
+context = htm.create_context(strategy: :recent, max_tokens: 4000)
+
+# Use important for strategic decisions
+context = htm.create_context(strategy: :important)
+```
+
+### Decay Function Analysis
+
+The balanced strategy uses a **1-hour half-life decay**:
+
+```
+score = importance * (1.0 / (1 + hours))
+
+Examples:
+- Just added (0 hours): importance * 1.0 (full weight)
+- 1 hour old: importance * 0.5 (half weight)
+- 3 hours old: importance * 0.25 (quarter weight)
+- 24 hours old: importance * 0.04 (4% weight)
+```
+
+This means:
+
+- Recent memories get full importance weight
+- Importance decays quickly in first few hours
+- Very old memories need high importance to compete
+
+## Consequences
+
+### Positive
+
+✅ **Flexible**: Choose strategy based on use case
+✅ **Predictable**: Clear sorting behavior, no hidden heuristics
+✅ **Fast**: Simple sorting algorithms, < 10ms
+✅ **Debuggable**: Easy to understand why context contains certain memories
+✅ **User control**: Explicit strategy selection
+✅ **Sensible default**: Balanced strategy works well for most cases
+✅ **Token-aware**: Respects max_tokens limit in all strategies
+
+### Negative
+
+❌ **Strategy selection burden**: User must understand differences
+❌ **No automatic optimization**: Doesn't learn optimal strategy
+❌ **Decay tuning**: 1-hour half-life may not suit all use cases
+❌ **No semantic clustering**: Doesn't group related memories together
+❌ **Position bias**: Earlier context may have more LLM influence
+
+### Neutral
+
+➡️ **Importance scoring**: Requires user to assign meaningful importance
+➡️ **Access tracking**: Recent strategy depends on access order
+➡️ **Token estimation**: Accuracy depends on token counting precision
+
+## Design Decisions
+
+### Decision: Three Strategies Instead of One
+**Rationale**: Different use cases benefit from different strategies. Flexibility > simplicity.
+
+**Alternative**: Single balanced strategy for all use cases
+**Rejected**: Forces one-size-fits-all approach, limits user control
+
+### Decision: Balanced as Default
+**Rationale**: Best general-purpose behavior, balances competing priorities
+
+**Alternative**: Recent as default
+**Rejected**: Important long-term knowledge gets lost in conversations
+
+### Decision: 1-Hour Decay Half-Life
+**Rationale**:
+
+- 1 hour matches typical development session length
+- Quick decay prevents stale context from dominating
+- Long enough to preserve within-session continuity
+
+**Alternative**: Configurable decay parameter
+**Deferred**: Can add if real-world usage shows need for tuning
+
+### Decision: Linear Decay (1 / (1 + hours))
+**Rationale**: Simple, predictable, computationally cheap
+
+**Alternative**: Exponential decay `exp(-λ * hours)`
+**Rejected**: More complex, harder to reason about, minimal practical difference
+
+### Decision: Token Limit Enforced Strictly
+**Rationale**: Never exceed LLM context window, prevent truncation errors
+
+**Alternative**: Soft limit with warnings
+**Rejected**: Hard limits prevent surprising behavior
+
+## Use Cases
+
+### Use Case 1: Conversational Chat
+```ruby
+# User having back-and-forth conversation with LLM
+# Recent context is critical for coherence
+
+context = htm.create_context(strategy: :recent, max_tokens: 8000)
+
+# Example memories in working memory:
+# - "User prefers debug_me over puts" (importance: 9, 5 days old)
+# - "What is the capital of France?" (importance: 1, 2 minutes ago)
+# - "Paris is the capital" (importance: 1, 1 minute ago)
+
+# Result: Recent conversation about Paris included first,
+# even though user preference is more important
+```
+
+### Use Case 2: Strategic Planning
+```ruby
+# LLM helping with architectural decisions
+# Important decisions matter more than recent chat
+
+context = htm.create_context(strategy: :important)
+
+# Example memories:
+# - "We decided to use PostgreSQL" (importance: 10, 3 days ago)
+# - "What time is it?" (importance: 1, 1 minute ago)
+# - "TimescaleDB chosen for time-series" (importance: 9, 2 days ago)
+
+# Result: Architectural decisions included first,
+# time check ignored if space limited
+```
+
+### Use Case 3: General Assistance (Balanced)
+```ruby
+# LLM helping with mixed tasks
+# Balance recent context + important knowledge
+
+context = htm.create_context(strategy: :balanced)
+
+# Example memories:
+# - "User prefers debug_me" (importance: 9, 5 days ago) → score: 0.007
+# - "PostgreSQL decision" (importance: 10, 3 days ago) → score: 0.014
+# - "Current debugging issue" (importance: 5, 10 minutes ago) → score: 3.0
+# - "Error: foreign key violation" (importance: 7, 2 minutes ago) → score: 21.0
+
+# Result: Recent debugging context ranked highest,
+# but important decisions still included if space permits
+```
+
+### Use Case 4: Custom Token Limit
+```ruby
+# Generate short summary for preview
+short_context = htm.create_context(strategy: :important, max_tokens: 500)
+
+# Generate full context for LLM
+full_context = htm.create_context(strategy: :balanced, max_tokens: 128_000)
+```
+
+## Performance Characteristics
+
+### Complexity
+
+- **Recent**: O(n) - reverse access order array
+- **Important**: O(n log n) - sort by importance
+- **Balanced**: O(n log n) - sort by computed score
+
+### Typical Performance
+
+- **Working memory size**: 50-200 nodes
+- **Sorting time**: < 5ms (all strategies)
+- **String assembly**: < 5ms
+- **Total**: < 10ms for context assembly
+
+### Memory Usage
+
+- **No duplication**: Nodes stored once, sorted references
+- **Temporary arrays**: O(n) for sorted node references
+- **Output string**: O(total_tokens) for assembled context
+
+## Risks and Mitigations
+
+### Risk: Suboptimal Decay Parameter
+
+- **Risk**: 1-hour half-life doesn't match real usage patterns
+- **Likelihood**: Medium (usage patterns vary)
+- **Impact**: Low (balanced strategy still works reasonably)
+- **Mitigation**:
+  - Monitor real-world usage patterns
+  - Make decay configurable if needed
+  - Document decay behavior clearly
+
+### Risk: Strategy Confusion
+
+- **Risk**: Users don't understand which strategy to use
+- **Likelihood**: Medium (three choices require understanding)
+- **Impact**: Low (balanced default works well)
+- **Mitigation**:
+  - Clear documentation with use cases
+  - Examples in API docs
+  - Sensible default (balanced)
+
+### Risk: Position Bias in LLM
+
+- **Risk**: LLM pays more attention to early context
+- **Likelihood**: High (known LLM behavior)
+- **Impact**: Medium (affects which memories have most influence)
+- **Mitigation**:
+  - Document bias in user guide
+  - Consider reverse ordering for some LLMs (future)
+  - Let users experiment with strategies
+
+### Risk: Importance Scoring Inconsistency
+
+- **Risk**: Users assign arbitrary importance scores
+- **Likelihood**: High (subjective scoring)
+- **Impact**: Medium (affects balanced and important strategies)
+- **Mitigation**:
+  - Document importance scoring guidelines
+  - Provide default importance (1.0) for most memories
+  - Consider learned importance in future
+
+## Future Enhancements
+
+### Automatic Strategy Selection
+```ruby
+# Detect conversation vs planning context
+context = htm.create_context_smart()
+
+# Uses recent for conversational turns
+# Uses important for strategic questions
+# Uses balanced for mixed contexts
+```
+
+### Configurable Decay
+```ruby
+# Adjust decay half-life
+context = htm.create_context(
+  strategy: :balanced,
+  decay_hours: 0.5  # Faster decay
+)
+```
+
+### Semantic Clustering
+```ruby
+# Group related memories together
+context = htm.create_context(
+  strategy: :clustered,
+  cluster_by: :embedding  # Group semantically related nodes
+)
+```
+
+### LLM-Optimized Ordering
+```ruby
+# Reverse ordering for LLMs with recency bias
+context = htm.create_context(
+  strategy: :balanced,
+  order: :reverse  # Most important last (closer to query)
+)
+```
+
+### Multi-Factor Scoring
+```ruby
+# Custom scoring function
+context = htm.create_context(
+  strategy: :custom,
+  score_fn: ->(node) {
+    recency = Time.now - node[:added_at]
+    importance = node[:importance]
+    access_count = node[:access_count]
+
+    importance * (1.0 / (1 + recency / 3600)) * Math.log(1 + access_count)
+  }
+)
+```
+
+## Alternatives Considered
+
+### Always Include Everything
+**Pros**: No selection logic needed, comprehensive context
+**Cons**: Exceeds token limits, includes irrelevant information
+**Decision**: ❌ Rejected - token limits are real constraints
+
+### LLM-Powered Selection
+**Pros**: Most intelligent selection, context-aware
+**Cons**: Slow (extra LLM call), expensive, circular dependency
+**Decision**: ❌ Rejected - too slow for context assembly path
+
+### Learned Importance
+**Pros**: Automatic optimization based on usage
+**Cons**: Complex, requires training data, non-deterministic
+**Decision**: 🔄 Deferred - consider for v2 after usage data
+
+### Semantic Similarity to Query
+**Pros**: Most relevant to current query
+**Cons**: Requires query embedding, slower, breaks generality
+**Decision**: 🔄 Deferred - recall() already does this, context assembly is different
+
+## References
+
+- [LLM Context Window Management](https://arxiv.org/abs/2307.03172)
+- [Attention Mechanisms in LLMs](https://arxiv.org/abs/1706.03762)
+- [Position Bias in Language Models](https://arxiv.org/abs/2302.00093)
+- [Working Memory in Cognitive Science](https://en.wikipedia.org/wiki/Working_memory)
+
+## Review Notes
+
+**AI Engineer**: ✅ Three strategies cover common use cases well. Balanced default is smart. Consider position bias documentation.
+
+**Performance Specialist**: ✅ O(n log n) sorting is fast enough for typical working memory sizes. No concerns.
+
+**Ruby Expert**: ✅ Clean API design. Consider default parameter: `create_context(strategy: :balanced)` → `create_context(strategy = :balanced)`.
+
+**Domain Expert**: ✅ Decay function is intuitive. 1-hour half-life matches development session rhythm.
+
+**Systems Architect**: ✅ Strategy pattern is appropriate. Document decision matrix for users.