adaptive-memory-multi-model-router 2.14.49 → 2.14.51

package/docs/REDDIT_GAP_ANALYSIS.md

@@ -0,0 +1,299 @@
+# TMLPD PI - Reddit Feature Gap Analysis
+
+> Based on Reddit (r/LocalLLaMA, r/AI_Agents, r/MachineLearning, r/llm) pain points.
+> Identifying what's missing vs what developers actually need.
+
+---
+
+## Top Reddit Complaints vs TMLPD Status
+
+| Issue | Reddit Popularity | TMLPD Status | Gap |
+|-------|------------------|--------------|-----|
+| **Reliable function calling** | 🔥🔥🔥 (1957 pts) | ❌ Not supported | **CRITICAL** |
+| **Cost tracking per model** | 🔥🔥🔥 (measurement obsession) | ✅ Full tracking | OK |
+| **Intelligent failover** | 🔥🔥🔥 (multi-provider) | ✅ Auto-fallback | OK |
+| **Token compression** | 🔥🔥 (70% reduction interest) | ❌ Not supported | **HIGH** |
+| **KV cache reuse** | 🔥🔥 (253 comments) | ❌ L1 cache only | **HIGH** |
+| **Batch processing** | 🔥🔥 (throughput) | ❌ Sequential only | **MEDIUM** |
+| **Local orchestration** | 🔥🔥 (privacy) | ⚠️ API-dependent | **MEDIUM** |
+| **Multi-model orchestration** | 🔥 (815 comments Claude Code) | ✅ HALO | OK |
+| **Rust-based speed** | 🔥 (no Python overhead) | ⚠️ TypeScript only | **LOW** |
+
+---
+
+## Missing Capabilities - Detailed Analysis
+
+### 1. Function Calling / Tool Use ❌
+
+**Reddit Pain:** "Function calling broke constantly" (1957 points)
+
+```python
+# What developers want:
+result = await client.execute_with_tools(
+    prompt="What's the weather in NYC?",
+    tools=[get_weather, get_time, search_web],
+    fallback_enabled=True
+)
+# Intelligent routing to providers with best function-calling accuracy
+```
+
+**Gap:** TMLPD only does text completion. No tool/function calling support.
+
+**Recommendation:** Add `execute_with_tools()` with:
+- Provider capability detection (Anthropic > OpenAI > Google for function calling)
+- Parallel tool execution
+- Fallback chain if primary provider's function calling fails
+
+---
+
+### 2. Token Compression ⚠️
+
+**Reddit Pain:** "ISON format for 70% token reduction" (high engagement)
+
+```python
+# What developers want:
+compressed = compress_context(conversation, ratio=0.7)
+# "ISON" or similar encoding
+
+result = await client.execute(compressed, model="premium")
+```
+
+**Gap:** TMLPD has no compression. Context window costs are high.
+
+**Recommendation:** Add compression utilities:
+- `compress_context(messages, strategy="ison")` 
+- `smart_truncate(conversation, max_tokens)`
+- Token counting before API calls
+
+---
+
+### 3. KV Cache Management ❌
+
+**Reddit Pain:** "Serving 1B+ tokens/day locally" - KV cache reuse critical
+
+```typescript
+// What developers want:
+const cache = await tmlpd.getKVCache(model="llama-3.3-70b");
+cache.store({ prompt: "...", kv_state: [...] });
+// Reuse KV state for similar prompts
+```
+
+**Gap:** We have L1 response cache, but not KV cache.
+
+**Recommendation:** Add `KVCacheManager`:
+- Store KV states for reuse
+- Semantic matching of KV cache entries
+- Automatic KV cache for repeated context patterns
+
+---
+
+### 4. Batch Processing ❌
+
+**Reddit Pain:** "Throughput optimization" for GPU clusters
+
+```python
+# What developers want:
+batch = await client.execute_batch([
+    "Explain quantum",
+    "Write Python async",
+    "Debug this code"
+], priority="high")  # Returns when all complete
+
+# With concurrency control and rate limiting
+```
+
+**Gap:** TMLPD has parallel but not batch queuing.
+
+**Recommendation:** Add `executeBatch()`:
+- Queue multiple prompts
+- Priority scheduling
+- Batch completion callbacks
+- Rate limit management across batch
+
+---
+
+### 5. Local Model Support ⚠️
+
+**Reddit Pain:** "Privacy-preserving multi-model pipelines", "vLLM multi-GPU"
+
+```python
+# What developers want:
+client = TMLPDClient({
+    providers: ["local/llama-3.3-70b", "local/codellama"],
+    local_endpoint: "http://localhost:8080"
+})
+
+result = await client.execute_parallel(prompt, [
+    "local/llama-3.3-70b",  # No API key needed
+    "openai/gpt-4o"         # Cloud fallback
+])
+```
+
+**Gap:** TMLPD assumes cloud API providers.
+
+**Recommendation:** Add `LocalProvider` support:
+- Ollama integration
+- vLLM integration  
+- LM Studio integration
+- Local provider health monitoring
+
+---
+
+### 6. Multi-Agent Task Orchestration ⚠️
+
+**Reddit Pain:** "Claude Code multi-agent system" (815 comments)
+
+```python
+# What developers want:
+orchestrator = MultiAgentOrchestrator([
+    {"name": "researcher", "model": "claude", "role": "research"},
+    {"name": "coder", "model": "codex", "role": "implement"},
+    {"name": "reviewer", "model": "claude", "role": "review"}
+])
+
+task = "Build a REST API"
+result = await orchestrator.execute(task, workflow="research→code→review")
+```
+
+**Gap:** We have HALO but it's a reference implementation, not production-ready.
+
+**Recommendation:** Enhance HALO to be production-grade:
+- Built-in agent communication protocol
+- State machine for agent transitions
+- Result sharing between agents
+
+---
+
+## Priority Roadmap
+
+### v1.2.0 (High Priority)
+1. ✅ `execute_with_tools()` - Function calling support
+2. ✅ `compress_context()` - Token compression utilities
+3. ✅ `LocalProvider` - Ollama/vLLM integration
+
+### v1.3.0 (Medium Priority)  
+4. ✅ `executeBatch()` - Batch processing
+5. ✅ `KVCacheManager` - KV state reuse
+6. ✅ Enhanced HALO - Production multi-agent
+
+### v1.4.0 (Future)
+- Rust core for speed
+- GPU cluster orchestration
+- Distributed TMLPD
+
+---
+
+## Quick Wins to Add Now
+
+### 1. Token Counter Utility
+
+```typescript
+// Add to TMLPD core
+export function countTokens(text: string, model: string = "gpt-4o"): number {
+  // Approximate: ~1.3 tokens per word for English
+  const words = text.trim().split(/\s+/);
+  return Math.ceil(words.length * 1.3);
+}
+
+export function estimateCost(prompt_tokens: number, completion_tokens: number, model: string): number {
+  const rates = {
+    "gpt-4o": { input: 0.005, output: 0.015 },
+    "claude-3.5-sonnet": { input: 0.003, output: 0.015 },
+    // ...
+  };
+  const rate = rates[model] || rates["gpt-4o"];
+  return (prompt_tokens * rate.input + completion_tokens * rate.output) / 1000;
+}
+```
+
+### 2. Context Truncation Helper
+
+```typescript
+export function truncateToFit(
+  messages: Message[], 
+  max_tokens: number,
+  strategy: "smart" | "first" | "last" = "smart"
+): Message[] {
+  // Smart: keep system + recent + preserve beginning of oldest
+  // First: keep system + first N messages
+  // Last: keep system + last N messages (most common)
+}
+```
+
+### 3. Batch Execution Helper
+
+```typescript
+export async function executeBatch(
+  prompts: string[],
+  options: {
+    concurrency?: number;  // Max parallel
+    model?: string;
+    callback?: (result: ExecutionResult, index: number) => void;
+  }
+): Promise<ExecutionResult[]> {
+  const { concurrency = 5, model, callback } = options;
+  const results: ExecutionResult[] = [];
+  
+  for (let i = 0; i < prompts.length; i += concurrency) {
+    const batch = prompts.slice(i, i + concurrency);
+    const batchResults = await Promise.all(
+      batch.map((p, j) => execute(p, model).then(r => {
+        if (callback) callback(r, i + j);
+        return r;
+      }))
+    );
+    results.push(...batchResults);
+  }
+  
+  return results;
+}
+```
+
+---
+
+## Missing Keywords for Discoverability
+
+Add to package.json:
+
+```json
+"keywords": [
+  "function-calling",
+  "tool-use",
+  "tool-execution",
+  "parallel-tools",
+  "token-compression",
+  "context-compression",
+  "kv-cache",
+  "kv-cache-reuse",
+  "batch-processing",
+  "batch-execution",
+  "local-llm",
+  "ollama",
+  "vllm",
+  "lm-studio",
+  "local-model",
+  "privacy-llm",
+  "multi-agent",
+  "agent-orchestration",
+  "agent-communication",
+  "workflow-orchestration",
+  "cost-estimation",
+  "token-counting"
+]
+```
+
+---
+
+## Summary
+
+**Critical gaps (must fix):**
+1. Function calling / tool use
+2. Token compression
+3. Local model support
+
+**Medium gaps:**
+4. Batch processing
+5. KV cache reuse
+6. Production multi-agent
+
+**TL;DR:** Reddit developers want reliability, cost control, and local execution. TMLPD has cost control covered. Need to add function calling, compression, and local provider support to address 80% of pain points.

package/docs/RELEASE_CHECKLIST.md

@@ -0,0 +1,32 @@
+# Release Checklist
+
+Use this checklist before tagging a release.
+
+## Mandatory quality gates
+
+- [ ] `npm test` passes
+- [ ] `npm run test:py` passes
+- [ ] `npm run eval:routing` passes
+- [ ] `npm run eval:golden` passes
+- [ ] `npm run eval:faults` passes
+- [ ] `npm run eval:report` passes
+
+## Evidence artifacts reviewed
+
+- [ ] `eval/results/latest.json` reviewed for routing summary
+- [ ] `eval/results/fault_injection_latest.json` reviewed for reliability scenarios
+- [ ] `eval/results/shadow_latest.json` reviewed for divergence/cost deltas
+- [ ] `eval/results/report_latest.md` attached to release review
+- [ ] Any baseline change in `eval/baselines/main.json` is intentional and explained
+
+## Documentation consistency
+
+- [ ] `docs/ENGINEERING_SPEC.md` reflects current behavior
+- [ ] `docs/CLAIMS_AND_EVIDENCE.md` mappings are still valid
+- [ ] Public claims do not exceed available evidence
+
+## Release hygiene
+
+- [ ] Version bump completed
+- [ ] Changelog updated
+- [ ] CI green on release commit

package/docs/REPRODUCIBILITY.md

@@ -0,0 +1,63 @@
+# Reproducibility Contract
+
+This document defines the reproducible evaluation contract for A3M Router.
+
+## Environment
+
+- Node: `>=18` (CI uses Node 22)
+- Python: `3.12` (for Python tests)
+- Install:
+  - `npm ci`
+  - `python3 -m pip install pytest pytest-asyncio`
+
+## Required commands
+
+Run in repository root:
+
+```bash
+npm test
+npm run test:py
+npm run eval:routing
+npm run eval:golden
+npm run eval:faults
+npm run eval:shadow
+npm run eval:report
+```
+
+## Deterministic inputs
+
+- Core regression dataset:
+  - `eval/benchmark_dataset.jsonl`
+- Golden snapshot:
+  - `eval/golden_routes.json`
+- Thresholds:
+  - `eval/thresholds.json`
+  - `eval/fault_injection_thresholds.json`
+
+## Artifacts generated
+
+- `eval/results/latest.json`
+- `eval/results/fault_injection_latest.json`
+- `eval/results/shadow_latest.json`
+- `eval/results/report_latest.md`
+
+## Experiment registry
+
+- Every eval run appends to:
+  - `eval/experiments.jsonl` (local artifact)
+- Record includes:
+  - timestamp
+  - commit (if available)
+  - experiment id
+  - dataset version
+  - metrics
+  - decision
+
+## Baseline update policy
+
+- Baseline file: `eval/baselines/main.json`
+- Only update baseline when behavior change is intentional.
+- PR must explain:
+  - what changed
+  - why baseline needs update
+  - expected impact on cost/quality/reliability