@miller-tech/uap 1.0.0

package/docs/integrations/RTK_INTEGRATION.md

@@ -0,0 +1,468 @@
+# RTK Integration Analysis & Implementation Plan
+
+## Executive Summary
+
+**RTK (Rust Token Killer)** is a high-performance CLI proxy that reduces LLM token consumption by **60-90%** on common development commands. It achieves this through intelligent output filtering, compression, and smart routing of CLI commands.
+
+### Key Metrics
+
+- **Token Savings**: 60-90% average (up to 99% for specific commands)
+- **Performance**: <10ms overhead per command
+- **Architecture**: Single Rust binary, zero dependencies
+- **Ecosystem**: 30+ command modules covering Git, JS/TS, Python, Go, Containers, and more
+
+---
+
+## Current UAP Status vs RTK Capabilities
+
+### What UAP Already Has (v2.0.1)
+
+✅ **MCP Router** - Token reduction via hierarchical routing (~98% token reduction)
+
+- Exposes 2 meta-tools: `discover_tools`, `execute_tool`
+- Hides individual tool definitions from LLM context
+- Works at the MCP protocol level
+
+### What RTK Brings (Complementary Layer)
+
+✅ **CLI Output Filtering** - Direct command output compression
+
+- Filters raw stdout/stderr before it reaches LLM
+- Applies language-specific strategies (stats extraction, error grouping, deduplication)
+- Works at the shell command execution level
+
+---
+
+## Strategic Integration Options
+
+### Option 1: Parallel Implementation (Recommended)
+
+```
+┌────────────────────────────────────────────────────────────────────────┐
+│                    TWO-LAYER TOKEN OPTIMIZATION                        │
+└────────────────────────────────────────────────────────────────────────┘
+
+Layer 1: MCP Router (UAP v2.0.1)
+────────────────────────────────
+LLM → [Discover Tools] → [Execute Tool] → Backend Services
+           ↓                    ↓
+      ~483 tokens        ~700 tokens
+      Total: ~1,200 tokens
+
+Layer 2: RTK CLI Proxy (New)
+─────────────────────────────
+Command Execution:
+LLM → rtk git status → [Filter/Compress] → Shell → Git
+                     ↓
+              ~200 tokens (vs ~2,000 raw)
+```
+
+**Benefits**:
+
+- **Synergy**: MCP Router reduces tool definition overhead; RTK reduces command output overhead
+- **Coverage**: All token consumption vectors addressed (tool discovery + command execution)
+- **Zero Conflict**: Independent layers with different scopes
+- **Best of Both**: Rust performance (RTK) + TypeScript ecosystem (UAP)
+
+**Estimated Combined Savings**: 95%+ total token reduction
+
+---
+
+### Option 2: RTK as UAP Subcommand
+
+```bash
+# Proposed syntax
+uap rtk git status          # Uses RTK binary internally
+uap rtk install             # Install RTK to ~/.local/bin/rtk
+uap rtk gain              # Show token savings analytics
+```
+
+**Benefits**:
+
+- Unified CLI experience
+- Single installation command
+- Integrated analytics dashboard
+
+**Implementation Complexity**: Medium (requires bundling Rust binary)
+
+---
+
+### Option 3: Pure TypeScript Implementation
+
+Re-implement RTK filtering logic in TypeScript/Node.js within UAP.
+
+**Benefits**:
+
+- No Rust dependency
+- Easier maintenance for JS/TS team
+- Single codebase
+
+**Drawbacks**:
+
+- Slower performance (Node.js vs Rust)
+- Loss of <10ms overhead guarantee
+- Larger binary size
+
+**Estimated Performance**: ~50-100ms overhead (vs <10ms in RTK)
+
+---
+
+## Recommended Implementation: Option 1 (Parallel)
+
+### Architecture Overview
+
+```
+┌────────────────────────────────────────────────────────────────────────┐
+│                    UAP + RTK Integration                               │
+└────────────────────────────────────────────────────────────────────────┘
+
+User Workflow:
+
+1. Install RTK via Homebrew or cargo
+   $ brew install rtk
+
+2. Initialize hook for Claude Code
+   $ rtk init --global
+   # Registers PreToolUse hook in ~/.claude/settings.json
+
+3. UAP MCP Router runs in background
+   $ uap mcp-router start &
+
+4. All commands transparently optimized:
+   - Git operations → RTK git (85-99% savings)
+   - Tool discovery → MCP Router (~98% savings)
+   - Test output → RTK test (90%+ savings)
+```
+
+### Implementation Phases
+
+#### Phase 1: Documentation & Setup (Week 1)
+
+**Tasks**:
+
+- [ ] Document RTK installation in UAP README
+- [ ] Add `uap install rtk` helper script (auto-installs RTK)
+- [ ] Create integration guide for MCP Router + RTK synergy
+
+**Deliverables**:
+
+- Updated documentation
+- Installation scripts
+- Example configurations
+
+---
+
+#### Phase 2: CLI Integration (Week 2-3)
+
+**Tasks**:
+
+- [ ] Add `uap rtk` subcommand wrapper
+- [ ] Implement auto-detection of RTK installation
+- [ ] Create unified analytics dashboard (`uap token-savings`)
+
+**Code Structure**:
+
+```typescript
+// src/cli/rtk.ts
+export async function rtkCommand(args: string[]) {
+  const rtkPath = detectRtkInstall();
+
+  if (!rtkPath) {
+    console.log('RTK not installed. Run: uap install rtk');
+    return;
+  }
+
+  // Spawn RTK process with args
+  await spawn(rtkPath, args, { stdio: 'inherit' });
+}
+```
+
+**Deliverables**:
+
+- `uap rtk` command
+- Auto-detection logic
+- Installation helper
+
+---
+
+#### Phase 3: Analytics Integration (Week 4)
+
+**Tasks**:
+
+- [ ] Merge RTK token tracking with UAP analytics
+- [ ] Create unified dashboard showing MCP Router + RTK savings
+- [ ] Add export functionality (JSON, CSV, Markdown)
+
+**Data Model**:
+
+```typescript
+interface TokenSavingsReport {
+  mcpRouter: {
+    totalTools: number;
+    traditionalTokens: number;
+    routerTokens: number;
+    savingsPercent: number;
+  };
+  rtk: {
+    totalCommands: number;
+    inputTokens: number;
+    outputTokens: number;
+    savingsPercent: number;
+  };
+  combined: {
+    totalSaved: number;
+    overallSavingsPercent: number;
+  };
+}
+```
+
+**Deliverables**:
+
+- Unified analytics dashboard
+- Export functionality
+- Visual reports (ASCII charts)
+
+---
+
+#### Phase 4: Advanced Features (Week 5-6)
+
+**Tasks**:
+
+- [ ] Implement RTK hook auto-configuration in UAP setup
+- [ ] Add RTK command suggestions based on usage patterns
+- [ ] Create custom filter strategies for UAP-specific commands
+
+**Example**:
+
+```rust
+// Custom rtk module for UAP
+#[command(name = "uap-command")]
+fn run_uap_command() {
+  // Analyze UAP command output
+  // Apply UAP-specific filtering rules
+}
+```
+
+**Deliverables**:
+
+- Auto-configured hooks
+- Smart suggestions
+- Custom filters
+
+---
+
+## RTK Module Coverage Analysis
+
+### Commands Already Supported by RTK (30+ modules)
+
+| Category        | Modules                                                     | Avg Savings | UAP Relevance   |
+| --------------- | ----------------------------------------------------------- | ----------- | --------------- |
+| **Git**         | status, diff, log, add, commit, push, pull                  | 85-99%      | ⭐⭐⭐ Critical |
+| **Code Search** | grep, find, diff                                            | 60-85%      | ⭐⭐ High       |
+| **JS/TS Stack** | lint, tsc, next, prettier, playwright, prisma, vitest, pnpm | 70-99%      | ⭐⭐⭐ Critical |
+| **Python**      | ruff, pytest, pip                                           | 80-95%      | ⭐⭐ High       |
+| **Go**          | go test/build/vet, golangci-lint                            | 75-90%      | ⭐⭐ High       |
+| **Containers**  | docker, podman, kubectl                                     | 60-80%      | ⭐⭐ High       |
+| **VCS**         | gh (GitHub CLI)                                             | 26-87%      | ⭐⭐ High       |
+| **Build/Lint**  | cargo, ruff, tsc                                            | 80-90%      | ⭐⭐⭐ Critical |
+| **Tests**       | vitest, pytest, playwright, go test                         | 90-99%      | ⭐⭐⭐ Critical |
+
+### Commands UAP Should Add (Custom Modules)
+
+1. **UAP Memory Operations**
+
+   ```bash
+   rtk uap memory query "session context"
+   rtk uap task list
+   rtk uap worktree status
+   ```
+
+2. **UAP Hook Management**
+
+   ```bash
+   rtk uap hooks list
+   rtk uap plugins install opencode
+   ```
+
+3. **UAP Droid/Agent Coordination**
+   ```bash
+   rtk uap agent status
+   rtk uap coord query
+   ```
+
+---
+
+## Performance Benchmarks (RTK)
+
+### Typical Command Savings
+
+| Command                   | Raw Tokens | RTK Tokens | Savings |
+| ------------------------- | ---------- | ---------- | ------- |
+| `git status`              | 2,000      | 400        | **80%** |
+| `cat file.rs` (100 lines) | 40,000     | 12,000     | **70%** |
+| `cargo test`              | 25,000     | 2,500      | **90%** |
+| `npm test`                | 25,000     | 2,500      | **90%** |
+| `git diff`                | 10,000     | 2,500      | **75%** |
+| `docker logs <container>` | 8,000      | 800        | **90%** |
+
+### Overhead Analysis
+
+```
+Command Execution Time:
+┌───────────────────────────────────────────────────────────────┐
+│ git status (raw)           │ 5ms execution + 2,000 tokens     │
+│ rtk git status             │ 15ms execution + 400 tokens      │
+│ Overhead                   │ +10ms (~2x slower, but 80% less │
+│                           │               context)            │
+└───────────────────────────────────────────────────────────────┘
+
+Trade-off: 10ms delay vs 1,600 token savings = ~$0.003 cost reduction
+(per million tokens @ $0.03/M)
+```
+
+---
+
+## Installation & Configuration
+
+### Recommended UAP + RTK Setup
+
+```bash
+# Step 1: Install UAP (already done)
+npm install -g universal-agent-protocol@2.0.1
+uap init
+
+# Step 2: Install RTK
+brew install rtk  # macOS/Linux
+# OR
+curl -fsSL https://raw.githubusercontent.com/rtk-ai/rtk/master/install.sh | sh
+# OR
+cargo install --git https://github.com/rtk-ai/rtk
+
+# Step 3: Initialize RTK hook for Claude Code
+rtk init --global
+
+# Step 4: Start MCP Router (UAP)
+uap mcp-router start &
+
+# Step 5: Verify installation
+uap token-savings  # Unified analytics
+rtk gain           # RTK-specific stats
+```
+
+### Configuration Files
+
+**RTK**: `~/.config/rtk/config.toml`
+
+```toml
+[tracking]
+database_path = "~/.local/share/rtk/history.db"
+
+[hooks]
+exclude_commands = ["curl", "wget"]  # Skip these commands
+
+[tee]
+enabled = true  # Save full output on failure
+mode = "failures"
+```
+
+**UAP**: `~/.opencode/config.json` (auto-created by `uap install opencode`)
+
+```json
+{
+  "uapEnabled": true,
+  "mcpRouterEnabled": true,
+  "rtkIntegration": {
+    "enabled": true,
+    "path": "~/.local/bin/rtk",
+    "autoDetect": true
+  }
+}
+```
+
+---
+
+## Risk Assessment
+
+### Low Risk Items ✅
+
+- **Documentation Updates**: Safe to add RTK references
+- **Installation Scripts**: Non-invasive, easy to revert
+- **Analytics Dashboard**: Read-only integration with RTK data
+
+### Medium Risk Items ⚠️
+
+- **CLI Wrappers**: Requires careful error handling for missing RTK binary
+- **Auto-detection**: May fail if RTK installed in non-standard location
+
+### High Risk Items ❌
+
+- **Custom UAP RTK Modules**: Could conflict with existing RTK modules
+- **Bundling Rust Binary**: Increases package size, complicates distribution
+
+---
+
+## Recommendation & Next Steps
+
+### Immediate Actions (Week 1)
+
+1. **Update Documentation**
+   - Add "RTK Integration" section to UAP README
+   - Create installation guide for both tools
+   - Document token savings expectations
+
+2. **Create Installation Helper**
+
+   ```bash
+   # scripts/install-rtk.sh
+   # Auto-detects OS and installs RTK via Homebrew/cargo
+   ```
+
+3. **Add `uap install rtk` Command**
+   ```typescript
+   // src/cli/rtk.ts
+   export async function installRtk() {
+     const installer = detectOS();
+     await installer.run();
+   }
+   ```
+
+### Short-term Goals (Weeks 2-4)
+
+4. **Implement `uap rtk` Wrapper**
+   - Pass-through to RTK binary
+   - Auto-detection and installation prompts
+
+5. **Create Unified Analytics**
+   - Merge MCP Router + RTK stats
+   - Add export functionality
+
+### Long-term Vision (Months 2-3)
+
+6. **Custom UAP RTK Modules**
+   - Memory operations, task management, droid coordination
+
+7. **Advanced Features**
+   - Smart command suggestions based on usage patterns
+   - Custom filter strategies for UAP-specific workflows
+
+---
+
+## Conclusion
+
+**RTK and UAP MCP Router are complementary tools** that address different layers of token optimization:
+
+- **MCP Router**: Reduces tool definition overhead at the protocol level (~98% savings)
+- **RTK**: Compresses CLI command output before it reaches the LLM (60-90% savings)
+
+**Recommended Approach**: Parallel implementation with unified analytics and CLI integration. This provides maximum token savings without architectural complexity or risk.
+
+**Estimated Combined Savings**: 95%+ total token reduction across all LLM interactions.
+
+---
+
+## References
+
+- **RTK Repository**: https://github.com/rtk-ai/rtk
+- **RTK Documentation**: https://www.rtk-ai.app
+- **MCP Router Docs**: `/src/mcp-router/README.md` (UAP v2.0.1)
+- **Architecture Deep Dive**: See RTK's `ARCHITECTURE.md` for implementation details