smartcontext-proxy 0.1.0 → 0.2.0

package/PLAN-v2.md

@@ -0,0 +1,390 @@
+# SmartContext Proxy v2.0 — Transparent Network Interceptor
+
+## Goal
+
+One-command install transparent proxy that automatically intercepts all LLM API traffic from any app on the system, optimizes context windows, and provides real-time control via system tray + web dashboard.
+
+## Architecture
+
+```
+Any App (CC, OC, curl, Python, browser...)
+    │
+    │ HTTPS → api.anthropic.com:443
+    │
+    ▼
+[macOS System Proxy / pf / Linux iptables]
+    │
+    ▼
+┌──────────────────────────────────────┐
+│  SmartContext Proxy (:4800)           │
+│                                      │
+│  ┌─ CONNECT Handler ──────────────┐  │
+│  │  Incoming HTTPS CONNECT        │  │
+│  │  ↓                             │  │
+│  │  Is hostname an LLM provider?  │  │
+│  │  ├─ NO → TCP tunnel (zero cost)│  │
+│  │  └─ YES → TLS intercept ──┐   │  │
+│  └────────────────────────────┘   │  │
+│                                   │  │
+│  ┌─ LLM Interceptor ──────────┐  │  │
+│  │  TLS terminate (local CA)   │  │  │
+│  │  Parse request              │  │  │
+│  │  Optimize context           │  │  │
+│  │  Forward to real provider   │  │  │
+│  │  Stream response back       │  │  │
+│  │  Index exchange (async)     │  │  │
+│  └─────────────────────────────┘  │  │
+│                                      │
+│  ┌─ Dashboard (:4800/) ───────────┐  │
+│  │  Real-time stats, live feed    │  │
+│  │  Savings, controls, settings   │  │
+│  └────────────────────────────────┘  │
+│                                      │
+│  ┌─ System Tray ──────────────────┐  │
+│  │  Status icon (green/yellow/red)│  │
+│  │  Click → open dashboard        │  │
+│  │  Quick: pause/resume/quit      │  │
+│  │  Tooltip: "142 req, $63 saved" │  │
+│  └────────────────────────────────┘  │
+└──────────────────────────────────────┘
+    │
+    ▼
+Real LLM Provider (api.anthropic.com, api.openai.com, ...)
+```
+
+## Implementation Phases
+
+### Phase 1: HTTPS CONNECT Proxy + Selective MITM
+
+**Goal**: Proxy intercepts HTTPS traffic, passes through non-LLM, MITM's LLM endpoints.
+
+#### 1.1 Local CA Certificate Manager
+- Generate RSA 2048 root CA cert + key on first run
+- Store in `~/.smartcontext/ca/` (root-ca.crt, root-ca.key)
+- Generate per-hostname certs on-the-fly (cached in `~/.smartcontext/ca/hosts/`)
+- Uses `node-forge` for cert generation (pure JS, no native deps)
+- **Files**: `src/tls/ca-manager.ts`
+
+#### 1.2 CA Trust Store Installation
+- macOS: `security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain`
+- Linux: copy to `/usr/local/share/ca-certificates/`, run `update-ca-certificates`
+- Node.js: set `NODE_EXTRA_CA_CERTS` env var
+- Uninstall: reverse all of the above
+- **Files**: `src/tls/trust-store.ts`
+
+#### 1.3 CONNECT Proxy Server
+- HTTP proxy server handling CONNECT method
+- For non-LLM hosts: blind TCP tunnel (zero overhead, no TLS termination)
+- For LLM hosts: TLS intercept with generated cert
+- Known LLM hosts list: `api.anthropic.com`, `api.openai.com`, `generativelanguage.googleapis.com`, `openrouter.ai`, `localhost:11434` (Ollama)
+- Configurable host patterns for custom providers
+- **Files**: `src/proxy/connect-proxy.ts`, `src/proxy/tunnel.ts`
+
+#### 1.4 TLS Interceptor
+- Terminate incoming TLS with per-host cert (signed by local CA)
+- Parse HTTP request inside tunnel
+- Apply context optimization
+- Establish new TLS connection to real provider
+- Stream response back through tunnel
+- **Files**: `src/proxy/tls-interceptor.ts`
+
+#### 1.5 Traffic Classifier
+- Hostname → provider mapping (exact match + patterns)
+- Path-based detection (`/v1/messages`, `/v1/chat/completions`, `/api/chat`)
+- Configurable allowlist/blocklist
+- **Files**: `src/proxy/classifier.ts`
+
+#### Phase 1 Tests
+- [ ] Non-LLM HTTPS traffic passes through untouched
+- [ ] LLM traffic is intercepted and optimized
+- [ ] Cert generated for api.anthropic.com on first request
+- [ ] Streaming works through MITM tunnel
+- [ ] Performance: <2ms overhead for non-LLM tunnel
+- [ ] Ollama local traffic intercepted (HTTP, no TLS needed)
+
+---
+
+### Phase 2: System Integration (Auto-Configuration)
+
+**Goal**: One command configures the OS to route traffic through SmartContext.
+
+#### 2.1 macOS System Proxy Configuration
+- `networksetup -setsecurewebproxy <interface> 127.0.0.1 4800`
+- `networksetup -setwebproxy <interface> 127.0.0.1 4800`
+- Auto-detect active network interface (Wi-Fi, Ethernet)
+- PAC file for selective proxying (only LLM hosts through proxy)
+- Bypass list for local traffic (localhost except Ollama port)
+- **Files**: `src/system/macos.ts`
+
+#### 2.2 Linux System Proxy Configuration
+- Set `http_proxy`/`https_proxy` env vars system-wide
+- GNOME: `gsettings set org.gnome.system.proxy`
+- Alternatively: iptables REDIRECT rules for specific IPs
+- **Files**: `src/system/linux.ts`
+
+#### 2.3 Install / Uninstall Commands
+- `smartcontext-proxy install`:
+  1. Generate CA cert
+  2. Install CA in trust store (requires sudo)
+  3. Configure system proxy
+  4. Start daemon
+  5. Install system service (auto-start on boot)
+- `smartcontext-proxy uninstall`:
+  1. Stop daemon
+  2. Remove system proxy config
+  3. Remove CA from trust store
+  4. Remove system service
+  5. Optionally delete data (`--purge`)
+- Atomic: if any step fails, rollback all previous
+- **Files**: `src/system/installer.ts`
+
+#### 2.4 PAC File Server
+- Serve proxy auto-config at `http://localhost:4800/proxy.pac`
+- Routes only LLM hostnames through proxy
+- Everything else DIRECT
+- Auto-update when provider list changes
+- **Files**: `src/system/pac.ts`
+
+#### Phase 2 Tests
+- [ ] `install` configures system proxy on macOS
+- [ ] `uninstall` cleanly removes everything
+- [ ] PAC file correctly routes LLM traffic only
+- [ ] Non-LLM browsing unaffected after install
+- [ ] Works on both Wi-Fi and Ethernet
+
+---
+
+### Phase 3: System Tray Application
+
+**Goal**: Native system tray icon with real-time status and quick controls.
+
+#### 3.1 Tray Technology
+- Use `@nicolo-ribaudo/trayicon` or custom native bridge
+- macOS: NSStatusItem via Objective-C bridge (or Swift)
+- Fallback: Electron-free `menubar`-style with native popup
+- Alternative: spawn a tiny Swift binary for tray (compiled at install time)
+- **Decision needed**: pure Node.js tray lib vs native Swift helper
+
+#### 3.2 Tray Icon States
+- 🟢 Green: running, optimizing
+- 🟡 Yellow: paused (pass-through mode)
+- 🔴 Red: error / proxy down
+- Icon shows miniature savings counter
+
+#### 3.3 Tray Menu
+```
+SmartContext Proxy          ● Running
+─────────────────────────────
+Today: 142 requests, $63 saved
+Savings: 68% avg
+─────────────────────────────
+Open Dashboard          ⌘D
+─────────────────────────────
+☑ Optimize Context
+☐ A/B Test Mode
+☐ Debug Headers
+─────────────────────────────
+Pause Optimization
+─────────────────────────────
+Settings...
+Logs...
+─────────────────────────────
+Quit SmartContext        ⌘Q
+```
+
+#### 3.4 Tray Click → Dashboard
+- Left click: open `http://localhost:4800` in default browser
+- Right click: show menu above
+- Tooltip on hover: "SmartContext: 142 req, $63 saved, 68% avg"
+
+#### 3.5 Notifications
+- macOS native notifications for:
+  - "SmartContext installed successfully"
+  - "Optimization paused"
+  - "⚠️ Quality drop detected (A/B test)"
+  - Daily summary: "Saved $X today (Y requests)"
+
+#### 3.6 Native macOS Tray (Swift helper)
+- Tiny Swift binary (~50KB) for NSStatusItem
+- Communicates with Node.js proxy via localhost HTTP API
+- Polls `/_sc/status` every 2s for icon updates
+- Compiled at `npm install` time via `swift build`
+- **Files**: `native/macos/SmartContextTray/`, `src/tray/bridge.ts`
+
+#### Phase 3 Tests
+- [ ] Tray icon appears on macOS
+- [ ] Icon color changes with state
+- [ ] Menu items trigger correct API calls
+- [ ] Click opens dashboard
+- [ ] Notifications fire on key events
+
+---
+
+### Phase 4: Full Dashboard Upgrade
+
+**Goal**: Production-grade dashboard matching SPEC.md screens.
+
+#### 4.1 Dashboard Screens (from SPEC)
+- **Home/Status**: savings cards, 7-day chart, provider status, performance
+- **Live Feed**: WebSocket real-time stream, expandable rows with retrieval details
+- **Sessions**: per-session breakdown, exchange viewer, export JSON
+- **Savings Report**: monthly breakdown, by-provider, by-model, projections
+- **Settings**: context tuning, provider management, logging, toggles
+- **A/B Test Results**: comparison table, quality match rates, diff inspector
+
+#### 4.2 WebSocket Live Feed
+- `ws://localhost:4800/_sc/ws` — real-time request stream
+- Push every request metric as it completes
+- Dashboard auto-updates without polling
+- **Files**: `src/ui/ws-feed.ts`
+
+#### 4.3 Charts
+- Lightweight `<canvas>` drawing (no chart library)
+- 7-day savings trend (bar chart)
+- Daily request volume (line chart)
+- Per-provider pie chart
+- **Files**: `src/ui/charts.ts`
+
+#### 4.4 Settings UI
+- Edit all context config from browser
+- Toggle pause/resume per provider
+- Manage LLM host patterns (add custom providers)
+- Debug mode toggle
+- Changes write to `~/.smartcontext/config.json`
+
+#### Phase 4 Tests
+- [ ] All 6 dashboard screens render
+- [ ] WebSocket feed updates in real-time
+- [ ] Charts display correct data
+- [ ] Settings changes persist
+- [ ] Dashboard loads in <200ms
+
+---
+
+### Phase 5: A/B Test Mode + LLM Diagnostics
+
+**Goal**: Quality validation and self-tuning.
+
+#### 5.1 A/B Test Mode
+- Every request sent twice (optimized + original)
+- Compare: semantic similarity, token delta, tool use match
+- Dashboard shows quality match rates
+- `--test-sample N%` for cost control
+
+#### 5.2 LLM-Assisted Diagnostics
+- On quality diff <0.85: diagnostic LLM analyzes what went wrong
+- Auto-tune after 50+ requests
+- `smartcontext-proxy diagnose` CLI command
+- Dashboard "Diagnose" button per request
+
+#### 5.3 Auto-Tuning
+- Analyze score distributions, miss patterns, chunk sizes
+- Propose config changes with reasoning
+- User approves/rejects from dashboard
+
+---
+
+### Phase 6: Production Hardening
+
+#### 6.1 Upstream Proxy Support
+- Detect and chain through corporate proxies
+- `HTTP_PROXY`/`HTTPS_PROXY` awareness
+- Proxy authentication (Basic, NTLM)
+
+#### 6.2 Cert Pinning Fallback
+- Detect cert pinning failures (TLS handshake error)
+- Auto-fallback to tunnel mode for pinned hosts
+- Log which apps use pinning
+
+#### 6.3 Performance
+- Connection pooling to providers
+- Cert caching (no regeneration per request)
+- Non-LLM tunnel: zero-copy forwarding
+- Benchmark: <5ms p95 overhead for LLM, <1ms for tunnel
+
+#### 6.4 Security
+- CA private key encrypted at rest (AES-256, machine-derived key)
+- No plaintext API keys (env var references)
+- Localhost-only binding
+- Auto-lock if proxy crashes (remove system proxy config)
+
+#### 6.5 Error Recovery
+- Crash guard: if proxy dies, system proxy auto-removed (watchdog)
+- Graceful degradation: if optimization fails, tunnel through
+- Health monitor: auto-restart on failure
+
+---
+
+## Tech Stack
+
+| Component | Technology |
+|-----------|-----------|
+| Proxy core | Node.js, raw `http`/`https`/`tls`/`net` modules |
+| Cert generation | `node-forge` (pure JS) |
+| Vector storage | LanceDB embedded |
+| Embeddings | Ollama (GPU) / ONNX (CPU fallback) |
+| Dashboard | Vanilla HTML/CSS/JS, WebSocket |
+| System tray (macOS) | Swift native binary (~50KB) |
+| System tray (Linux) | libappindicator / StatusNotifierItem |
+| System config | `networksetup` (macOS), `gsettings`/env (Linux) |
+| Token counting | `tiktoken` |
+
+## Dependencies (new)
+
+- `node-forge` — TLS cert generation (pure JS)
+- `ws` — WebSocket server for live feed
+
+## File Structure (v2)
+
+```
+smartcontext-proxy/
+├── src/
+│   ├── index.ts              # CLI + entry point
+│   ├── proxy/
+│   │   ├── connect-proxy.ts  # HTTP CONNECT proxy server
+│   │   ├── tunnel.ts         # Blind TCP tunnel (non-LLM)
+│   │   ├── tls-interceptor.ts # TLS MITM for LLM traffic
+│   │   ├── classifier.ts     # LLM vs non-LLM traffic detection
+│   │   ├── server.ts         # (existing) request handling
+│   │   ├── router.ts         # (existing) provider routing
+│   │   └── stream.ts         # (existing) SSE pass-through
+│   ├── tls/
+│   │   ├── ca-manager.ts     # Root CA generation + per-host certs
+│   │   └── trust-store.ts    # OS trust store install/uninstall
+│   ├── system/
+│   │   ├── installer.ts      # install/uninstall orchestrator
+│   │   ├── macos.ts          # macOS proxy + keychain config
+│   │   ├── linux.ts          # Linux proxy config
+│   │   └── pac.ts            # PAC file generator/server
+│   ├── tray/
+│   │   └── bridge.ts         # Node.js ↔ native tray communication
+│   ├── providers/            # (existing)
+│   ├── context/              # (existing)
+│   ├── embedding/            # (existing)
+│   ├── storage/              # (existing)
+│   ├── metrics/              # (existing)
+│   ├── ui/
+│   │   ├── dashboard.ts      # (upgrade) full dashboard
+│   │   ├── ws-feed.ts        # WebSocket live feed
+│   │   └── charts.ts         # Canvas chart rendering
+│   ├── daemon/               # (existing)
+│   ├── config/               # (existing)
+│   └── test/
+├── native/
+│   └── macos/
+│       └── SmartContextTray/ # Swift tray app
+├── adapters/openclaw/        # (existing)
+├── package.json
+└── README.md
+```
+
+## Execution Order
+
+1. **Phase 1** (HTTPS CONNECT + MITM) — foundational, everything depends on this
+2. **Phase 2** (System integration) — makes it actually transparent
+3. **Phase 3** (System tray) — user control from minute one
+4. **Phase 4** (Dashboard upgrade) — full observability
+5. **Phase 5** (A/B + diagnostics) — quality assurance
+6. **Phase 6** (Hardening) — production readiness
+
+Phases 3+4 can run in parallel after Phase 2.

package/dist/src/context/ab-test.d.ts

@@ -0,0 +1,32 @@
+export interface ABComparison {
+    requestId: number;
+    timestamp: number;
+    provider: string;
+    model: string;
+    optimizedTokens: number;
+    originalTokens: number;
+    savingsPercent: number;
+    semanticSimilarity: number;
+    qualityMatch: boolean;
+    responseA: string;
+    responseB: string;
+    latencyA: number;
+    latencyB: number;
+}
+export declare function enableABTest(sampleRate?: number): void;
+export declare function disableABTest(): void;
+export declare function isABEnabled(): boolean;
+export declare function shouldABTest(): boolean;
+export declare function getABResults(limit?: number): ABComparison[];
+export declare function getABSummary(): {
+    total: number;
+    qualityMatch: number;
+    minorDiff: number;
+    significantDiff: number;
+    avgSavings: number;
+};
+/**
+ * Send the original (unoptimized) request to the provider for comparison.
+ * This is Path B — the result is stored but NOT returned to the client.
+ */
+export declare function sendOriginalForComparison(originalBody: Buffer, hostname: string, port: number, path: string, headers: Record<string, string>, requestId: number, provider: string, model: string, optimizedResponse: string, optimizedTokens: number, originalTokens: number, savingsPercent: number): Promise<void>;

package/dist/src/context/ab-test.js

@@ -0,0 +1,133 @@
+"use strict";
+var __importDefault = (this && this.__importDefault) || function (mod) {
+    return (mod && mod.__esModule) ? mod : { "default": mod };
+};
+Object.defineProperty(exports, "__esModule", { value: true });
+exports.enableABTest = enableABTest;
+exports.disableABTest = disableABTest;
+exports.isABEnabled = isABEnabled;
+exports.shouldABTest = shouldABTest;
+exports.getABResults = getABResults;
+exports.getABSummary = getABSummary;
+exports.sendOriginalForComparison = sendOriginalForComparison;
+const node_http_1 = __importDefault(require("node:http"));
+const node_https_1 = __importDefault(require("node:https"));
+let abResults = [];
+let abEnabled = false;
+let abSampleRate = 100; // percent
+function enableABTest(sampleRate = 100) {
+    abEnabled = true;
+    abSampleRate = Math.min(100, Math.max(1, sampleRate));
+}
+function disableABTest() {
+    abEnabled = false;
+}
+function isABEnabled() {
+    return abEnabled;
+}
+function shouldABTest() {
+    if (!abEnabled)
+        return false;
+    return Math.random() * 100 < abSampleRate;
+}
+function getABResults(limit = 50) {
+    return abResults.slice(-limit);
+}
+function getABSummary() {
+    const total = abResults.length;
+    if (total === 0)
+        return { total: 0, qualityMatch: 0, minorDiff: 0, significantDiff: 0, avgSavings: 0 };
+    let qualityMatch = 0;
+    let minorDiff = 0;
+    let significantDiff = 0;
+    let totalSavings = 0;
+    for (const r of abResults) {
+        if (r.semanticSimilarity >= 0.95)
+            qualityMatch++;
+        else if (r.semanticSimilarity >= 0.85)
+            minorDiff++;
+        else
+            significantDiff++;
+        totalSavings += r.savingsPercent;
+    }
+    return {
+        total,
+        qualityMatch,
+        minorDiff,
+        significantDiff,
+        avgSavings: Math.round(totalSavings / total),
+    };
+}
+/**
+ * Send the original (unoptimized) request to the provider for comparison.
+ * This is Path B — the result is stored but NOT returned to the client.
+ */
+async function sendOriginalForComparison(originalBody, hostname, port, path, headers, requestId, provider, model, optimizedResponse, optimizedTokens, originalTokens, savingsPercent) {
+    const startTime = Date.now();
+    try {
+        const useHTTPS = port === 443 || hostname.includes('.');
+        const transport = useHTTPS ? node_https_1.default : node_http_1.default;
+        const url = `${useHTTPS ? 'https' : 'http'}://${hostname}:${port}${path}`;
+        const forwardHeaders = { ...headers };
+        forwardHeaders['content-length'] = String(originalBody.length);
+        const responseB = await new Promise((resolve, reject) => {
+            const req = transport.request(url, { method: 'POST', headers: forwardHeaders }, (res) => {
+                let data = '';
+                res.on('data', (chunk) => (data += chunk));
+                res.on('end', () => resolve(data));
+            });
+            req.on('error', reject);
+            req.write(originalBody);
+            req.end();
+        });
+        const latencyB = Date.now() - startTime;
+        // Simple text similarity (cosine of character trigrams)
+        const similarity = textSimilarity(optimizedResponse, responseB);
+        abResults.push({
+            requestId,
+            timestamp: Date.now(),
+            provider,
+            model,
+            optimizedTokens,
+            originalTokens,
+            savingsPercent,
+            semanticSimilarity: similarity,
+            qualityMatch: similarity >= 0.95,
+            responseA: optimizedResponse.slice(0, 500),
+            responseB: responseB.slice(0, 500),
+            latencyA: 0, // filled externally
+            latencyB,
+        });
+        // Keep max 1000 results
+        if (abResults.length > 1000) {
+            abResults = abResults.slice(-500);
+        }
+    }
+    catch {
+        // A/B test failure is non-critical
+    }
+}
+/** Simple text similarity using character trigrams */
+function textSimilarity(a, b) {
+    if (a === b)
+        return 1.0;
+    if (!a || !b)
+        return 0;
+    const trigramsA = getTrigrams(a.toLowerCase());
+    const trigramsB = getTrigrams(b.toLowerCase());
+    let intersection = 0;
+    for (const t of trigramsA) {
+        if (trigramsB.has(t))
+            intersection++;
+    }
+    const union = trigramsA.size + trigramsB.size - intersection;
+    return union === 0 ? 0 : intersection / union;
+}
+function getTrigrams(s) {
+    const set = new Set();
+    for (let i = 0; i < s.length - 2; i++) {
+        set.add(s.substring(i, i + 3));
+    }
+    return set;
+}
+//# sourceMappingURL=ab-test.js.map