agent-cache-optimizer 0.1.1

package/docs/cross-cli.md

@@ -0,0 +1,89 @@
+# KV Cache Optimization — Cross-CLI Architecture
+
+## Core Principle (CLI-agnostic)
+
+All LLM CLI agents assemble a system prompt from:
+1. **Stable blocks**: CLAUDE.md / AGENTS.md / rules / tool definitions / agent configs
+2. **Semi-dynamic blocks**: currentDate (changes daily)
+3. **Dynamic blocks**: session handoff / memory injections / conversation history
+
+Prefix-match KV caches (DeepSeek, Anthropic, Google, OpenAI) reuse computed
+states when the prompt PREFIX is byte-identical to a cached request. Dynamic
+content at the front busts everything.
+
+## Architecture
+
+```
+┌──────────────────────────────────────────────────────┐
+│                 agent-cache-optimizer-core                  │
+│                                                       │
+│  hash(track) → stability score → reorder             │
+│  Fully content-agnostic, zero external deps           │
+│  Input: string[] (system blocks)                      │
+│  Output: string[] (reordered blocks)                  │
+│  State: per-agent stability.json                      │
+└──────────────────────────────────────────────────────┘
+         │                │                │
+    ┌────▼─────┐    ┌─────▼──────┐    ┌────▼──────┐
+    │ OpenCode  │    │Claude Code │    │   Codex    │
+    │  adapter  │    │  adapter   │    │  adapter   │
+    │           │    │            │    │            │
+    │ plugin.ts │    │ hook.js    │    │ plugin.py  │
+    │ system    │    │ pre-launch │    │ (TBD)      │
+    │ .transform│    │ validator  │    │            │
+    └───────────┘    └────────────┘    └────────────┘
+```
+
+## Per-CLI Strategy
+
+### OpenCode (✅ implemented)
+
+- Hook: `experimental.chat.system.transform` — direct system prompt access
+- Fallback: `chat.params` + `chat.headers` for diagnostics
+- Adapter: `plugins/agent-cache-optimizer.ts`
+
+### Claude Code (see [adapters/claude-code.md](../adapters/claude-code.md))
+
+- No direct prompt-transform hook available
+- Anthropic has native prompt caching (automatic prefix reuse)
+- Strategy: **pre-session validator** + CLAUDE.md structure optimization
+- Check: CLAUDE.md for date stamps, session IDs, dynamic includes
+- Check: `.claude/settings.json` hooks for cache-busting patterns
+- Wrap `claude` invocation to compare cache metrics before/after
+
+### Codex (OpenAI)
+
+- Plugin API likely similar to OpenCode (both from AI SDK ecosystem)
+- Strategy: adapt OpenCode plugin once Codex plugin API is confirmed
+- OpenAI prompt caching uses similar prefix-match mechanism
+
+### Gemini CLI
+
+- Uses `activate_skill` + GEMINI.md for project context
+- Google context caching API has explicit cache tokens
+- Strategy: inject cache_control markers at stable block boundaries
+
+## Shared Core Module
+
+`agent-cache-optimizer-core.ts`:
+
+```typescript
+// Pure functions, no CLI-specific dependencies
+export function hashBlock(content: string): string
+export function coldStartScore(block: string, index: number, total: number): number
+export function classifyBlocks(blocks: string[], db: StabilityDB): Classified
+export function updateStabilityDB(db: StabilityDB, blocks: string[]): StabilityDB
+```
+
+Each adapter imports these and adds CLI-specific glue:
+1. Extract blocks from CLI's prompt representation
+2. Call `classifyBlocks()`
+3. Inject reordered blocks back into CLI's prompt
+
+## Roadmap
+
+1. [x] OpenCode plugin (V3: per-agent + block splitting + diagnostics)
+2. [ ] Extract shared core to `agent-cache-optimizer-core.ts`
+3. [ ] Claude Code pre-session validator
+4. [ ] Codex adapter (once API confirmed)
+5. [ ] Gemini CLI adapter (Google context caching)

package/docs/upstream.md

@@ -0,0 +1,65 @@
+# KV Cache Optimization for DeepSeek — Upstream Changes Needed
+
+## Problem
+
+OpenCode's system prompt assembly places **dynamic blocks** (SessionStart HANDOFF,
+.remember/ files, memory-dream injections, currentDate) at the **front** of the
+system prompt array. This means the very first bytes of every session's system
+prompt are different, which completely busts prefix-match KV caches (DeepSeek,
+Anthropic prompt caching, Google context caching).
+
+## Current prompt assembly order (observed)
+
+```
+[0] SessionStart HANDOFF + REMEMBER + MEMORY  ← ⚡ changes every session
+[1] CLAUDE.md                                  ← ✅ stable
+[2] Agent definitions                          ← ✅ stable
+[3] MCP server instructions                    ← ✅ stable  
+[4] Skills list                                ← ✅ stable
+[5] currentDate                                ← ⚠️ changes daily
+[6] Tool definitions                           ← ✅ stable
+```
+
+## Impact
+
+- **Every new session: 100% cache miss** on system prompt
+- **Every subagent call: fresh KV computation** for the full system prompt
+- **Estimated waste**: 40-60% of system prompt tokens recomputed unnecessarily
+- For DeepSeek where cache hit cost is near-zero, this is purely wasted compute
+
+## Fix (in OpenCode core)
+
+Move dynamic blocks to the **end** of the system prompt array:
+
+```
+[0] CLAUDE.md                                  ← ✅ stable
+[1] Agent definitions                          ← ✅ stable
+[2] MCP server instructions                    ← ✅ stable
+[3] Skills list                                ← ✅ stable
+[4] Tool definitions                           ← ✅ stable
+[5] currentDate                                ← ⚠️ changes daily
+[6] SessionStart HANDOFF + REMEMBER + MEMORY  ← ⚡ changes per session
+```
+
+This way, the KV cache prefix (blocks [0]-[4]) stays identical across sessions
+and subagent calls, giving 70-80% cache reuse.
+
+## Implementation in OpenCode
+
+In the prompt assembly code (likely in the system prompt builder), change:
+
+1. **Build stable prefix first**: all static configuration (CLAUDE.md, agent
+   definitions, MCP/Skills/tools lists)
+2. **Append semi-dynamic content**: currentDate
+3. **Append fully-dynamic content last**: SessionStart hook output,
+   .remember/ injection, memory-dream retrieval results
+
+## Interim workaround
+
+A plugin (`agent-cache-optimizer.ts`) uses `experimental.chat.system.transform` to
+reorder blocks at runtime. This works but:
+- Depends on an experimental hook
+- Adds a small processing overhead
+- Can't reorder content WITHIN blocks, only BETWEEN blocks
+
+The proper fix is in the core prompt builder.

package/package.json

@@ -0,0 +1,70 @@
+{
+  "name": "agent-cache-optimizer",
+  "version": "0.1.1",
+  "description": "Content-agnostic KV cache optimizer for LLM CLI agents — boosts prompt cache hit rate by 40-88% through automatic stability tracking and block reordering",
+  "keywords": [
+    "kv-cache",
+    "prompt-caching",
+    "llm",
+    "deepseek",
+    "anthropic",
+    "openai",
+    "opencode",
+    "claude-code",
+    "optimization",
+    "token-savings"
+  ],
+  "license": "MIT",
+  "author": {
+    "name": "Chris",
+    "url": "https://github.com/chris"
+  },
+  "repository": {
+    "type": "git",
+    "url": "https://github.com/uuie/agent-cache-optimizer"
+  },
+  "type": "module",
+  "main": "./src/index.ts",
+  "exports": {
+    ".": "./src/index.ts",
+    "./core": "./src/core.ts",
+    "./heuristics": "./src/heuristics.ts",
+    "./splitting": "./src/splitting.ts"
+  },
+  "files": [
+    "src/",
+    "adapters/",
+    "scripts/",
+    "skills/",
+    "docs/",
+    "README.md",
+    "README.zh-CN.md",
+    "LICENSE",
+    "CHANGELOG.md",
+    "tsconfig.json"
+  ],
+  "scripts": {
+    "test": "vitest run",
+    "test:watch": "vitest",
+    "typecheck": "tsc --noEmit",
+    "format": "prettier --write \"src/**/*.ts\" \"*.md\"",
+    "format:check": "prettier --check \"src/**/*.ts\" \"*.md\"",
+    "check": "bash scripts/cache-status.sh",
+    "audit": "bash scripts/check-cache-friendly.sh --all"
+  },
+  "devDependencies": {
+    "@opencode-ai/plugin": "^1.15.0",
+    "@types/node": "^26.0.0",
+    "prettier": "^3.3.0",
+    "typescript": "^6.0.3",
+    "vitest": "^2.1.0"
+  },
+  "peerDependencies": {
+    "@opencode-ai/plugin": "^1.15.0"
+  },
+  "peerDependenciesMeta": {
+    "@opencode-ai/plugin": {
+      "optional": true
+    }
+  }
+}

package/scripts/cache-status.sh

@@ -0,0 +1,170 @@
+#!/usr/bin/env bash
+# cache-status.sh — display KV Cache Optimizer status
+# Usage: cache-status.sh [--json]
+set -euo pipefail
+
+CACHE_DIR="${HOME}/.cache/opencode/agent-cache-optimizer"
+BOLD='\033[1m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; RED='\033[0;31m'; CYAN='\033[0;36m'; NC='\033[0m'
+
+# ── Check if plugin has run ────────────────────────────────────────
+
+if [[ ! -d "$CACHE_DIR" ]]; then
+  echo "╔══════════════════════════════════════════════════════════════╗"
+  echo "║              KV Cache Optimizer Status                       ║"
+  echo "╠══════════════════════════════════════════════════════════════╣"
+  echo "║ Status:  NO DATA                                             ║"
+  echo "║                                                               ║"
+  echo "║ Plugin has not been invoked yet.                              ║"
+  echo "║                                                               ║"
+  echo "║ 1. Verify agent-cache-optimizer.ts is in opencode.json plugins      ║"
+  echo "║ 2. Restart OpenCode                                           ║"
+  echo "║ 3. Make at least one request to trigger the hook              ║"
+  echo "╚══════════════════════════════════════════════════════════════╝"
+  exit 0
+fi
+
+# ── Parse stability DBs ────────────────────────────────────────────
+
+declare -A agent_obs agent_positions agent_stable_pct agent_mode
+
+for db in "$CACHE_DIR"/stability-*.json; do
+  [[ -f "$db" ]] || continue
+  agent=$(basename "$db" .json | sed 's/^stability-//')
+
+  obs=$(python3 -c "
+import json
+d=json.load(open('$db'))
+print(d.get('observations', 0))
+" 2>/dev/null || echo 0)
+
+  pos_count=$(python3 -c "
+import json
+d=json.load(open('$db'))
+print(len(d.get('positions', {})))
+" 2>/dev/null || echo 0)
+
+  # Average score across all known hashes
+  avg_score=$(python3 -c "
+import json
+d=json.load(open('$db'))
+scores = list(d.get('scores', {}).values())
+if scores:
+    stable = sum(1 for s in scores if s >= 0.7)
+    dynamic = sum(1 for s in scores if s <= 0.3)
+    print(f'{stable}/{len(scores)}')
+else:
+    print('0/0')
+" 2>/dev/null || echo "0/0")
+
+  agent_obs[$agent]=$obs
+  agent_positions[$agent]=$pos_count
+  agent_stable_pct[$agent]=$avg_score
+
+  if [[ $obs -ge 3 ]]; then
+    agent_mode[$agent]="WARM"
+  else
+    agent_mode[$agent]="COLD"
+  fi
+done
+
+# ── Parse diag log ─────────────────────────────────────────────────
+
+diag_entries=0
+last_entry=""
+if [[ -f "$CACHE_DIR/diag.log" ]]; then
+  diag_entries=$(wc -l < "$CACHE_DIR/diag.log" | tr -d ' ')
+  last_entry=$(tail -1 "$CACHE_DIR/diag.log" 2>/dev/null || echo "(empty)")
+fi
+
+# ── Display ────────────────────────────────────────────────────────
+
+echo ""
+echo -e "${BOLD}╔══════════════════════════════════════════════════════════════╗${NC}"
+echo -e "${BOLD}║              KV Cache Optimizer Status                       ║${NC}"
+echo -e "${BOLD}╠══════════════════════════════════════════════════════════════╣${NC}"
+
+# Status line
+if [[ $diag_entries -gt 0 ]]; then
+  echo -e "║ ${GREEN}Status:  ACTIVE${NC}                                               ║"
+else
+  echo -e "║ ${RED}Status:  NO ACTIVITY (check plugin loading)${NC}                    ║"
+fi
+
+# Mode: cold/warm per agent
+modes=""
+for agent in "${!agent_obs[@]}"; do
+  modes+="$agent=${agent_mode[$agent]} "
+done
+printf "║ Mode:    %-52s ║\n" "$modes"
+
+# Uptime from diag.log first/last
+if [[ $diag_entries -gt 0 ]]; then
+  first_ts=$(head -1 "$CACHE_DIR/diag.log" | grep -oP '^\[[^\]]+\]' | tr -d '[]' || echo "?")
+  last_ts=$(tail -1 "$CACHE_DIR/diag.log" | grep -oP '^\[[^\]]+\]' | tr -d '[]' || echo "?")
+  printf "║ Uptime:  %-52s ║\n" "$first_ts → $last_ts"
+fi
+
+echo -e "${BOLD}╠══════════════════════════════════════════════════════════════╣${NC}"
+
+# Per-agent breakdown
+if [[ ${#agent_obs[@]} -eq 0 ]]; then
+  echo -e "║ ${YELLOW}No per-agent data yet — make requests with different agents${NC}     ║"
+else
+  printf "║ ${CYAN}%-20s %6s %8s %10s${NC}  ║\n" "Agent" "Obs" "Positions" "Stable"
+  for agent in $(printf '%s\n' "${!agent_obs[@]}" | sort); do
+    printf "║ %-20s %6s %8s %10s  ║\n" \
+      "$agent" "${agent_obs[$agent]}" "${agent_positions[$agent]}" "${agent_stable_pct[$agent]}"
+  done
+fi
+
+echo -e "${BOLD}╠══════════════════════════════════════════════════════════════╣${NC}"
+
+# Last reorder entries
+if [[ $diag_entries -gt 0 ]]; then
+  echo -e "║ ${CYAN}Last reorders (diag.log):${NC}                                    ║"
+  tail -5 "$CACHE_DIR/diag.log" 2>/dev/null | while IFS= read -r line; do
+    # Truncate to fit in box
+    printf "║   %-56s ║\n" "${line:0:56}"
+  done
+else
+  echo -e "║ ${YELLOW}No reorder events logged yet${NC}                                  ║"
+fi
+
+echo -e "${BOLD}╠══════════════════════════════════════════════════════════════╣${NC}"
+
+# Estimated savings
+total_obs=0
+for o in "${agent_obs[@]}"; do total_obs=$((total_obs + o)); done
+if [[ $total_obs -gt 0 ]]; then
+  total_stable=0
+  total_blocks=0
+  for agent in "${!agent_stable_pct[@]}"; do
+    s=$(echo "${agent_stable_pct[$agent]}" | cut -d/ -f1)
+    t=$(echo "${agent_stable_pct[$agent]}" | cut -d/ -f2)
+    total_stable=$((total_stable + s))
+    total_blocks=$((total_blocks + t))
+  done
+  if [[ $total_blocks -gt 0 ]]; then
+    pct=$((total_stable * 100 / total_blocks))
+    printf "║ Estimated cache reuse: %-36s ║\n" "~${pct}% of system prompt"
+  fi
+fi
+
+echo -e "${BOLD}╚══════════════════════════════════════════════════════════════╝${NC}"
+echo ""
+
+# ── Quick stats ────────────────────────────────────────────────────
+
+echo "Files:"
+echo "  $(ls "$CACHE_DIR"/stability-*.json 2>/dev/null | wc -l) stability DBs"
+echo "  ${diag_entries} diagnostic log entries"
+echo ""
+
+if [[ $diag_entries -eq 0 && ${#agent_obs[@]} -eq 0 ]]; then
+  echo "⚠️  Plugin hasn't fired yet. Troubleshooting:"
+  echo "  1. grep 'agent-cache-optimizer' ~/.config/opencode/opencode.json"
+  echo "  2. Restart OpenCode"
+  echo "  3. Check: the experimental.chat.system.transform hook"
+  echo "     may not be available in your OpenCode version."
+  echo "     Fallback: chat.params hook writes to diag.log on first call."
+fi

package/scripts/check-cache-friendly.sh

@@ -0,0 +1,122 @@
+#!/usr/bin/env bash
+# check-cache-friendly.sh — scan any CLI agent's config for KV-cache-busting patterns
+# Usage: ./check-cache-friendly.sh [file]        (default: CLAUDE.md)
+#        ./check-cache-friendly.sh --opencode     (scan opencode config)
+#        ./check-cache-friendly.sh --all          (scan all known configs)
+set -euo pipefail
+
+RED='\033[0;31m'; YELLOW='\033[1;33m'; GREEN='\033[0;32m'; NC='\033[0m'
+
+warn()  { echo -e "${YELLOW}⚠️  $1${NC}"; }
+good()  { echo -e "${GREEN}✅ $1${NC}"; }
+bad()   { echo -e "${RED}❌ $1${NC}"; }
+
+check_file() {
+  local file="$1"
+  local label="${2:-$file}"
+  if [[ ! -f "$file" ]]; then
+    echo "   (file not found: $file)"
+    return
+  fi
+
+  echo ""
+  echo "─── $label ───"
+  local issues=0
+
+  # 1. Date stamps in first 10 lines
+  if head -10 "$file" | grep -qP '\d{4}-\d{2}-\d{2}'; then
+    warn "Date stamp in first 10 lines → move to file end for cache stability"
+    ((issues++))
+  fi
+
+  # 2. Session IDs
+  if grep -qP 'ses_[a-z0-9]{8,}' "$file" 2>/dev/null; then
+    warn "Session ID patterns found → these change every session"
+    ((issues++))
+  fi
+
+  # 3. Dynamic file references (but NOT npm package names)
+  if grep -qP '@remember\b|\.remember/|memory-dream(?!@)' "$file" 2>/dev/null; then
+    warn "Dynamic file/memory references → content changes between sessions"
+    ((issues++))
+  fi
+
+  # 4. Very short first line (often a date or status) — text files only
+  #    Skip JSON/YAML config files and markdown section headers (## / #)
+  local first_line
+  first_line=$(head -1 "$file")
+  if [[ ! "$file" =~ \.(json|yaml|yml|toml|lock)$ ]]; then
+    if [[ ${#first_line} -lt 30 && ! "$first_line" =~ ^#+\  ]]; then
+      warn "Very short first line (${#first_line} chars): '$first_line'"
+      ((issues++))
+    fi
+  fi
+
+  # 5. File volatility check
+  local size lines mtime
+  size=$(wc -c < "$file" | tr -d ' ')
+  lines=$(wc -l < "$file" | tr -d ' ')
+  mtime=$(stat -c %Y "$file" 2>/dev/null || echo 0)
+  local now
+  now=$(date +%s)
+  local age_hrs=$(( (now - mtime) / 3600 ))
+
+  echo "   Size: ${size}B, Lines: ${lines}, Age: ${age_hrs}h"
+
+  if [[ $issues -eq 0 ]]; then
+    good "No cache-busting patterns detected"
+  fi
+}
+
+check_opencode() {
+  local config_dir="${HOME}/.config/opencode"
+  echo ""
+  echo "══════════ OpenCode Config ══════════"
+
+  # Main config
+  check_file "$config_dir/opencode.json" "opencode.json"
+
+  # Append files
+  for f in "$config_dir"/oh-my-opencode-slim/*_append.md; do
+    [[ -f "$f" ]] || continue
+    check_file "$f" "$(basename "$f")"
+  done
+
+  # .remember files
+  for f in "$config_dir"/.remember/*.md; do
+    [[ -f "$f" ]] || continue
+    local name; name=$(basename "$f")
+    local lines; lines=$(wc -l < "$f" | tr -d ' ')
+    local size; size=$(wc -c < "$f" | tr -d ' ')
+    echo "   .remember/$name: ${lines} lines, ${size}B"
+  done
+}
+
+check_claude() {
+  echo "══════════ Claude Code Config ══════════"
+  check_file "${HOME}/.claude/CLAUDE.md" "CLAUDE.md (global)"
+  for f in "${HOME}"/.claude/rules/*.md; do
+    [[ -f "$f" ]] || continue
+    check_file "$f" "rules/$(basename "$f")"
+  done
+  check_file "$(pwd)/CLAUDE.md" "CLAUDE.md (project)" 2>/dev/null || true
+  check_file "$(pwd)/AGENTS.md" "AGENTS.md (project)" 2>/dev/null || true
+}
+
+# ── Main ───────────────────────────────────────────────────────────
+
+case "${1:-}" in
+  --opencode) check_opencode ;;
+  --claude)   check_claude ;;
+  --all)
+    check_opencode
+    echo ""
+    check_claude
+    ;;
+  *)
+    check_file "${1:-CLAUDE.md}" "${1:-CLAUDE.md}"
+    ;;
+esac
+
+echo ""
+echo "Done."

package/skills/cache-status/SKILL.md

@@ -0,0 +1,81 @@
+---
+name: cache-status
+description: Show KV cache optimizer status — stability DBs, reordering stats, diagnostics. Trigger: /cache-status
+---
+
+# KV Cache Optimizer Status
+
+Show the current state of the KV cache optimizer plugin (agent-cache-optimizer.ts).
+
+## Data Locations
+
+- Stability DBs: `~/.cache/opencode/agent-cache-optimizer/stability-*.json`
+- Diagnostic log: `~/.cache/opencode/agent-cache-optimizer/diag.log`
+
+## Display Format
+
+Read all `stability-*.json` files and `diag.log`, then present:
+
+```
+╔══════════════════════════════════════════════════════════════╗
+║              KV Cache Optimizer Status                       ║
+╠══════════════════════════════════════════════════════════════╣
+║ Status:  ACTIVE / NO DATA / ERROR                            ║
+║ Mode:    COLD START (heuristics) / WARM (hash-based)         ║
+║ Uptime:  first_seen → last_seen                             ║
+╠══════════════════════════════════════════════════════════════╣
+║ Per-Agent Breakdown:                                         ║
+║   orchestrator:  12 obs, 88% cacheable                       ║
+║   oracle:         5 obs, 75% cacheable                       ║
+║   fixer:          3 obs, 90% cacheable                       ║
+║   ...                                                        ║
+╠══════════════════════════════════════════════════════════════╣
+║ Last 5 reorders (from diag.log):                             ║
+║   [time] [agent] S:X U:Y D:Z T:N obs:M                       ║
+║   ...                                                        ║
+╠══════════════════════════════════════════════════════════════╣
+║ Estimated savings: XX KB / session                           ║
+╚══════════════════════════════════════════════════════════════╝
+```
+
+## Key Metrics
+
+From each `stability-{agent}.json`:
+- `observations`: number of calls tracked
+- `positions`: number of distinct block positions
+- Average `scores` per block → classification quality
+- Ratio of stable vs dynamic blocks
+
+From `diag.log`:
+- Last N reorder operations
+- Confirm plugin is being invoked
+
+## Interpretation
+
+- **NO DATA**: Plugin hasn't been invoked yet. Restart OpenCode after adding the plugin to `opencode.json`.
+- **COLD START**: First 2 sessions per agent — using position/size heuristics.
+- **WARM**: 3+ sessions — using hash-based stability tracking. More accurate.
+- **High stable %**: Good cache reuse across sessions.
+- **High unknown %**: Heuristics can't classify some blocks → will improve with more observations.
+
+## If Plugin Not Working
+
+1. Check `opencode.json` has `"file:///home/chris/.config/opencode/plugins/agent-cache-optimizer.ts"` in `plugin` array
+2. Check `diag.log` exists and has entries
+3. If no diag.log: the `experimental.chat.system.transform` hook may not be firing → fall back to `chat.params` diagnostic
+4. Run `opencode debug agent orchestrator` to verify plugin config
+
+## Quick Health Check
+
+```bash
+# Check if plugin is loaded
+ls ~/.cache/opencode/agent-cache-optimizer/
+
+# View last reorder
+tail -5 ~/.cache/opencode/agent-cache-optimizer/diag.log
+
+# Count observations per agent
+for f in ~/.cache/opencode/agent-cache-optimizer/stability-*.json; do
+  echo "$(basename $f .json): $(python3 -c "import json; d=json.load(open('$f')); print(d['observations'])") obs"
+done
+```