ultimate-pi 0.1.0 → 0.1.3

package/vault/wiki/sources/wozcode.md

@@ -0,0 +1,79 @@
+---
+type: source
+source_type: product-documentation
+author: WithWoz Inc.
+date_published: 2026
+url: https://www.wozcode.com/how-it-works
+confidence: medium
+key_claims:
+  - "25-55% token reduction vs vanilla Claude Code"
+  - "Three levers: smarter search, batched edits, quality loop"
+  - "AST truncation stubs function bodies, returns only signatures"
+  - "Haiku subagents handle ~40% of coding work (exploration)"
+  - "100% local architecture, zero cloud code transmission"
+  - "Post-edit syntax validation (TS, JSON/YAML, SQL) catches errors before model retries"
+  - "Fuzzy edit matching tolerates whitespace drift and visually-identical characters"
+  - "Claims measured from live Anthropic API usage fields, not simulations"
+tags:
+  - wozcode
+  - token-reduction
+  - claude-code
+  - agent-architecture
+created: 2026-04-30
+updated: 2026-04-30
+status: ingested
+title: "WOZCODE"
+
+---# WOZCODE
+
+WOZCODE is a Claude Code plugin by WithWoz Inc. that reduces token spend through three compounding levers. Patent-pending. Runs 100% locally — code never leaves the developer's machine.
+
+## Three Core Levers
+
+### 1. Smarter Search (Input Token Reduction)
+- Ranked snippets instead of full-file grep dumps
+- AST truncation: stubs function bodies, returns only signatures
+- Returns only what the model needs for the current task
+
+### 2. Batched & Fuzzy Edits (Output Token Reduction)
+- Multiple file edits merged into a single model call
+- Fuzzy edit matching: tolerates whitespace drift, indentation changes, curly vs straight quotes, em-dashes
+- Near-misses land without retry round-trips
+
+### 3. Quality Loop (Rework Reduction)
+- Post-edit syntax validation: TS compiler, JSON/YAML/HTML parsers, SQL linter
+- SQL dialect auto-fix: rewrites backtick identifiers, unquoted reserved aliases, `COUNT(DISTINCT a, b)`, `date_trunc("month", col)`
+- Better error context: dialect-specific hints, real file content diffs instead of "string not found"
+
+## Agent Architecture
+
+| Agent | Model | Role |
+|-------|-------|------|
+| `woz:code` | User's frontier model (Opus/Sonnet) | Main thread, writes/edits code with full tool access |
+| `woz:explore` | Haiku | Read-only exploration, returns summaries to parent. ~15× cheaper. |
+
+~40% of coding work is exploration — automatically routed to Haiku.
+
+## Privacy Architecture
+
+- **Never sent**: source code, file contents, file paths, grep output, tool inputs/outputs, prompts, model responses, Anthropic API key
+- **In the loop for tool execution, not API transport**: API requests go direct from machine to Anthropic
+- **What WOZCODE servers do**: stats dashboard, auth endpoint, billing (via Stripe)
+- **Aggregated anonymous stats**: session-level token/cost/time metrics only
+
+## Competitor Comparison (from WOZCODE)
+
+| Tool | Approach | WOZCODE's Critique |
+|------|----------|-------------------|
+| SDL-MCP | Graph-based code exploration | Covers only exploration, not editing/validation |
+| Caveman | System-prompt prose compression | Most tokens are in tool calls, not prose |
+| WOZCODE | Full-session coverage | Covers exploration, editing, and validation |
+
+## Claims to Verify
+
+Claims are **medium confidence** (single source, commercial product, patent-pending means details obscured):
+- [ ] 25-55% token savings — benchmarkable on our codebase
+- [ ] 5-10× faster on DB tasks — domain-specific claim
+- [ ] AST truncation effectiveness on dynamic languages
+- [ ] Fuzzy edit matching false-positive rate
+- [ ] Post-edit validation latency overhead

package/.agents/skills/compress/SKILL.md

@@ -1,111 +0,0 @@
----
-name: compress
-description: >
-  Compress natural language memory files (CLAUDE.md, todos, preferences) into caveman format
-  to save input tokens. Preserves all technical substance, code, URLs, and structure.
-  Compressed version overwrites the original file. Human-readable backup saved as FILE.original.md.
-  Trigger: /caveman:compress <filepath> or "compress memory file"
----
-
-# Caveman Compress
-
-## Purpose
-
-Compress natural language files (CLAUDE.md, todos, preferences) into caveman-speak to reduce input tokens. Compressed version overwrites original. Human-readable backup saved as `<filename>.original.md`.
-
-## Trigger
-
-`/caveman:compress <filepath>` or when user asks to compress a memory file.
-
-## Process
-
-1. This SKILL.md lives alongside `scripts/` in the same directory. Find that directory.
-
-2. Run:
-
-cd <directory_containing_this_SKILL.md> && python3 -m scripts <absolute_filepath>
-
-3. The CLI will:
-- detect file type (no tokens)
-- call Claude to compress
-- validate output (no tokens)
-- if errors: cherry-pick fix with Claude (targeted fixes only, no recompression)
-- retry up to 2 times
-- if still failing after 2 retries: report error to user, leave original file untouched
-
-4. Return result to user
-
-## Compression Rules
-
-### Remove
-- Articles: a, an, the
-- Filler: just, really, basically, actually, simply, essentially, generally
-- Pleasantries: "sure", "certainly", "of course", "happy to", "I'd recommend"
-- Hedging: "it might be worth", "you could consider", "it would be good to"
-- Redundant phrasing: "in order to" → "to", "make sure to" → "ensure", "the reason is because" → "because"
-- Connective fluff: "however", "furthermore", "additionally", "in addition"
-
-### Preserve EXACTLY (never modify)
-- Code blocks (fenced ``` and indented)
-- Inline code (`backtick content`)
-- URLs and links (full URLs, markdown links)
-- File paths (`/src/components/...`, `./config.yaml`)
-- Commands (`npm install`, `git commit`, `docker build`)
-- Technical terms (library names, API names, protocols, algorithms)
-- Proper nouns (project names, people, companies)
-- Dates, version numbers, numeric values
-- Environment variables (`$HOME`, `NODE_ENV`)
-
-### Preserve Structure
-- All markdown headings (keep exact heading text, compress body below)
-- Bullet point hierarchy (keep nesting level)
-- Numbered lists (keep numbering)
-- Tables (compress cell text, keep structure)
-- Frontmatter/YAML headers in markdown files
-
-### Compress
-- Use short synonyms: "big" not "extensive", "fix" not "implement a solution for", "use" not "utilize"
-- Fragments OK: "Run tests before commit" not "You should always run tests before committing"
-- Drop "you should", "make sure to", "remember to" — just state the action
-- Merge redundant bullets that say the same thing differently
-- Keep one example where multiple examples show the same pattern
-
-CRITICAL RULE:
-Anything inside ``` ... ``` must be copied EXACTLY.
-Do not:
-- remove comments
-- remove spacing
-- reorder lines
-- shorten commands
-- simplify anything
-
-Inline code (`...`) must be preserved EXACTLY.
-Do not modify anything inside backticks.
-
-If file contains code blocks:
-- Treat code blocks as read-only regions
-- Only compress text outside them
-- Do not merge sections around code
-
-## Pattern
-
-Original:
-> You should always make sure to run the test suite before pushing any changes to the main branch. This is important because it helps catch bugs early and prevents broken builds from being deployed to production.
-
-Compressed:
-> Run tests before push to main. Catch bugs early, prevent broken prod deploys.
-
-Original:
-> The application uses a microservices architecture with the following components. The API gateway handles all incoming requests and routes them to the appropriate service. The authentication service is responsible for managing user sessions and JWT tokens.
-
-Compressed:
-> Microservices architecture. API gateway route all requests to services. Auth service manage user sessions + JWT tokens.
-
-## Boundaries
-
-- ONLY compress natural language files (.md, .txt, extensionless)
-- NEVER modify: .py, .js, .ts, .json, .yaml, .yml, .toml, .env, .lock, .css, .html, .xml, .sql, .sh
-- If file has mixed content (prose + code), compress ONLY the prose sections
-- If unsure whether something is code or prose, leave it unchanged
-- Original file is backed up as FILE.original.md before overwriting
-- Never compress FILE.original.md (skip it)

package/.agents/skills/compress/scripts/__init__.py

@@ -1,9 +0,0 @@
-"""Caveman compress scripts.
-
-This package provides tools to compress natural language markdown files
-into caveman format to save input tokens.
-"""
-
-__all__ = ["cli", "compress", "detect", "validate"]
-
-__version__ = "1.0.0"

package/.agents/skills/compress/scripts/__main__.py

@@ -1,3 +0,0 @@
-from .cli import main
-
-main()

package/.agents/skills/compress/scripts/benchmark.py

@@ -1,78 +0,0 @@
-#!/usr/bin/env python3
-from pathlib import Path
-import sys
-
-# Support both direct execution and module import
-try:
-    from .validate import validate
-except ImportError:
-    sys.path.insert(0, str(Path(__file__).parent))
-    from validate import validate
-
-try:
-    import tiktoken
-    _enc = tiktoken.get_encoding("o200k_base")
-except ImportError:
-    _enc = None
-
-
-def count_tokens(text):
-    if _enc is None:
-        return len(text.split())  # fallback: word count
-    return len(_enc.encode(text))
-
-
-def benchmark_pair(orig_path: Path, comp_path: Path):
-    orig_text = orig_path.read_text()
-    comp_text = comp_path.read_text()
-
-    orig_tokens = count_tokens(orig_text)
-    comp_tokens = count_tokens(comp_text)
-    saved = 100 * (orig_tokens - comp_tokens) / orig_tokens if orig_tokens > 0 else 0.0
-    result = validate(orig_path, comp_path)
-
-    return (comp_path.name, orig_tokens, comp_tokens, saved, result.is_valid)
-
-
-def print_table(rows):
-    print("\n| File | Original | Compressed | Saved % | Valid |")
-    print("|------|----------|------------|---------|-------|")
-    for r in rows:
-        print(f"| {r[0]} | {r[1]} | {r[2]} | {r[3]:.1f}% | {'✅' if r[4] else '❌'} |")
-
-
-def main():
-    # Direct file pair: python3 benchmark.py original.md compressed.md
-    if len(sys.argv) == 3:
-        orig = Path(sys.argv[1]).resolve()
-        comp = Path(sys.argv[2]).resolve()
-        if not orig.exists():
-            print(f"❌ Not found: {orig}")
-            sys.exit(1)
-        if not comp.exists():
-            print(f"❌ Not found: {comp}")
-            sys.exit(1)
-        print_table([benchmark_pair(orig, comp)])
-        return
-
-    # Glob mode: repo_root/tests/caveman-compress/
-    tests_dir = Path(__file__).parent.parent.parent / "tests" / "caveman-compress"
-    if not tests_dir.exists():
-        print(f"❌ Tests dir not found: {tests_dir}")
-        sys.exit(1)
-
-    rows = []
-    for orig in sorted(tests_dir.glob("*.original.md")):
-        comp = orig.with_name(orig.stem.removesuffix(".original") + ".md")
-        if comp.exists():
-            rows.append(benchmark_pair(orig, comp))
-
-    if not rows:
-        print("No compressed file pairs found.")
-        return
-
-    print_table(rows)
-
-
-if __name__ == "__main__":
-    main()

package/.agents/skills/compress/scripts/cli.py

@@ -1,73 +0,0 @@
-#!/usr/bin/env python3
-"""
-Caveman Compress CLI
-
-Usage:
-    caveman <filepath>
-"""
-
-import sys
-from pathlib import Path
-
-from .compress import compress_file
-from .detect import detect_file_type, should_compress
-
-
-def print_usage():
-    print("Usage: caveman <filepath>")
-
-
-def main():
-    if len(sys.argv) != 2:
-        print_usage()
-        sys.exit(1)
-
-    filepath = Path(sys.argv[1])
-
-    # Check file exists
-    if not filepath.exists():
-        print(f"❌ File not found: {filepath}")
-        sys.exit(1)
-
-    if not filepath.is_file():
-        print(f"❌ Not a file: {filepath}")
-        sys.exit(1)
-
-    filepath = filepath.resolve()
-
-    # Detect file type
-    file_type = detect_file_type(filepath)
-
-    print(f"Detected: {file_type}")
-
-    # Check if compressible
-    if not should_compress(filepath):
-        print("Skipping: file is not natural language (code/config)")
-        sys.exit(0)
-
-    print("Starting caveman compression...\n")
-
-    try:
-        success = compress_file(filepath)
-
-        if success:
-            print("\nCompression completed successfully")
-            backup_path = filepath.with_name(filepath.stem + ".original.md")
-            print(f"Compressed: {filepath}")
-            print(f"Original:   {backup_path}")
-            sys.exit(0)
-        else:
-            print("\n❌ Compression failed after retries")
-            sys.exit(2)
-
-    except KeyboardInterrupt:
-        print("\nInterrupted by user")
-        sys.exit(130)
-
-    except Exception as e:
-        print(f"\n❌ Error: {e}")
-        sys.exit(1)
-
-
-if __name__ == "__main__":
-    main()

package/.agents/skills/compress/scripts/compress.py

@@ -1,227 +0,0 @@
-#!/usr/bin/env python3
-"""
-Caveman Memory Compression Orchestrator
-
-Usage:
-    python scripts/compress.py <filepath>
-"""
-
-import os
-import re
-import subprocess
-from pathlib import Path
-from typing import List
-
-OUTER_FENCE_REGEX = re.compile(
-    r"\A\s*(`{3,}|~{3,})[^\n]*\n(.*)\n\1\s*\Z", re.DOTALL
-)
-
-# Filenames and paths that almost certainly hold secrets or PII. Compressing
-# them ships raw bytes to the Anthropic API — a third-party data boundary that
-# developers on sensitive codebases cannot cross. detect.py already skips .env
-# by extension, but credentials.md / secrets.txt / ~/.aws/credentials would
-# slip through the natural-language filter. This is a hard refuse before read.
-SENSITIVE_BASENAME_REGEX = re.compile(
-    r"(?ix)^("
-    r"\.env(\..+)?"
-    r"|\.netrc"
-    r"|credentials(\..+)?"
-    r"|secrets?(\..+)?"
-    r"|passwords?(\..+)?"
-    r"|id_(rsa|dsa|ecdsa|ed25519)(\.pub)?"
-    r"|authorized_keys"
-    r"|known_hosts"
-    r"|.*\.(pem|key|p12|pfx|crt|cer|jks|keystore|asc|gpg)"
-    r")$"
-)
-
-SENSITIVE_PATH_COMPONENTS = frozenset({".ssh", ".aws", ".gnupg", ".kube", ".docker"})
-
-SENSITIVE_NAME_TOKENS = (
-    "secret", "credential", "password", "passwd",
-    "apikey", "accesskey", "token", "privatekey",
-)
-
-
-def is_sensitive_path(filepath: Path) -> bool:
-    """Heuristic denylist for files that must never be shipped to a third-party API."""
-    name = filepath.name
-    if SENSITIVE_BASENAME_REGEX.match(name):
-        return True
-    lowered_parts = {p.lower() for p in filepath.parts}
-    if lowered_parts & SENSITIVE_PATH_COMPONENTS:
-        return True
-    # Normalize separators so "api-key" and "api_key" both match "apikey".
-    lower = re.sub(r"[_\-\s.]", "", name.lower())
-    return any(tok in lower for tok in SENSITIVE_NAME_TOKENS)
-
-
-def strip_llm_wrapper(text: str) -> str:
-    """Strip outer ```markdown ... ``` fence when it wraps the entire output."""
-    m = OUTER_FENCE_REGEX.match(text)
-    if m:
-        return m.group(2)
-    return text
-
-from .detect import should_compress
-from .validate import validate
-
-MAX_RETRIES = 2
-
-
-# ---------- Claude Calls ----------
-
-
-def call_claude(prompt: str) -> str:
-    api_key = os.environ.get("ANTHROPIC_API_KEY")
-    if api_key:
-        try:
-            import anthropic
-
-            client = anthropic.Anthropic(api_key=api_key)
-            msg = client.messages.create(
-                model=os.environ.get("CAVEMAN_MODEL", "claude-sonnet-4-5"),
-                max_tokens=8192,
-                messages=[{"role": "user", "content": prompt}],
-            )
-            return strip_llm_wrapper(msg.content[0].text.strip())
-        except ImportError:
-            pass  # anthropic not installed, fall back to CLI
-    # Fallback: use claude CLI (handles desktop auth)
-    try:
-        result = subprocess.run(
-            ["claude", "--print"],
-            input=prompt,
-            text=True,
-            capture_output=True,
-            check=True,
-        )
-        return strip_llm_wrapper(result.stdout.strip())
-    except subprocess.CalledProcessError as e:
-        raise RuntimeError(f"Claude call failed:\n{e.stderr}")
-
-
-def build_compress_prompt(original: str) -> str:
-    return f"""
-Compress this markdown into caveman format.
-
-STRICT RULES:
-- Do NOT modify anything inside ``` code blocks
-- Do NOT modify anything inside inline backticks
-- Preserve ALL URLs exactly
-- Preserve ALL headings exactly
-- Preserve file paths and commands
-- Return ONLY the compressed markdown body — do NOT wrap the entire output in a ```markdown fence or any other fence. Inner code blocks from the original stay as-is; do not add a new outer fence around the whole file.
-
-Only compress natural language.
-
-TEXT:
-{original}
-"""
-
-
-def build_fix_prompt(original: str, compressed: str, errors: List[str]) -> str:
-    errors_str = "\n".join(f"- {e}" for e in errors)
-    return f"""You are fixing a caveman-compressed markdown file. Specific validation errors were found.
-
-CRITICAL RULES:
-- DO NOT recompress or rephrase the file
-- ONLY fix the listed errors — leave everything else exactly as-is
-- The ORIGINAL is provided as reference only (to restore missing content)
-- Preserve caveman style in all untouched sections
-
-ERRORS TO FIX:
-{errors_str}
-
-HOW TO FIX:
-- Missing URL: find it in ORIGINAL, restore it exactly where it belongs in COMPRESSED
-- Code block mismatch: find the exact code block in ORIGINAL, restore it in COMPRESSED
-- Heading mismatch: restore the exact heading text from ORIGINAL into COMPRESSED
-- Do not touch any section not mentioned in the errors
-
-ORIGINAL (reference only):
-{original}
-
-COMPRESSED (fix this):
-{compressed}
-
-Return ONLY the fixed compressed file. No explanation.
-"""
-
-
-# ---------- Core Logic ----------
-
-
-def compress_file(filepath: Path) -> bool:
-    # Resolve and validate path
-    filepath = filepath.resolve()
-    MAX_FILE_SIZE = 500_000  # 500KB
-    if not filepath.exists():
-        raise FileNotFoundError(f"File not found: {filepath}")
-    if filepath.stat().st_size > MAX_FILE_SIZE:
-        raise ValueError(f"File too large to compress safely (max 500KB): {filepath}")
-
-    # Refuse files that look like they contain secrets or PII. Compressing ships
-    # the raw bytes to the Anthropic API — a third-party boundary — so we fail
-    # loudly rather than silently exfiltrate credentials or keys. Override is
-    # intentional: the user must rename the file if the heuristic is wrong.
-    if is_sensitive_path(filepath):
-        raise ValueError(
-            f"Refusing to compress {filepath}: filename looks sensitive "
-            "(credentials, keys, secrets, or known private paths). "
-            "Compression sends file contents to the Anthropic API. "
-            "Rename the file if this is a false positive."
-        )
-
-    print(f"Processing: {filepath}")
-
-    if not should_compress(filepath):
-        print("Skipping (not natural language)")
-        return False
-
-    original_text = filepath.read_text(errors="ignore")
-    backup_path = filepath.with_name(filepath.stem + ".original.md")
-
-    # Check if backup already exists to prevent accidental overwriting
-    if backup_path.exists():
-        print(f"⚠️ Backup file already exists: {backup_path}")
-        print("The original backup may contain important content.")
-        print("Aborting to prevent data loss. Please remove or rename the backup file if you want to proceed.")
-        return False
-
-    # Step 1: Compress
-    print("Compressing with Claude...")
-    compressed = call_claude(build_compress_prompt(original_text))
-
-    # Save original as backup, write compressed to original path
-    backup_path.write_text(original_text)
-    filepath.write_text(compressed)
-
-    # Step 2: Validate + Retry
-    for attempt in range(MAX_RETRIES):
-        print(f"\nValidation attempt {attempt + 1}")
-
-        result = validate(backup_path, filepath)
-
-        if result.is_valid:
-            print("Validation passed")
-            break
-
-        print("❌ Validation failed:")
-        for err in result.errors:
-            print(f"   - {err}")
-
-        if attempt == MAX_RETRIES - 1:
-            # Restore original on failure
-            filepath.write_text(original_text)
-            backup_path.unlink(missing_ok=True)
-            print("❌ Failed after retries — original restored")
-            return False
-
-        print("Fixing with Claude...")
-        compressed = call_claude(
-            build_fix_prompt(original_text, compressed, result.errors)
-        )
-        filepath.write_text(compressed)
-
-    return True