icycache 0.3.2__tar.gz

icycache-0.3.2/.gitignore

@@ -0,0 +1,19 @@
+venv/
+.venv/
+db/
+logs/
+__pycache__/
+*.pyc
+*.bak*
+fix_*.py
+*.pre_purge*
+.pytest_cache/
+*.egg-info/
+build/
+dist/
+node_modules/
+*.tgz
+.commit_msg.txt
+
+# Scratch de travail (copies isolées, db, logs, captures) - hors livrable
+scratch/

icycache-0.3.2/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 Malik (MlkBlk-IT)
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

icycache-0.3.2/PKG-INFO

@@ -0,0 +1,422 @@
+Metadata-Version: 2.5
+Name: icycache
+Version: 0.3.2
+Summary: IcyCache - universal LLM gateway with a local cache. I see cache, I burn cache.
+Project-URL: Homepage, https://github.com/MlkBlk-IT/icycache
+Project-URL: Repository, https://github.com/MlkBlk-IT/icycache
+Project-URL: Issues, https://github.com/MlkBlk-IT/icycache/issues
+Author: MlkBlk-IT
+License: MIT
+License-File: LICENSE
+Keywords: cache,fastapi,llm,llm-proxy,local-llm,mcp,mcp-server,semantic-cache,token-economy
+Classifier: Development Status :: 4 - Beta
+Classifier: Framework :: FastAPI
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Internet :: Proxy Servers
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Python: >=3.10
+Requires-Dist: fastapi<1,>=0.115
+Requires-Dist: httpx<1,>=0.27
+Requires-Dist: mcp<2,>=1.2.0
+Requires-Dist: nltk>=3.8
+Requires-Dist: psutil>=6.0
+Requires-Dist: pyahocorasick>=2.1
+Requires-Dist: pydantic>=2.9
+Requires-Dist: python-dotenv>=1.0
+Requires-Dist: pyyaml>=6.0
+Requires-Dist: sse-starlette>=2.1
+Requires-Dist: tqdm>=4.66
+Requires-Dist: uvicorn>=0.30
+Requires-Dist: websockets>=12.0
+Provides-Extra: dev
+Requires-Dist: pytest>=8.0; extra == 'dev'
+Requires-Dist: ruff>=0.5.0; extra == 'dev'
+Provides-Extra: tray
+Requires-Dist: pillow>=10.0; extra == 'tray'
+Requires-Dist: pystray>=0.19; extra == 'tray'
+Requires-Dist: pywin32>=306; (sys_platform == 'win32') and extra == 'tray'
+Description-Content-Type: text/markdown
+
+> ## ⚠️ NOTICE — The rest of this repository (code, comments, scripts, docs, commits, issues) is written in **French**. Only this README is in English.
+> ✅ **SWAP DONE — 2026-10-24.** IcyCache was renamed from CacheForge. The **universal multi-upstream gateway**
+> is now the **single active service on `:8080`**, and the historical server holds the MCP layer.
+> - `src/server.py` → **the gateway** (18 upstreams in `config/icycache.gateway.json`: OmniRoute, LM Studio, Ollama, OpenAI, Anthropic, NVIDIA, Groq, Cerebras, OpenRouter, Kilo, Nous, Copilot, AI Gateway, HuggingFace, Google AI Studio, LLM7.io, Mistral, Cloudflare).
+> - `src/mcp_server.py` → **the MCP layer** (SSE `:8081`, Streamable HTTP `:8084`, stdio).
+> - Both start together via `tray.py`; `:8080` is the single entry point for all LLM traffic.
+> - Legacy entry point kept: `python src/server.py --port 8090` for isolated testing.
+>
+> ### Update (2026-10-31) — proxy robustness
+> - Legacy cache keys (built before `_tools_sig`) still hit, so v1 cache entries are not orphaned.
+> - A client disconnect mid-stream is detected and logged; an aborted stream is not cached.
+> - Upstream errors now return the upstream's own status and body instead of a generic 502.
+>
+> ### DELTA-002 — upstream resilience + opt-in semantic cache (2026-10-26)
+> - **BUG-001 — 502 stuck after a 400:** the httpx client is now drained, recycled, and the request retried once
+>   (`src/upstream_client.py`). The gateway recovers instead of staying stuck.
+> - **Number normalization:** `"two plus two"` → `2+2`. 11 new rules (1111 → **1122**).
+> - **Semantic cache:** implemented (`src/icy_semantic.py`) but **off by default** (`semantic_enabled: false`).
+> - **Tool registry:** the 100 verified MCP tool anchors (DELTA-001) are registered and asserted at startup.
+>
+> ### Available MCP transports (all three active at once)
+>
+> | Transport | Endpoint | Target client |
+> |-----------|----------|---------------|
+> | **stdio** | `python mcp_stdio.py` | Claude Desktop, IDEs, any stdio client |
+> | **SSE** (legacy) | `http://127.0.0.1:8081/sse` | Hermes, older clients |
+> | **Streamable HTTP** (standard) | `http://127.0.0.1:8084/mcp` | Recent MCP clients, MCP Inspector |
+>
+> In stdio mode, go through `mcp_stdio.py`: it reserves stdout for JSON-RPC.
+> (Nothing is written to stdout anymore, but the wrapper remains the safety net.)
+
+# 🔥 IcyCache — Universal LLM Gateway with local cache
+
+[![version](https://img.shields.io/badge/v0.3.2-e10600?style=for-the-badge&labelColor=0d0d0d)](https://github.com/MlkBlk-IT/icycache/releases) [![M8ven Score](https://m8ven.ai/badge/mcp/mlkblk-it-mcp-cacheforge-16d3le)](https://m8ven.ai/mcp/mlkblk-it-mcp-cacheforge-16d3le?s=readme)
+
+> **"Forge the cache, not the answer."**
+> HTTP gateway that sits between **any client** and **any upstream** (local or cloud),
+> caches responses and serves them back **instantly** — no CPU regeneration.
+>
+> 📖 **Full configuration guide** : [`docs/CONFIGURATION.md`](docs/CONFIGURATION.md)
+> — multi-upstream routing, per-client setup (Hermes, Claude Desktop, Cursor, VS Code,
+> Ollama, direct SDK), MISS→HIT proof, troubleshooting, firewall enforcement.
+
+---
+
+## Dashboard
+
+![IcyCache dashboard](docs/images/dashboard.png)
+
+*Read-only web dashboard (`http://127.0.0.1:8080/dashboard`) on a local instance, captured 2026-10-31.*
+
+## The problem
+
+| Problem | Consequence |
+|---------|-------------|
+| **Latency** | An identical request burns CPU/RAM again for an already-computed answer |
+| **No semantic cache** | Standard systems hash the raw prompt: `Explain Hyper-V` and `Please explain Hyper-V thanks` generate **two** distinct requests |
+| **No observability** | Impossible to measure the tokens and time actually saved |
+| **Traffic bypassing the proxy** | Every client talking directly to its provider breaks the cache promise |
+
+IcyCache attacks all four: normalization before hashing, SQLite cache with TTL, exportable metrics,
+and **a single entry point** for all LLM traffic.
+
+> **Semantic cache status:** implemented but **opt-in** (`semantic_enabled: false` by default). It needs an
+> embedding endpoint (LM Studio on `:1234`) and a Qdrant instance; when either is missing the cache
+> degrades silently to a miss instead of failing the request. See [`config/semantic_cache.json`](config/semantic_cache.json).
+
+---
+
+## Target architecture
+
+```
+   Any client                 (Hermes, Claude Code, scripts, IDE, SDK, CLI)
+   base_url = http://127.0.0.1:8080
+              │
+              ▼
+┌───────────────────────────────────────────────────────────────────────┐
+│  ICYCACHE GATEWAY  127.0.0.1:8080   (src/server.py)                   │
+│                                                                       │
+│  ROUTING      URL prefix  /<upstream>/v1/...                          │
+│               header      X-IcyCache-Upstream: lmstudio               │
+│               body        "upstream": "ollama"                        │
+│               default     gateway.default_upstream (config)           │
+│                                                                       │
+│  NORMALIZATION NFKD → lowercase → Aho-Corasick → accents → punctuation│
+│  CACHE KEY     SHA256 of the NORMALIZED payload (+ upstream + endpoint)│
+│  DECISION      HIT → instant response | MISS → forward + save         │
+│  TOKENS        REAL usage from the upstream. Never an estimate.       │
+│  OBSERVABLE    WebSocket /ws/events : live deltas + final             │
+│  MCP (facade)  stats / clear / search / realtime / tool-cache         │
+└───────────────────────────────────────────────────────────────────────┘
+        │                    ▲                              │
+        ▼                    │ same path back                │
+ { OmniRoute | LM Studio | Ollama | OpenAI | Anthropic | … }
+        │
+        ▼
+   SQLite db/cache.db   (events: upstream, real tokens, latencies, ttft)
+```
+
+**Rule: everything that leaves a client and comes back to it goes through IcyCache.**
+OmniRoute is no longer the entry point: it is **one upstream among others**.
+
+---
+
+## Components
+
+| File | Role |
+|------|------|
+| `src/server.py` | **Universal multi-upstream gateway** (FastAPI): routing, cache, real tokens, WebSocket |
+| `src/mcp_gateway.py` | **MCP facade** of the gateway: 7 tools, tool-cache, loop guard, mirror REST API |
+| `src/icy_core.py` | Shared core: DB (`events`, `agent_events`, `tool_cache`), WebSocket hub, loop guard |
+| `src/icy_normalize.py` | Standalone normalization engine (faithful port, no import side effects) |
+| `src/icy_cache_store.py` | LLM cache store (compatible `cache` table, additive `raw_hash` column) |
+| `src/upstream_utils.py` | Usage normalization, headers, cache_control/prompt_cache_key mutation, SSE meter |
+| `src/mcp_stdio_bridge.py` | stdio bridge: reserves stdout for MCP JSON-RPC, logs and prints to stderr |
+| `src/mcp_server.py` | **MCP server** (FastMCP): tools, tool-cache, loop guard — listens on :8081 (SSE) / :8084 (HTTP) |
+| `config/icycache.gateway.json` | Gateway config: upstreams, cache, loop guard, tool cache |
+| `config/normalization_rules.json` | 1122 normalization rules (incl. number words → digits) |
+| `src/upstream_client.py` | **Resilient upstream client** — one pool per base URL, drain + recycle + retry |
+| `src/icy_semantic.py` | **Opt-in semantic cache** — embeddings + Qdrant store, silent miss when unavailable |
+| `config/semantic_cache.json` | Semantic cache settings (`semantic_enabled`, thresholds, embedder URL) |
+| `config/tools_registry.json` | Registry of the 100 verified MCP tool anchors (DELTA-001) |
+| `tools/smoke_gateway.py` | **Real** multi-upstream smoke test (isolated ports, temp DB) |
+| `tools/enforce_gateway.ps1` | Firewall rules: makes bypass impossible |
+| `tools/verify_enforcement.py` | **Proof**: can the client still reach an upstream directly? |
+| `tests/test_gateway.py` | 10 isolated tests (mock upstream, temp DB) |
+| `tests/test_semantic_cache.py` | Semantic cache unit tests |
+| `tests/test_upstream_recovery.py` | Upstream recovery / retry tests (BUG-001) |
+| `tests/test_anchors.py` | Normalization anchor tests (DELTA-002) |
+
+---
+
+## Routing: pick your upstream
+
+| Method | Example | Priority |
+|--------|---------|----------|
+| URL prefix | `POST http://127.0.0.1:8080/lmstudio/v1/chat/completions` | 1 |
+| Header | `X-IcyCache-Upstream: lmstudio` | 2 |
+| Body | `{"upstream": "lmstudio", ...}` | 3 |
+| Config default | `gateway.default_upstream` | 4 |
+
+No **implicit** default in the code: without an explicit source, the request is rejected with
+`400` listing the available upstreams. An unknown upstream is rejected with `404`.
+
+---
+
+## HTTP routes (gateway)
+
+| Route | Method | Role |
+|-------|--------|------|
+| `/v1/chat/completions` | POST | OpenAI-compatible (streaming and non-streaming) |
+| `/v1/messages` | POST | Native Anthropic (`kind: anthropic`) |
+| `/v1/embeddings` | POST | Embeddings (cached) |
+| `/v1/models` | GET | Upstream models |
+| `/{upstream}/v1/{rest}` | * | Explicit routing by prefix |
+| `/ws/events` | WS | Realtime events: live `delta` + `final` (real tokens) |
+| `/health`, `/api/status` | GET | Health, reachable upstreams |
+| `/api/metrics`, `/api/metrics/series`, `/api/metrics/export` | GET | Metrics (dashboard shapes) |
+| `/api/agent-events` | GET | Agent / tool-cache events |
+| `/api/cache-clear` | POST | Empty the cache — header `X-IcyCache-Confirm: FORCE_DELETE` |
+| `/dashboard` | GET | Web dashboard (read-only) |
+
+**Confirmation headers:** `X-IcyCache-Confirm` (unified naming). The old `X-CacheForge-Confirm`
+is still accepted by the gateway for compatibility.
+
+---
+
+## Tokens: the rule of the real
+
+| Source | Value |
+|--------|-------|
+| `usage` field of the upstream response | **always authoritative** |
+| OpenAI streaming | `stream_options.include_usage=true` injected → usage from last chunk |
+| Anthropic streaming | `message_start` + `message_delta` |
+| Usage absent | `NULL` + `tokens_source: "n/a"` — **never an estimate** |
+| HIT | `saved_prompt_tokens` / `saved_completion_tokens` = real usage of the 1st call |
+
+`events` columns: `upstream`, `client`, `prompt_tokens`, `completion_tokens`, `cached_tokens`,
+`cache_write_tokens`, `latency_ms`, `ttft_ms`.
+
+---
+
+## MCP tools (facade)
+
+### MCP server (`src/mcp_server.py`) — 11 tools
+
+| Tool | Role |
+|------|------|
+| `get_cache_stats` | Entries, hits, breakdown by upstream/model |
+| `clear_cache` | Empty the cache (`confirm='FORCE_DELETE'`) |
+| `normalize_preview` | Preview of normalization + cache key |
+| `search_cached_prompts` | Search in normalized prompts |
+| **`proxy_tool`** | **Call a tool on a target MCP server** (tool calling) |
+| **`proxy_tools`** | **Batch call of target tools** |
+| **`list_proxy_servers`** | **Routable MCP servers** |
+| **`list_proxy_tools`** | **Tools exposed by a target server** |
+| `get_tool_cache_stats` | Tool-cache state + agent events |
+| `clear_tool_cache` | Empty the tool cache (`confirm='FORCE_DELETE'`) |
+| `check_tool_loop` | Detect an agent loop in an LLM history |
+
+### Gateway server (`src/mcp_gateway.py`) — 11 tools
+
+The 7 observability tools (`get_cache_stats`, `clear_cache`, `normalize_preview`,
+`search_cached_prompts`, `get_realtime_tokens`, `list_upstreams`, `tool_cache_stats`)
+**plus** the 4 relay tools (`proxy_tool`, `proxy_tools`, `list_proxy_servers`,
+`list_proxy_tools`).
+
+MCP is a **facade**: it observes and drives. It is not in the request path
+(an MCP client calls its tools deliberately — you cannot intercept an
+OpenAI/Anthropic stream with it). The request path is the HTTP gateway.
+
+Agent loops: `icy_core.llm_loop_check()` detects the same tool called again with the same
+arguments **and** the same results (Anthropic and OpenAI formats). Threshold: 3 repetitions = warning.
+
+---
+
+## Tool calling: safety rules
+
+The tool proxy (`src/tool_proxy.py`) relays calls to target MCP servers.
+Three non-negotiable rules:
+
+1. **An MCP error is never cached** and never returns `ok: true`.
+   An `isError: true` becomes `ok: false` with the real message (no opaque `ExceptionGroup`).
+2. **Write tools are never cached.** Replaying a cached response
+   must have no side effect; otherwise a write is silently skipped.
+   - `cache_tools` present → **strict allowlist** (only tools listed as `true` are cached).
+   - otherwise `cache: true` → **anti-write heuristic** (`create_`, `write_`, `delete_`, `push_`, … excluded).
+   - otherwise → no cache.
+3. **MCP sessions are persistent** (one process per server, not per call).
+   Without this, every tool call respawned a process: 21s measured on `npx`, 12s on `uvx`.
+
+### Example
+
+```json
+// via any MCP client
+{"server": "github", "tool": "search_repositories", "args": {"query": "icycache"}}
+// -> {"ok": true, "cached": false, "latency_ms": 4867, "result": {...}}
+```
+
+### Target server configuration
+
+`config/icycache.json` → `tool_proxy.servers`. Two portability tokens:
+
+| Token | Value |
+|-------|-------|
+| `{PROJECT_ROOT}` | project root (moving IcyCache needs no edit) |
+| `{PYTHON}` | project venv interpreter (the only one with `fastapi`/`mcp`) |
+
+On Windows, `.cmd` commands (like `npx`) are resolved automatically:
+`CreateProcess` cannot launch an `npx.cmd` script without a shell.
+
+---
+
+## Enforcement: making bypass impossible
+
+```powershell
+# 1. Install the rules (ELEVATED PowerShell)
+powershell -ExecutionPolicy Bypass -File tools\enforce_gateway.ps1 -Action apply `
+    -TargetProgram "C:\Users\Tech\AppData\Local\hermes\hermes-agent\apps\desktop\release\win-unpacked\Hermes.exe" `
+    -AllowPort 8080 -BlockPorts 20128,1234,11434
+
+# 2. The proof: can the client still reach an upstream directly?
+python tools\verify_enforcement.py --program "D:\IcyCache\venv\Scripts\python.exe" --allow-port 8080
+```
+
+Strategy (Windows Firewall, per program): block upstream local ports **including on
+127.0.0.1**, block all remote traffic, explicitly allow `127.0.0.1:8080`.
+The `verify_enforcement.py` program runs **with the client binary** (rules are per program)
+and checks that only IcyCache remains reachable.
+
+---
+
+## Environment variables (unified naming)
+
+| Variable | Role | Old name (still read by `src/mcp_server.py` for `ICYCACHE_*`) |
+|----------|------|----------------------------------------------------------|
+| `ICYCACHE_DB` | Cache DB path | `CACHEFORGE_DB` |
+| `ICYCACHE_LOG` | Log path | `CACHEFORGE_LOG` |
+| `ICYCACHE_CONFIG` | Config file (gateway) | — |
+| `ICYCACHE_TARGET` | Upstream override (isolated tests) | `CACHEFORGE_TARGET` |
+| `ICYCACHE_TOOLPROXY_CONFIG` | Tool-proxy config (default: `config/icycache.json`) | — |
+
+---
+
+## Installation
+
+> **The registry publication is pending.** The repository, the PyPI distribution and the npm package
+> all carry the same name — **`icycache`** — and the packages are built, tagged and ready to publish
+> (`v0.3.2`). The names were free on both registries, and `icycache-mcp` is the second binary the
+> npm package exposes.
+>
+> The registry commands below are the final ones, but they return a 404 until the first publish.
+> Install from source in the meantime:
+> `pip install "icycache @ git+https://github.com/MlkBlk-IT/icycache.git"`.
+
+```bash
+uv tool install icycache          # or: pip install icycache
+```
+
+| Registry | Package | Page |
+|---|---|---|
+| PyPI | `icycache` | [pypi.org/project/icycache](https://pypi.org/project/icycache/) |
+| npm | `icycache` | [npmjs.com/package/icycache](https://www.npmjs.com/package/icycache) |
+
+This gives you the `icycache` console script:
+
+```bash
+icycache serve          # the gateway, http://127.0.0.1:8080
+icycache mcp            # the MCP layer (SSE :8081 / HTTP :8084 / stdio)
+icycache doctor         # diagnostic: paths, config, upstreams, database
+```
+
+Installed packages never write into `site-packages`: `db/` and `logs/` go to
+`%LOCALAPPDATA%\IcyCache` (Windows) or `~/.local/share/IcyCache`.
+Full guide: [`docs/INSTALL.md`](docs/INSTALL.md).
+
+### npm (Node-only clients)
+
+The npm package is `icycache` — the same name as the repository. It exposes two binaries:
+
+```bash
+npx -y icycache --help              # the CLI (serve / mcp / doctor)
+npx -y -p icycache icycache-mcp     # MCP server on stdio
+```
+
+### From source (Windows)
+
+```powershell
+# 1. Environment (Python 3.12.10)
+python -m venv venv
+.\venv\Scripts\Activate.ps1
+pip install -e ".[dev]"
+
+# 2. Manual start
+.\start_icycache.ps1
+
+# 3. Start at login (Task Scheduler, or Startup fallback without admin)
+.\install_autostart.ps1
+```
+
+### Build the packages yourself
+
+```bash
+python -m build          # -> dist/icycache-<version>-py3-none-any.whl + .tar.gz
+```
+
+> Building and publishing this package: see [`PUBLISHING.md`](PUBLISHING.md).
+
+---
+
+## Tests
+
+```powershell
+# Isolated suite (mock upstream, temp DB, no port occupied)
+.\venv\Scripts\python.exe -m pytest tests\test_gateway.py -v
+
+# Isolated integration suite (temp DB + fake upstream :8190/:8191)
+.\venv\Scripts\python.exe tests\run_isolated.py
+
+# Offline PrefixProxy test (fake provider, no external call)
+.\venv\Scripts\python.exe tests\test_prefix_proxy.py
+
+# Real smoke test (LM Studio / OmniRoute, port 8090, temp DB)
+.\venv\Scripts\python.exe tools\smoke_gateway.py --port 8090
+
+# DELTA-002 suite (semantic cache, upstream recovery, normalization anchors)
+.\venv\Scripts\python.exe -m pytest tests\test_semantic_cache.py tests\test_upstream_recovery.py tests\test_anchors.py -v
+```
+
+The smoke test touches **neither** the production service (`:8080`) **nor** the dashboard.
+`tests\run_isolated.py` uses a **temporary** database (`ICYCACHE_DB`) and never touches
+the production database.
+
+---
+
+## License
+
+MIT