token-optimise 0.1.0__tar.gz

token_optimise-0.1.0/.env.example

@@ -0,0 +1,3 @@
+EMBEDDER="all-MiniLM-L6-v2"
+ALLOWED_DIR=your/allowed/dir/path
+GROQ_API_KEY=you_groq_api_key

token_optimise-0.1.0/.gitignore

@@ -0,0 +1,21 @@
+.env
+venv/
+__pycache__/
+*.pyc
+memory.db
+chroma_db/
+.DS_Store
+storage/
+server.log
+extenstion/
+*.py[cod]
+.pytest_cache/
+
+.vscode/
+.idea/
+tests/
+adapters/
+*.log
+.env
+.env.*
+!.env.example

token_optimise-0.1.0/Dockerfile

@@ -0,0 +1,32 @@
+FROM python:3.11-slim
+
+WORKDIR /app
+
+# Set PYTHONPATH so all imports resolve correctly
+ENV PYTHONPATH=/app
+
+RUN pip install --no-cache-dir \
+    streamlit==1.35.0 \
+    streamlit-autorefresh==1.0.1 \
+    requests==2.31.0 \
+    pandas==2.2.0 \
+    plotly==5.22.0 \
+    psycopg2-binary==2.9.9 \
+    pydantic==2.7.0 \
+    pydantic-settings==2.2.0 \
+    python-dotenv==1.0.0 \
+    fastapi==0.110.0 \
+    uvicorn==0.29.0
+
+# Copy as proper packages so imports work
+COPY token_optime/ ./token_optime/
+COPY src/ ./src/
+
+RUN mkdir -p /root/.streamlit
+RUN printf '[server]\nport = 7738\naddress = "0.0.0.0"\nheadless = true\n\n[browser]\ngatherUsageStats = false\n' \
+    > /root/.streamlit/config.toml
+
+EXPOSE 7737
+EXPOSE 7738
+
+CMD ["python", "src/mcp/dash_api.py"]

token_optimise-0.1.0/PKG-INFO

@@ -0,0 +1,498 @@
+Metadata-Version: 2.5
+Name: token-optimise
+Version: 0.1.0
+Summary: Token-optimised MCP proxy server for Claude Desktop
+Requires-Python: >=3.10
+Requires-Dist: chromadb>=1.5.9
+Requires-Dist: fastapi>=0.140.13
+Requires-Dist: fastmcp>=3.4.5
+Requires-Dist: groq>=1.6.0
+Requires-Dist: httpx>=0.28.1
+Requires-Dist: huggingface-hub>=1.28.0
+Requires-Dist: langchain-chroma>=1.1.0
+Requires-Dist: langchain-community>=0.4.2
+Requires-Dist: langchain-core>=1.6.0
+Requires-Dist: langchain-huggingface>=1.2.2
+Requires-Dist: langchain-text-splitters>=1.1.2
+Requires-Dist: mcp>=1.29.0
+Requires-Dist: numpy>=2.5.1
+Requires-Dist: pandas>=3.0.5
+Requires-Dist: plotly>=6.9.0
+Requires-Dist: pydantic-settings>=2.14.2
+Requires-Dist: pydantic>=2.13.4
+Requires-Dist: pymupdf4llm>=1.28.0
+Requires-Dist: pymupdf>=1.28.0
+Requires-Dist: python-dotenv>=1.2.2
+Requires-Dist: requests>=2.34.2
+Requires-Dist: scipy>=1.18.0
+Requires-Dist: sentence-transformers>=5.6.1
+Requires-Dist: sse-starlette>=3.4.6
+Requires-Dist: streamlit-autorefresh>=1.0.1
+Requires-Dist: streamlit>=1.61.1
+Requires-Dist: tiktoken>=0.13.0
+Requires-Dist: uvicorn>=0.51.0
+Description-Content-Type: text/markdown
+
+# 🦾 Token-optimeee
+
+A middleware MCP server that sits between Claude Desktop and your downstream MCP servers — reducing token consumption through semantic caching, dynamic tool selection, and response trimming.
+
+**LLM Backend:** Groq | **Vector Store:** ChromaDB | **Dashboard:** Streamlit
+
+---
+
+## 🎯 What Problem This Solves
+
+Every Claude Desktop session sends the full schema of every connected MCP tool with every prompt. With 23+ tools registered, that's thousands of tokens injected per query — even when only one tool is needed.
+
+Token-optimeee optimises at three levels:
+
+1. **Semantic Tool Selection** — Only the most relevant tool schema is sent to Claude per query (ChromaDB similarity search)
+2. **Semantic Caching** — Repeated or similar queries skip tool calls entirely and return cached answers (similarity ≥ 0.8)
+3. **Response Trimming** — Verbose tool responses are trimmed to ≤200 tokens before entering the context window
+
+**Real result from testing:** 91.8% schema token reduction per call.
+
+---
+
+## 📁 Project Structure
+```
+.
+├── README.md                       
+├── config.py                      # Settings & env loading
+├── start.sh                       # One-click setup and launch
+├── stop.sh                        # One-click
+├── requirements.txt               # Python dependencies
+├── remote_servers.json            # Remote MCP URLs
+│
+├── src/
+│ ├── front.py                     # Streamlit dashboard
+│ ├── core/
+│ │ ├── cache.py                   # Semantic cache
+│ │ ├── trim.py                    # Response trimmer
+│ │ ├── tool_selection.py          # Tool selection
+│ │ ├── client.py                  # Groq/Claude client
+│ │ ├── document_search.py         # RAG (index + search)
+│ │ └── db.py                      # SQLite audit logging
+│ │
+│ └── mcp/
+│ ├── server.py                    # Main MCP server
+│ └── server_http.py               # FastAPI + metrics
+│
+└── storage/
+├── chroma_db/                     # ChromaDB persistent store
+└── token_audit.db                 # SQLite audit logs
+```
+
+
+---
+
+## Our Test Case
+
+ 1. list files on /Users/../Desktop
+ 2. list files on /Users/../Desktop  
+
+ 3. index /Users/../Desktop/leave_policy.pdf doc_id leave
+ 4. what is the leave policy?
+ 5. what is the leave policy?                   
+ 6. read a file                  
+
+
+---
+
+## 🏗️ System Architecture
+
+```
+                    Claude Desktop
+                          │
+                          ▼
+┌──────────────────────────────────────────────────────────┐
+│ Token-Optimised MCP Server                               │
+│ ┌─────────────────────────────────────────────────────┐  │
+│ │ Exposed Tools                                       │  │
+│ │ • find_tool (orchestrator)                          │  │
+│ │ • index_document / ask_document                     │  │
+│ │ • search_all_documents / index_documents_folder     │  │
+│ └─────────────────────────────────────────────────────┘  │
+│ ┌─────────────────────────────────────────────────────┐  │
+│ │ Optimization Layer                                  │  │
+│ │ ┌──────────────┐ ┌──────────────────────────────┐   │  │
+│ │ │ Semantic     │ │ Tool Selection               │   │  │
+│ │ │ Cache        │ │ (ChromaDB → top-k tools)     │   │  │
+│ │ │ (sim ≥ 0.8)  │ │                              │   │  │
+│ │ └──────────────┘ └──────────────────────────────┘   │  │
+│ │ ┌──────────────┐ ┌──────────────────────────────┐   │  │
+│ │ │ Response     │ │ Token Audit                  │   │  │
+│ │ │ Trimmer      │ │ (SQLite audit log)           │   │  │
+│ │ │ (≤500 tok)   │ │                              │   │  │
+│ │ └──────────────┘ └──────────────────────────────┘   │  │
+│ └─────────────────────────────────────────────────────┘  │
+└────────────┬──────────────────┬──────────────────────────┘
+             │                  │
+             ▼                  ▼
+      ┌─────────────┐    ┌──────────────┐
+      │ Filesystem  │    │ Memory MCP   │
+      │ MCP         │    │ (remote)     │
+      │ (14 tools)  │    │ (9 tools)    │
+      └─────────────┘    └──────────────┘
+```
+
+
+---
+
+## 🚀 Setup — One Command
+
+### Prerequisites
+- Python 3.10+
+- Node.js + npx
+- Groq API key (free at [console.groq.com](https://console.groq.com))
+- Claude Desktop (latest)
+
+### Run
+
+## Run
+
+```bash
+chmod +x token.sh
+./token.sh start     # start
+./token.sh stop      # stop
+./token.sh restart   # restart
+./token.sh status    # check if running
+```
+
+That's it. The script will:
+- Check Python and Node are installed
+- Create a virtual environment and install packages (first run: 4-5 mins)
+- Create `.env` from `.env.example` and prompt you to add your Groq key
+- Close Claude Desktop automatically
+- Write the MCP config
+- Start the server in the background (terminal can be closed)
+- Open the dashboard in your browser
+- Reopen Claude Desktop
+
+### Stop
+
+```bash
+./stop.sh
+```
+
+Or click the **⏹ Stop Token-optime** button in the dashboard.
+
+---
+
+## 🔧 Ports
+
+| Service | Port |
+|---|---|
+| FastAPI / MCP | 7737 |
+| Streamlit Dashboard | 7738 |
+
+These are intentionally non-default to avoid conflicts with other projects.
+
+---
+
+## 💬 For Best Results — Tell Claude to Use Token-optime
+
+Claude Desktop has its own built-in tools (memory, web search) that it may prefer by default. For consistent routing through Token-optime, start each conversation with:
+
+> Use token:execute for every task. Never call other tools directly.
+
+For PDF tasks specifically:
+> First call token:list_indexed_documents, then token:ask_document.
+
+**Why is this needed?** Claude decides which tool to call — Token-optime can't force it. The system prompt in the config nudges Claude, but an explicit instruction in the chat is the most reliable way to ensure Token-optime is used. This is an honest limitation of how Claude Desktop works, not a bug in Token-optime.
+
+---
+
+## 📊 Dashboard
+
+Access at `http://localhost:7738`
+
+| Metric | What it means |
+|---|---|
+| **Schema Tokens Without Token-optime** | Tokens Claude would receive with all tool schemas |
+| **Schema Tokens With Token-optime** | Tokens Claude actually received (1 selected schema) |
+| **Total Tokens Saved** | Schema savings + response trim savings |
+| **Cache Hit Rate** | % of queries that returned cached answers |
+| **Real Groq Token Usage** | Exact token counts from Groq API (not estimates) |
+| **Cost Saved** | Estimated savings based on Sonnet 4.6 pricing |
+
+> **Note:** Cost estimates use Claude Sonnet 4.6 pricing ($3/1M tokens, ₹84/$). Actual cost depends on which model you use in Claude Desktop.
+
+
+
+---
+
+## 🐛 Troubleshooting
+
+**Token-optime already running error**
+./stop.sh
+./start.sh
+
+**Port 7737 in use**
+```bash
+lsof -i :7737
+kill <PID>
+```
+
+**Claude not using token:execute**
+- Check Settings → Developer → MCP Servers — `token` should show green
+- Add explicit instruction at start of conversation (see above)
+
+**Tool not found for query**
+- Rephrase more specifically — e.g. "list files in /Users/name/Desktop" instead of "show my stuff"
+- Check `server_out.log` for similarity scores
+
+**Dashboard offline**
+```bash
+cat server_out.log | tail -50
+```
+
+---
+
+## ⚠️ Known Limitations
+
+- **Claude's built-in tools take priority** — memory and web search bypass Token-optime because Claude prefers its native tools
+- **Cost estimates are approximate** — based on Sonnet 4.6 pricing; varies by model
+- **Windows not supported** — Mac and Linux only
+- **Tilde paths** — if a tool call fails with "file not found", use the full path e.g. `/Users/name/Desktop/file.pdf`
+
+---
+
+## 📖 Docs
+
+- `docs/COMPONENTS.md` — Detailed components guide
+- `docs/QUICKSTART.md` — 5-minute quick start
+
+
+
+## 📁 Project Structure
+```
+.
+├── README.md                       
+├── config.py                      # Settings & env loading
+├── install.sh                     # One-command installer for new users
+├── stop.sh                        # One-click
+├── requirements.txt               # Python dependencies
+├── Dockerfile                     # For EC2 dashboard deployment
+├── pyproject.toml                 # PyPI package definition
+│
+├── src/
+│ ├── front.py                     # Streamlit dashboard
+│ ├── dash_api.py                  # Lightweight API for EC2 deployment
+│ ├── core/
+│ │ ├── cache.py                   # Semantic cache
+│ │ ├── trim.py                    # Response trimmer
+│ │ ├── tool_selection.py          # Tool selection
+│ │ ├── client.py                  # Groq/Claude client
+│ │ ├── document_search.py         # RAG (index + search)
+│ │ └── db.py                      # SQLite audit logging
+│ │
+│ └── mcp/
+│ ├── server.py                    # Main MCP server
+│ └── server_http.py               # FastAPI + metrics
+│
+└── storage/
+├── chroma_db/                     # ChromaDB persistent store
+└── token_audit.db                 # SQLite audit logs
+```
+
+
+---
+
+## Our Test Case
+
+ 1. list files on /Users/../Desktop
+ 2. list files on /Users/../Desktop  
+
+ 3. index /Users/../Desktop/leave_policy.pdf doc_id leave
+ 4. what is the leave policy?
+ 5. what is the leave policy?                   
+ 6. read a file                  
+
+
+---
+
+## 🏗️ System Architecture
+
+```
+                    Claude Desktop
+                          │
+                          ▼
+┌──────────────────────────────────────────────────────────┐
+│ Token-Optimised MCP Server                               │
+│ ┌─────────────────────────────────────────────────────┐  │
+│ │ Exposed Tools                                       │  │
+│ │ • find_tool (orchestrator)                          │  │
+│ │ • index_document / ask_document                     │  │
+│ │ • search_all_documents / index_documents_folder     │  │
+│ └─────────────────────────────────────────────────────┘  │
+│ ┌─────────────────────────────────────────────────────┐  │
+│ │ Optimization Layer                                  │  │
+│ │ ┌──────────────┐ ┌──────────────────────────────┐   │  │
+│ │ │ Semantic     │ │ Tool Selection               │   │  │
+│ │ │ Cache        │ │ (ChromaDB → top-k tools)     │   │  │
+│ │ │ (sim ≥ 0.8)  │ │                              │   │  │
+│ │ └──────────────┘ └──────────────────────────────┘   │  │
+│ │ ┌──────────────┐ ┌──────────────────────────────┐   │  │
+│ │ │ Response     │ │ Token Audit                  │   │  │
+│ │ │ Trimmer      │ │ (SQLite audit log)           │   │  │
+│ │ │ (≤500 tok)   │ │                              │   │  │
+│ │ └──────────────┘ └──────────────────────────────┘   │  │
+│ └─────────────────────────────────────────────────────┘  │
+└────────────┬──────────────────┬──────────────────────────┘
+             │                  │
+             ▼                  ▼
+      ┌─────────────┐    ┌──────────────┐
+      │ Filesystem  │    │ Memory MCP   │
+      │ MCP         │    │ (remote)     │
+      │ (14 tools)  │    │ (9 tools)    │
+      └─────────────┘    └──────────────┘
+```
+
+
+
+---
+
+## 🚀 Install — One Command
+
+### Prerequisites
+- Python 3.10+
+- Node.js + npx
+- Groq API key — free at [console.groq.com](https://console.groq.com)
+- Claude Desktop (latest)
+
+### New users — two commands
+
+```bash
+pip install token-optime
+token-optime
+```
+
+The installer will:
+- Check Python and Node are installed
+- Ask for your Groq API key
+- Write the MCP config into Claude Desktop automatically
+- Restart Claude Desktop
+
+### Existing users / developers
+
+```bash
+git clone https://github.com/yourrepo/tom.git
+cd tom
+cp .env.example .env      # add your GROQ_API_KEY
+./token.sh start
+```
+
+### token.sh commands
+
+```bash
+./token.sh start     # start
+./token.sh stop      # stop
+./token.sh restart   # restart
+./token.sh status    # check if running
+```
+
+---
+
+## 📊 Dashboard
+
+### Local (your metrics only)
+http://localhost:7738
+
+
+---
+
+## 🔧 Ports
+
+| Service | Port |
+|---|---|
+| FastAPI / MCP | 7737 |
+| Streamlit Dashboard | 7738 |
+
+These are intentionally non-default to avoid conflicts with other projects.
+
+---
+
+## 💬 For Best Results — Tell Claude to Use Token-optime
+
+Claude Desktop has its own built-in tools (memory, web search) that it may prefer by default. For consistent routing through Token-optime, start each conversation with:
+
+Add Instructions for Claude :
+> Paste it here Settings → Instruction for Claude 
+```
+You have access to a token MCP server. Follow these rules strictly:
+For ALL tasks — files, Gmail, Notion, memory, or anything else — always use token:execute first before calling any other tool.
+For PDF questions always call token:list_indexed_documents first then token:ask_document.
+After every response call token:wick_track.
+
+DOCUMENT/PDF TASKS:
+- ALWAYS call list_indexed_docs first, then ask_document immediately.
+- NEVER ask the user for clarification.
+
+AFTER EVERY RESPONSE: Call wick_track.
+```
+
+> Use token:execute for every task. Never call other tools directly.
+
+For PDF tasks specifically:
+> First call token:list_indexed_documents, then token:ask_document.
+
+**Why is this needed?** Claude decides which tool to call — Token-optime can't force it. The system prompt in the config nudges Claude, but an explicit instruction in the chat is the most reliable way to ensure Token-optime is used. This is an honest limitation of how Claude Desktop works, not a bug in Token-optime.
+
+---
+# Dashboard Metrics
+
+| Metric | What it means |
+|--------|--------------|
+| Schema Tokens — Estimated Baseline | Tokens Claude would receive with all tool schemas |
+| Schema Tokens — With Token-optime | Tokens Claude actually received (1 selected schema) |
+| Total Tokens Saved | Schema savings + response trim savings |
+| Cache Hit Rate | % of queries that returned cached answers |
+| Groq Cost (actual ✅) | Real measured cost at Groq pricing |
+| Claude Cost Saved (estimated ⚠️) | Range across Haiku→Opus (model unknown) |
+| Live USD/INR Rate | Fetched live every hour |
+
+
+
+---
+
+## 🐛 Troubleshooting
+
+**Server not starting**
+```bash
+tail -50 server_out.log
+```
+
+**Port 7737 in use**
+```bash
+lsof -i :7737
+kill <PID>
+./token.sh start
+```
+
+**Claude not using token:execute**
+- Check Settings → Developer → MCP Servers — `token` should show green
+- Add explicit instruction at start of conversation
+
+**Dashboard offline**
+```bash
+./token.sh status
+./token.sh restart
+```
+
+---
+
+## ⚠️ Known Limitations
+
+- **Claude's built-in tools take priority** — memory and web search bypass Token-optime because Claude prefers its native tools
+- **Cost estimates are approximate** — based on Sonnet 4.6 pricing; varies by model
+- **Windows not supported** — Mac and Linux only
+- **Tilde paths** — if a tool call fails with "file not found", use the full path e.g. `/Users/name/Desktop/file.pdf`
+
+---
+
+## 📖 Docs
+
+- `docs/COMPONENTS.md` — Detailed components guide
+- `docs/QUICKSTART.md` — 5-minute quick start