tokenzero 0.1.0__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- mcp_server.py +346 -0
- proxy.py +205 -0
- scanner.py +398 -0
- squeeze.py +887 -0
- tokenzero-0.1.0.dist-info/METADATA +180 -0
- tokenzero-0.1.0.dist-info/RECORD +10 -0
- tokenzero-0.1.0.dist-info/WHEEL +5 -0
- tokenzero-0.1.0.dist-info/entry_points.txt +3 -0
- tokenzero-0.1.0.dist-info/licenses/LICENSE +22 -0
- tokenzero-0.1.0.dist-info/top_level.txt +4 -0
|
@@ -0,0 +1,180 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: tokenzero
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Summary: Zero Token Waste - AST Code Skeletonizer & Live AI Monitor for Coding Agents
|
|
5
|
+
License-Expression: MIT
|
|
6
|
+
Keywords: tokenzero,tokens,ai-agents,ast,antigravity,claude,cursor,optimization,mcp
|
|
7
|
+
Classifier: Development Status :: 4 - Beta
|
|
8
|
+
Classifier: Intended Audience :: Developers
|
|
9
|
+
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
|
10
|
+
Classifier: Programming Language :: Python :: 3
|
|
11
|
+
Classifier: Operating System :: OS Independent
|
|
12
|
+
Requires-Python: >=3.10
|
|
13
|
+
Description-Content-Type: text/markdown
|
|
14
|
+
License-File: LICENSE
|
|
15
|
+
Requires-Dist: rich>=13.0.0
|
|
16
|
+
Requires-Dist: fastapi>=0.110.0
|
|
17
|
+
Requires-Dist: uvicorn>=0.27.0
|
|
18
|
+
Requires-Dist: httpx>=0.27.0
|
|
19
|
+
Requires-Dist: tiktoken>=0.7.0
|
|
20
|
+
Dynamic: license-file
|
|
21
|
+
|
|
22
|
+
# โก TokenZero
|
|
23
|
+
|
|
24
|
+
**Zero Token Waste for AI Coding Agents (Antigravity, Claude Code, Cursor, Codex).**
|
|
25
|
+
|
|
26
|
+
TokenZero cuts AI coding token consumption by **70% to 96%** without sacrificing intelligence or code accuracy. It acts as an automated AST code skeletonizer, live real-time token monitor, and tool call interceptor.
|
|
27
|
+
|
|
28
|
+
---
|
|
29
|
+
|
|
30
|
+
## ๐ฏ The Problem TokenZero Solves
|
|
31
|
+
|
|
32
|
+
When AI agents inspect codebases, they routinely load entire 500โ2,000 line source files just to read a single 15-line function.
|
|
33
|
+
- **Without TokenZero**: Reading 15 functions across 5 files consumes **~40,000โ60,000 tokens**.
|
|
34
|
+
- **With TokenZero**: Agents inspect the AST skeletal map and execute targeted partial line-range reads (`StartLine` & `EndLine`), consuming **~2,500โ5,000 tokens** (an **85โ95% direct reduction**).
|
|
35
|
+
|
|
36
|
+
---
|
|
37
|
+
|
|
38
|
+
## โจ Features
|
|
39
|
+
|
|
40
|
+
- **๐ Multi-Language AST Skeletonizer**:
|
|
41
|
+
- Python (`.py` via Python AST)
|
|
42
|
+
- JavaScript & TypeScript (`.js`, `.jsx`, `.ts`, `.tsx`, `.mjs`, `.cjs` via structural brace-tracking parser)
|
|
43
|
+
- Go (`.go`)
|
|
44
|
+
- **โก Real-Time Lifecycle Hooks (<2ms)**:
|
|
45
|
+
- Intercepts agent tool calls (`read_file`, `edit_file`, `run_command`) on-the-fly via `.agents/hooks.json`.
|
|
46
|
+
- Blocks automated edit popups from IDE diff viewers.
|
|
47
|
+
- Zero performance overhead.
|
|
48
|
+
- **๐ฅ๏ธ Live Activity Dashboard (`tokenzero watch`)**:
|
|
49
|
+
- Pure terminal table view showing each file inspection, function targeted, lines read, and **Tokens Used**.
|
|
50
|
+
- **ROI & Savings Comparison**: Compares unoptimized token consumption against TokenZero usage and displays exact dollar savings.
|
|
51
|
+
- **๐งน Universal Clean Removal (`tokenzero remove`)**:
|
|
52
|
+
- One-command uninstaller that restores the project to its exact original state.
|
|
53
|
+
- **๐๏ธ Standalone Compression Proxy (`tokenzero proxy`)**:
|
|
54
|
+
- OpenAI-compatible local HTTP proxy for automated prompt payload compression.
|
|
55
|
+
|
|
56
|
+
---
|
|
57
|
+
|
|
58
|
+
## ๐ Quick Start
|
|
59
|
+
|
|
60
|
+
### 1. Installation
|
|
61
|
+
|
|
62
|
+
Install globally using `pipx` or standard `pip`:
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
# Recommended: Global CLI installation via pipx
|
|
66
|
+
pipx install git+https://github.com/your-username/tokenzero.git
|
|
67
|
+
|
|
68
|
+
# Or install from source:
|
|
69
|
+
git clone https://github.com/your-username/tokenzero.git
|
|
70
|
+
cd tokenzero
|
|
71
|
+
pip install -e .
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
### 2. One-Time Universal Setup
|
|
75
|
+
|
|
76
|
+
Automatically configure **Google Antigravity**, **Cursor IDE**, **Claude Desktop**, and **Claude Code** with 1 command:
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
tokenzero setup
|
|
80
|
+
```
|
|
81
|
+
*(Note: If you skip this, running `tokenzero init` or `tokenzero watch` in any project will also auto-configure all your installed AI agents silently on first run!)*
|
|
82
|
+
|
|
83
|
+
TokenZero registers the Model Context Protocol (MCP) server across your system. **Zero files are added to your project directories!**
|
|
84
|
+
|
|
85
|
+
### 3. Initialize in Any Project (Zero-File Mode)
|
|
86
|
+
|
|
87
|
+
Navigate to any codebase (Python, React/Node.js, TypeScript, Go):
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
cd /path/to/your/project
|
|
91
|
+
tokenzero init
|
|
92
|
+
```
|
|
93
|
+
|
|
94
|
+
TokenZero indexes the AST code skeleton into your global cache (`~/.tokenzero/cache/`). Your Git repository remains **100% clean with zero untracked files**!
|
|
95
|
+
|
|
96
|
+
*(Optional: Use `tokenzero init --local` if you prefer repository-specific rules with auto `.gitignore`).*
|
|
97
|
+
|
|
98
|
+
### 4. Live Monitor
|
|
99
|
+
|
|
100
|
+
Open a second terminal or split pane and run:
|
|
101
|
+
|
|
102
|
+
```bash
|
|
103
|
+
tokenzero watch
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
Watch live tool calls stream in real time as your AI agent inspects and edits code!
|
|
107
|
+
|
|
108
|
+
```bash
|
|
109
|
+
tokenzero watch
|
|
110
|
+
```
|
|
111
|
+
|
|
112
|
+
Watch live tool calls stream in real time as your AI agent inspects and edits code!
|
|
113
|
+
|
|
114
|
+
---
|
|
115
|
+
|
|
116
|
+
## ๐ Live Dashboard Preview
|
|
117
|
+
|
|
118
|
+
```text
|
|
119
|
+
โโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโโณโโโโโโโโโโโโโโโโโโ
|
|
120
|
+
โ Time โ Action โ File & Function โ Lines Read โ Tokens Used โ
|
|
121
|
+
โกโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฉ
|
|
122
|
+
โ 2026-11-15 23:11:58 โ READ โ src/context/AppContext.jsx:L1-L100 โ 100 / 749 lines โ ~700 tokens โ
|
|
123
|
+
โ 2026-11-15 23:12:01 โ EXEC โ Check line count of initialData:wc -l โ - โ - โ
|
|
124
|
+
โ 2026-11-15 23:12:11 โ READ โ src/data/initialData.js:L1-L80 โ 80 / 330 lines โ ~560 tokens โ
|
|
125
|
+
โ 2026-11-15 23:12:14 โ READ โ src/data/initialData.js:L81-L160 โ 80 / 330 lines โ ~560 tokens โ
|
|
126
|
+
โ 2026-11-15 23:12:44 โ READ โ ASSIGNMENT_EXECUTION_PLAN.md:L1-L80 โ 80 / 150 lines โ ~560 tokens โ
|
|
127
|
+
โ 2026-11-15 23:12:49 โ READ โ WRITEUP_2_PAGES.md:L1-L60 โ 60 / 161 lines โ ~420 tokens โ
|
|
128
|
+
โโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโโดโโโโโโโโโโโโโโโโโโ
|
|
129
|
+
|
|
130
|
+
โญโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ ๐ TokenZero ROI & Savings Comparison โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฎ
|
|
131
|
+
โ Activity: 28 events across 18 files | ๐ด Actual Cost (Without TokenZero): 29,534 tokens ($0.0738) โ โ
|
|
132
|
+
โ ๐ข Used (With TokenZero): 8,064 tokens ($0.0202) โ
|
|
133
|
+
โ ๐ฐ Net Impact: Saved +21,470 tokens (72.7% reduction) โข You Saved $0.0537 USD! โ
|
|
134
|
+
โฐโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโฏ
|
|
135
|
+
```
|
|
136
|
+
|
|
137
|
+
---
|
|
138
|
+
|
|
139
|
+
## ๐ ๏ธ CLI Reference
|
|
140
|
+
|
|
141
|
+
| Command | Description |
|
|
142
|
+
| :--- | :--- |
|
|
143
|
+
| `tokenzero init [path]` | Scans codebase, generates AST repo map, and sets up real-time hooks across Antigravity, Claude, Cursor, Windsurf, Copilot. |
|
|
144
|
+
| `tokenzero watch [path]` | Starts live real-time terminal monitor for AI tool events. |
|
|
145
|
+
| `tokenzero stats [path] [--once]` | Prints summary audit report and savings metrics. |
|
|
146
|
+
| `tokenzero mcp` | Starts Model Context Protocol (MCP) stdio server for native AI tool calling. |
|
|
147
|
+
| `tokenzero remove [path]` | Completely cleans up `.tokenzero/`, rules, and hooks from the repo. |
|
|
148
|
+
| `tokenzero proxy` | Starts local FastAPI token optimization proxy on `http://127.0.0.1:8080`. |
|
|
149
|
+
|
|
150
|
+
*(Note: `tokensqueeze` is also retained as a backwards-compatible alias for all commands).*
|
|
151
|
+
|
|
152
|
+
---
|
|
153
|
+
|
|
154
|
+
## ๐ Model Context Protocol (MCP) Integration
|
|
155
|
+
|
|
156
|
+
TokenZero supports the standard Model Context Protocol (MCP). To connect TokenZero directly to **Claude Code**, **Claude Desktop**, or **Cursor**:
|
|
157
|
+
|
|
158
|
+
Add to your `claude_desktop_config.json` or `.cursor/mcp.json`:
|
|
159
|
+
|
|
160
|
+
```json
|
|
161
|
+
{
|
|
162
|
+
"mcpServers": {
|
|
163
|
+
"tokenzero": {
|
|
164
|
+
"command": "tokenzero",
|
|
165
|
+
"args": ["mcp"]
|
|
166
|
+
}
|
|
167
|
+
}
|
|
168
|
+
}
|
|
169
|
+
```
|
|
170
|
+
|
|
171
|
+
Now your AI assistant can invoke:
|
|
172
|
+
- `tokenzero_get_repo_map`: Instant AST symbol inspection before reading code.
|
|
173
|
+
- `tokenzero_read_lines`: Target specific line numbers (`start_line` to `end_line`), saving 80โ95% tokens.
|
|
174
|
+
- `tokenzero_get_stats`: Live savings metrics and ROI tracking.
|
|
175
|
+
|
|
176
|
+
---
|
|
177
|
+
|
|
178
|
+
## ๐ License
|
|
179
|
+
|
|
180
|
+
MIT License. Free and open source for all developers and teams.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
mcp_server.py,sha256=00epxkhL_0dyQvY5Guy7H7ok7pdx1MZ0msWRx2HGpAU,11692
|
|
2
|
+
proxy.py,sha256=IfA7Q0ZAfM56Wxt1C-Metf28dyuOXr5v0xkOJGN6yNY,8036
|
|
3
|
+
scanner.py,sha256=gmiwio7J5Nk0wAZaZgU7bW21eo5K7OcG_Yl5ovTqrJE,13119
|
|
4
|
+
squeeze.py,sha256=s9wOttarBsl_qqISOKxI7xqQmn0AcED_ULgeG7vP9fs,39995
|
|
5
|
+
tokenzero-0.1.0.dist-info/licenses/LICENSE,sha256=eeJplWLBC9igrUm4Kz87MFLm38pBWsjWDuN-aEJdrk8,1080
|
|
6
|
+
tokenzero-0.1.0.dist-info/METADATA,sha256=2Sv_lO4CdS93I_MtVIypzN51Rti1uRxHB8wArF2TxVw,8684
|
|
7
|
+
tokenzero-0.1.0.dist-info/WHEEL,sha256=mOdXR9C3wMVov7cSY8PY-XCzo3SmmWCHvGQr5csJoao,91
|
|
8
|
+
tokenzero-0.1.0.dist-info/entry_points.txt,sha256=T9CdyRDvN2jDqb_KYRAr8lK5CBf7RYoqw72cELuPjV8,71
|
|
9
|
+
tokenzero-0.1.0.dist-info/top_level.txt,sha256=xZAsuFXPMmRPQK3rY_vpUdHmK7OFk0HBiBSLyHcSRd0,33
|
|
10
|
+
tokenzero-0.1.0.dist-info/RECORD,,
|
|
@@ -0,0 +1,22 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 TokenZero Contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
|
22
|
+
|