token-guardian 0.1.0b1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- token_guardian-0.1.0b1/.gitignore +9 -0
- token_guardian-0.1.0b1/PKG-INFO +217 -0
- token_guardian-0.1.0b1/README.md +197 -0
- token_guardian-0.1.0b1/app/__init__.py +1 -0
- token_guardian-0.1.0b1/app/cli.py +638 -0
- token_guardian-0.1.0b1/app/models/__init__.py +2 -0
- token_guardian-0.1.0b1/app/models/schemas.py +101 -0
- token_guardian-0.1.0b1/app/providers/__init__.py +2 -0
- token_guardian-0.1.0b1/app/providers/base.py +15 -0
- token_guardian-0.1.0b1/app/providers/catalog.py +61 -0
- token_guardian-0.1.0b1/app/providers/registry.py +23 -0
- token_guardian-0.1.0b1/app/services/__init__.py +2 -0
- token_guardian-0.1.0b1/app/services/analyzer_service.py +52 -0
- token_guardian-0.1.0b1/app/services/database.py +58 -0
- token_guardian-0.1.0b1/app/services/llm_gateway_service.py +104 -0
- token_guardian-0.1.0b1/app/services/metrics_service.py +78 -0
- token_guardian-0.1.0b1/app/services/review_service.py +70 -0
- token_guardian-0.1.0b1/app/services/review_session_service.py +124 -0
- token_guardian-0.1.0b1/app/utils/__init__.py +2 -0
- token_guardian-0.1.0b1/app/utils/prompt_analysis.py +84 -0
- token_guardian-0.1.0b1/app/utils/token_estimator.py +24 -0
- token_guardian-0.1.0b1/docs/assets/token-guardian-cli.svg +11 -0
- token_guardian-0.1.0b1/docs/integration-strategy.md +51 -0
- token_guardian-0.1.0b1/docs/quickstart.md +71 -0
- token_guardian-0.1.0b1/docs/release-checklist.md +35 -0
- token_guardian-0.1.0b1/pyproject.toml +74 -0
- token_guardian-0.1.0b1/tests/__init__.py +1 -0
- token_guardian-0.1.0b1/tests/conftest.py +14 -0
- token_guardian-0.1.0b1/tests/test_analyzer_service.py +65 -0
- token_guardian-0.1.0b1/tests/test_cli.py +486 -0
- token_guardian-0.1.0b1/tests/test_llm_gateway_helpers.py +44 -0
- token_guardian-0.1.0b1/tests/test_llm_gateway_service.py +89 -0
- token_guardian-0.1.0b1/tests/test_metrics_service.py +18 -0
- token_guardian-0.1.0b1/tests/test_prompt_analysis.py +34 -0
- token_guardian-0.1.0b1/tests/test_review_session_service.py +48 -0
- token_guardian-0.1.0b1/tests/test_token_estimator.py +10 -0
|
@@ -0,0 +1,217 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: token-guardian
|
|
3
|
+
Version: 0.1.0b1
|
|
4
|
+
Summary: CLI-first guardrail for prompt token, cost, and context observability.
|
|
5
|
+
Author: Token Guardian Contributors
|
|
6
|
+
License: MIT
|
|
7
|
+
Requires-Python: >=3.12
|
|
8
|
+
Requires-Dist: inquirerpy<1.0.0,>=0.3.4
|
|
9
|
+
Requires-Dist: pydantic<3.0.0,>=2.8.0
|
|
10
|
+
Requires-Dist: rich<14.0.0,>=13.9.0
|
|
11
|
+
Provides-Extra: dev
|
|
12
|
+
Requires-Dist: black<25.0.0,>=24.8.0; extra == 'dev'
|
|
13
|
+
Requires-Dist: mypy<2.0.0,>=1.11.0; extra == 'dev'
|
|
14
|
+
Requires-Dist: pytest-cov<6.0.0,>=5.0.0; extra == 'dev'
|
|
15
|
+
Requires-Dist: pytest<9.0.0,>=8.3.0; extra == 'dev'
|
|
16
|
+
Requires-Dist: ruff<1.0.0,>=0.6.0; extra == 'dev'
|
|
17
|
+
Provides-Extra: gateway
|
|
18
|
+
Requires-Dist: litellm<2.0.0,>=1.51.0; extra == 'gateway'
|
|
19
|
+
Description-Content-Type: text/markdown
|
|
20
|
+
|
|
21
|
+
# Token Guardian
|
|
22
|
+
|
|
23
|
+
Token Guardian is a CLI-first preflight guardrail for LLM prompts.
|
|
24
|
+
|
|
25
|
+
```text
|
|
26
|
+
_______ _ _____ _ _
|
|
27
|
+
|__ __| | | / ____| | (_)
|
|
28
|
+
| | ___ | | _____ _ __ | | __ _ _ __ _ _ __ __| |_ __ _ _ __
|
|
29
|
+
| |/ _ \| |/ / _ \ '_ \ | | |_ | | | |/ _` | '__/ _` | |/ _` | '_ \
|
|
30
|
+
| | (_) | < __/ | | | | |__| | |_| | (_| | | | (_| | | (_| | | | |
|
|
31
|
+
|_|\___/|_|\_\___|_| |_| \_____|\__,_|\__,_|_| \__,_|_|\__,_|_| |_|
|
|
32
|
+
|
|
33
|
+
[ shielded prompt preflight ]
|
|
34
|
+
```
|
|
35
|
+
|
|
36
|
+
Before any real model call, it shows:
|
|
37
|
+
|
|
38
|
+
- estimated tokens
|
|
39
|
+
- estimated cost
|
|
40
|
+
- context usage
|
|
41
|
+
- context risk
|
|
42
|
+
- prompt reduction hints
|
|
43
|
+
|
|
44
|
+
Then it shows the analysis before the execution flow continues.
|
|
45
|
+
|
|
46
|
+
## Preview
|
|
47
|
+
|
|
48
|
+
Token Guardian opens with an interactive terminal flow that lets developers choose a host profile, select a model, submit a prompt, and inspect the preflight analysis before any real LLM call.
|
|
49
|
+
|
|
50
|
+
## Release Status
|
|
51
|
+
|
|
52
|
+
Current release: `0.1.0-beta.1`
|
|
53
|
+
|
|
54
|
+
This project is ready for early adopters who want to test a prompt guardrail in real workflows. It is not yet positioned as a fully hardened enterprise product.
|
|
55
|
+
|
|
56
|
+
## What It Is
|
|
57
|
+
|
|
58
|
+
Token Guardian helps developers inspect a prompt before it reaches an LLM.
|
|
59
|
+
|
|
60
|
+
It is designed to answer three questions quickly:
|
|
61
|
+
|
|
62
|
+
- how many tokens this prompt is likely to use
|
|
63
|
+
- how much this prompt may cost
|
|
64
|
+
- whether this prompt is too large, noisy, or risky for the selected model context
|
|
65
|
+
|
|
66
|
+
## Who It Is For
|
|
67
|
+
|
|
68
|
+
- developers using LLMs in daily coding workflows
|
|
69
|
+
- teams that want a simple prompt review step before execution
|
|
70
|
+
- open-source users who want a local, cross-platform CLI
|
|
71
|
+
|
|
72
|
+
## What It Does Not Do
|
|
73
|
+
|
|
74
|
+
- it does not intercept every editor chat automatically
|
|
75
|
+
- it does not enforce organization-wide policy remotely
|
|
76
|
+
- it does not replace provider-side safety systems
|
|
77
|
+
- it does not depend on MCP or VS Code integration as a product requirement
|
|
78
|
+
|
|
79
|
+
## Why CLI First
|
|
80
|
+
|
|
81
|
+
This is the most reliable way to enforce a real preflight gate:
|
|
82
|
+
|
|
83
|
+
- it works on Windows, macOS, and Linux
|
|
84
|
+
- it does not depend on editor-specific chat hooks
|
|
85
|
+
- it always shows the analysis before any real LLM call
|
|
86
|
+
- it is easy to share with other developers
|
|
87
|
+
|
|
88
|
+
## Install
|
|
89
|
+
|
|
90
|
+
### Windows
|
|
91
|
+
|
|
92
|
+
```powershell
|
|
93
|
+
powershell -ExecutionPolicy Bypass -File scripts\install.ps1
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
### macOS / Linux
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
bash scripts/install.sh
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
## Run
|
|
103
|
+
|
|
104
|
+
### Windows
|
|
105
|
+
|
|
106
|
+
```powershell
|
|
107
|
+
powershell -File scripts\run-cli.ps1
|
|
108
|
+
```
|
|
109
|
+
|
|
110
|
+
### macOS / Linux
|
|
111
|
+
|
|
112
|
+
```bash
|
|
113
|
+
bash scripts/run-cli.sh
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
You can also run the command directly:
|
|
117
|
+
|
|
118
|
+
```bash
|
|
119
|
+
token-guardian
|
|
120
|
+
```
|
|
121
|
+
|
|
122
|
+
## What the menu does
|
|
123
|
+
|
|
124
|
+
When the CLI opens, you get a simple menu:
|
|
125
|
+
|
|
126
|
+
1. revisar prompt
|
|
127
|
+
2. revisar prompt e enviar para LLM
|
|
128
|
+
3. ver metricas
|
|
129
|
+
4. listar modelos suportados
|
|
130
|
+
5. sair
|
|
131
|
+
|
|
132
|
+
The intended flow is:
|
|
133
|
+
|
|
134
|
+
1. choose the host profile such as `Claude Code`, `Codex`, or `GitHub Copilot`
|
|
135
|
+
2. choose a suggested model for that profile
|
|
136
|
+
3. paste the prompt and press `Enter`
|
|
137
|
+
4. inspect the markdown analysis
|
|
138
|
+
5. if you are in execution mode, Token Guardian continues to the LLM automatically
|
|
139
|
+
|
|
140
|
+
In the interactive terminal flow, selections use `espaco` to mark and `Enter` to confirm.
|
|
141
|
+
|
|
142
|
+
Prompt UX in the current menu:
|
|
143
|
+
|
|
144
|
+
- the prompt is entered in a single step
|
|
145
|
+
- `Enter` sends the prompt
|
|
146
|
+
- output token estimation is automatic in the interactive flow
|
|
147
|
+
|
|
148
|
+
## Main commands
|
|
149
|
+
|
|
150
|
+
The interactive menu is the default, but advanced commands are still available:
|
|
151
|
+
|
|
152
|
+
```bash
|
|
153
|
+
token-guardian
|
|
154
|
+
token-guardian menu
|
|
155
|
+
token-guardian models
|
|
156
|
+
token-guardian metrics
|
|
157
|
+
token-guardian review --provider anthropic --model claude-sonnet-4 --prompt "Revise esta arquitetura."
|
|
158
|
+
token-guardian run --provider anthropic --model claude-sonnet-4 --prompt "Revise esta arquitetura."
|
|
159
|
+
```
|
|
160
|
+
|
|
161
|
+
## Real LLM invocation
|
|
162
|
+
|
|
163
|
+
If you want Token Guardian to call the provider after the analysis step, install the gateway extra:
|
|
164
|
+
|
|
165
|
+
```bash
|
|
166
|
+
pip install -e .[gateway]
|
|
167
|
+
```
|
|
168
|
+
|
|
169
|
+
Then use:
|
|
170
|
+
|
|
171
|
+
```bash
|
|
172
|
+
token-guardian run --provider anthropic --model claude-sonnet-4 --prompt "Revise esta arquitetura."
|
|
173
|
+
```
|
|
174
|
+
|
|
175
|
+
The `run` flow analyzes first and then calls the LLM directly.
|
|
176
|
+
|
|
177
|
+
## Metrics and observability
|
|
178
|
+
|
|
179
|
+
Token Guardian stores local usage data in SQLite.
|
|
180
|
+
|
|
181
|
+
Tracked data includes:
|
|
182
|
+
|
|
183
|
+
- total requests
|
|
184
|
+
- total tokens
|
|
185
|
+
- estimated cumulative cost
|
|
186
|
+
- most used models
|
|
187
|
+
- most used providers
|
|
188
|
+
|
|
189
|
+
Database file:
|
|
190
|
+
|
|
191
|
+
- `token_guardian.db`
|
|
192
|
+
|
|
193
|
+
To inspect metrics:
|
|
194
|
+
|
|
195
|
+
```bash
|
|
196
|
+
token-guardian metrics
|
|
197
|
+
```
|
|
198
|
+
|
|
199
|
+
## Supported models
|
|
200
|
+
|
|
201
|
+
See the live list from the CLI:
|
|
202
|
+
|
|
203
|
+
```bash
|
|
204
|
+
token-guardian models
|
|
205
|
+
```
|
|
206
|
+
|
|
207
|
+
## Developer commands
|
|
208
|
+
|
|
209
|
+
Run the full test suite:
|
|
210
|
+
|
|
211
|
+
```bash
|
|
212
|
+
pytest
|
|
213
|
+
```
|
|
214
|
+
|
|
215
|
+
## License
|
|
216
|
+
|
|
217
|
+
MIT
|
|
@@ -0,0 +1,197 @@
|
|
|
1
|
+
# Token Guardian
|
|
2
|
+
|
|
3
|
+
Token Guardian is a CLI-first preflight guardrail for LLM prompts.
|
|
4
|
+
|
|
5
|
+
```text
|
|
6
|
+
_______ _ _____ _ _
|
|
7
|
+
|__ __| | | / ____| | (_)
|
|
8
|
+
| | ___ | | _____ _ __ | | __ _ _ __ _ _ __ __| |_ __ _ _ __
|
|
9
|
+
| |/ _ \| |/ / _ \ '_ \ | | |_ | | | |/ _` | '__/ _` | |/ _` | '_ \
|
|
10
|
+
| | (_) | < __/ | | | | |__| | |_| | (_| | | | (_| | | (_| | | | |
|
|
11
|
+
|_|\___/|_|\_\___|_| |_| \_____|\__,_|\__,_|_| \__,_|_|\__,_|_| |_|
|
|
12
|
+
|
|
13
|
+
[ shielded prompt preflight ]
|
|
14
|
+
```
|
|
15
|
+
|
|
16
|
+
Before any real model call, it shows:
|
|
17
|
+
|
|
18
|
+
- estimated tokens
|
|
19
|
+
- estimated cost
|
|
20
|
+
- context usage
|
|
21
|
+
- context risk
|
|
22
|
+
- prompt reduction hints
|
|
23
|
+
|
|
24
|
+
Then it shows the analysis before the execution flow continues.
|
|
25
|
+
|
|
26
|
+
## Preview
|
|
27
|
+
|
|
28
|
+
Token Guardian opens with an interactive terminal flow that lets developers choose a host profile, select a model, submit a prompt, and inspect the preflight analysis before any real LLM call.
|
|
29
|
+
|
|
30
|
+
## Release Status
|
|
31
|
+
|
|
32
|
+
Current release: `0.1.0-beta.1`
|
|
33
|
+
|
|
34
|
+
This project is ready for early adopters who want to test a prompt guardrail in real workflows. It is not yet positioned as a fully hardened enterprise product.
|
|
35
|
+
|
|
36
|
+
## What It Is
|
|
37
|
+
|
|
38
|
+
Token Guardian helps developers inspect a prompt before it reaches an LLM.
|
|
39
|
+
|
|
40
|
+
It is designed to answer three questions quickly:
|
|
41
|
+
|
|
42
|
+
- how many tokens this prompt is likely to use
|
|
43
|
+
- how much this prompt may cost
|
|
44
|
+
- whether this prompt is too large, noisy, or risky for the selected model context
|
|
45
|
+
|
|
46
|
+
## Who It Is For
|
|
47
|
+
|
|
48
|
+
- developers using LLMs in daily coding workflows
|
|
49
|
+
- teams that want a simple prompt review step before execution
|
|
50
|
+
- open-source users who want a local, cross-platform CLI
|
|
51
|
+
|
|
52
|
+
## What It Does Not Do
|
|
53
|
+
|
|
54
|
+
- it does not intercept every editor chat automatically
|
|
55
|
+
- it does not enforce organization-wide policy remotely
|
|
56
|
+
- it does not replace provider-side safety systems
|
|
57
|
+
- it does not depend on MCP or VS Code integration as a product requirement
|
|
58
|
+
|
|
59
|
+
## Why CLI First
|
|
60
|
+
|
|
61
|
+
This is the most reliable way to enforce a real preflight gate:
|
|
62
|
+
|
|
63
|
+
- it works on Windows, macOS, and Linux
|
|
64
|
+
- it does not depend on editor-specific chat hooks
|
|
65
|
+
- it always shows the analysis before any real LLM call
|
|
66
|
+
- it is easy to share with other developers
|
|
67
|
+
|
|
68
|
+
## Install
|
|
69
|
+
|
|
70
|
+
### Windows
|
|
71
|
+
|
|
72
|
+
```powershell
|
|
73
|
+
powershell -ExecutionPolicy Bypass -File scripts\install.ps1
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
### macOS / Linux
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
bash scripts/install.sh
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
## Run
|
|
83
|
+
|
|
84
|
+
### Windows
|
|
85
|
+
|
|
86
|
+
```powershell
|
|
87
|
+
powershell -File scripts\run-cli.ps1
|
|
88
|
+
```
|
|
89
|
+
|
|
90
|
+
### macOS / Linux
|
|
91
|
+
|
|
92
|
+
```bash
|
|
93
|
+
bash scripts/run-cli.sh
|
|
94
|
+
```
|
|
95
|
+
|
|
96
|
+
You can also run the command directly:
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
token-guardian
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
## What the menu does
|
|
103
|
+
|
|
104
|
+
When the CLI opens, you get a simple menu:
|
|
105
|
+
|
|
106
|
+
1. revisar prompt
|
|
107
|
+
2. revisar prompt e enviar para LLM
|
|
108
|
+
3. ver metricas
|
|
109
|
+
4. listar modelos suportados
|
|
110
|
+
5. sair
|
|
111
|
+
|
|
112
|
+
The intended flow is:
|
|
113
|
+
|
|
114
|
+
1. choose the host profile such as `Claude Code`, `Codex`, or `GitHub Copilot`
|
|
115
|
+
2. choose a suggested model for that profile
|
|
116
|
+
3. paste the prompt and press `Enter`
|
|
117
|
+
4. inspect the markdown analysis
|
|
118
|
+
5. if you are in execution mode, Token Guardian continues to the LLM automatically
|
|
119
|
+
|
|
120
|
+
In the interactive terminal flow, selections use `espaco` to mark and `Enter` to confirm.
|
|
121
|
+
|
|
122
|
+
Prompt UX in the current menu:
|
|
123
|
+
|
|
124
|
+
- the prompt is entered in a single step
|
|
125
|
+
- `Enter` sends the prompt
|
|
126
|
+
- output token estimation is automatic in the interactive flow
|
|
127
|
+
|
|
128
|
+
## Main commands
|
|
129
|
+
|
|
130
|
+
The interactive menu is the default, but advanced commands are still available:
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
token-guardian
|
|
134
|
+
token-guardian menu
|
|
135
|
+
token-guardian models
|
|
136
|
+
token-guardian metrics
|
|
137
|
+
token-guardian review --provider anthropic --model claude-sonnet-4 --prompt "Revise esta arquitetura."
|
|
138
|
+
token-guardian run --provider anthropic --model claude-sonnet-4 --prompt "Revise esta arquitetura."
|
|
139
|
+
```
|
|
140
|
+
|
|
141
|
+
## Real LLM invocation
|
|
142
|
+
|
|
143
|
+
If you want Token Guardian to call the provider after the analysis step, install the gateway extra:
|
|
144
|
+
|
|
145
|
+
```bash
|
|
146
|
+
pip install -e .[gateway]
|
|
147
|
+
```
|
|
148
|
+
|
|
149
|
+
Then use:
|
|
150
|
+
|
|
151
|
+
```bash
|
|
152
|
+
token-guardian run --provider anthropic --model claude-sonnet-4 --prompt "Revise esta arquitetura."
|
|
153
|
+
```
|
|
154
|
+
|
|
155
|
+
The `run` flow analyzes first and then calls the LLM directly.
|
|
156
|
+
|
|
157
|
+
## Metrics and observability
|
|
158
|
+
|
|
159
|
+
Token Guardian stores local usage data in SQLite.
|
|
160
|
+
|
|
161
|
+
Tracked data includes:
|
|
162
|
+
|
|
163
|
+
- total requests
|
|
164
|
+
- total tokens
|
|
165
|
+
- estimated cumulative cost
|
|
166
|
+
- most used models
|
|
167
|
+
- most used providers
|
|
168
|
+
|
|
169
|
+
Database file:
|
|
170
|
+
|
|
171
|
+
- `token_guardian.db`
|
|
172
|
+
|
|
173
|
+
To inspect metrics:
|
|
174
|
+
|
|
175
|
+
```bash
|
|
176
|
+
token-guardian metrics
|
|
177
|
+
```
|
|
178
|
+
|
|
179
|
+
## Supported models
|
|
180
|
+
|
|
181
|
+
See the live list from the CLI:
|
|
182
|
+
|
|
183
|
+
```bash
|
|
184
|
+
token-guardian models
|
|
185
|
+
```
|
|
186
|
+
|
|
187
|
+
## Developer commands
|
|
188
|
+
|
|
189
|
+
Run the full test suite:
|
|
190
|
+
|
|
191
|
+
```bash
|
|
192
|
+
pytest
|
|
193
|
+
```
|
|
194
|
+
|
|
195
|
+
## License
|
|
196
|
+
|
|
197
|
+
MIT
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
"""Token Guardian application package."""
|