modelproof 1.0.0__tar.gz → 1.0.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- modelproof-1.0.1/PKG-INFO +218 -0
- modelproof-1.0.1/README.id.md +194 -0
- modelproof-1.0.1/README.md +193 -0
- {modelproof-1.0.0 → modelproof-1.0.1}/app.js +28 -28
- {modelproof-1.0.0 → modelproof-1.0.1}/bin/modelproof.js +4 -4
- modelproof-1.0.1/favicon.svg +5 -0
- {modelproof-1.0.0 → modelproof-1.0.1}/index.html +32 -12
- {modelproof-1.0.0 → modelproof-1.0.1}/modelproof/__init__.py +1 -1
- {modelproof-1.0.0 → modelproof-1.0.1}/modelproof/cli.py +3 -3
- {modelproof-1.0.0 → modelproof-1.0.1}/package.json +3 -2
- {modelproof-1.0.0 → modelproof-1.0.1}/pyproject.toml +1 -1
- modelproof-1.0.0/PKG-INFO +0 -111
- modelproof-1.0.0/README.md +0 -86
- {modelproof-1.0.0 → modelproof-1.0.1}/.gitignore +0 -0
- {modelproof-1.0.0 → modelproof-1.0.1}/LICENSE +0 -0
- {modelproof-1.0.0 → modelproof-1.0.1}/server.py +0 -0
|
@@ -0,0 +1,218 @@
|
|
|
1
|
+
Metadata-Version: 2.5
|
|
2
|
+
Name: modelproof
|
|
3
|
+
Version: 1.0.1
|
|
4
|
+
Summary: Zero-persistence client-side and CLI forensic scanner to detect LLM model spoofing, masking, and downgrading.
|
|
5
|
+
Project-URL: Homepage, https://github.com/siumik/ai-model-mask-checker
|
|
6
|
+
Project-URL: Repository, https://github.com/siumik/ai-model-mask-checker
|
|
7
|
+
Author: ModelProof Contributors
|
|
8
|
+
License-Expression: MIT
|
|
9
|
+
License-File: LICENSE
|
|
10
|
+
Keywords: ai,anthropic,benchmark,deepseek,forensics,llm,model-spoofing,openai,qwen,reverse-proxy,security
|
|
11
|
+
Classifier: Development Status :: 5 - Production/Stable
|
|
12
|
+
Classifier: Intended Audience :: Developers
|
|
13
|
+
Classifier: License :: OSI Approved :: MIT License
|
|
14
|
+
Classifier: Programming Language :: Python :: 3
|
|
15
|
+
Classifier: Programming Language :: Python :: 3.8
|
|
16
|
+
Classifier: Programming Language :: Python :: 3.9
|
|
17
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
18
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
21
|
+
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
22
|
+
Classifier: Topic :: Security
|
|
23
|
+
Requires-Python: >=3.8
|
|
24
|
+
Description-Content-Type: text/markdown
|
|
25
|
+
|
|
26
|
+
<p align="center">
|
|
27
|
+
<img src="favicon.svg" width="64" height="64" alt="ModelProof Logo" />
|
|
28
|
+
</p>
|
|
29
|
+
|
|
30
|
+
<h1 align="center">ModelProof</h1>
|
|
31
|
+
|
|
32
|
+
<p align="center">
|
|
33
|
+
<strong>Zero-persistence client-side and CLI forensic scanner to detect LLM model spoofing, reverse-proxy masking, and silent downgrades.</strong>
|
|
34
|
+
</p>
|
|
35
|
+
|
|
36
|
+
<p align="center">
|
|
37
|
+
<a href="https://www.npmjs.com/package/modelproof"><img src="https://img.shields.io/npm/v/modelproof.svg?color=10b981&label=npm" alt="npm version" /></a>
|
|
38
|
+
<a href="https://pypi.org/project/modelproof/"><img src="https://img.shields.io/pypi/v/modelproof.svg?color=10b981&label=pypi" alt="pypi version" /></a>
|
|
39
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License" /></a>
|
|
40
|
+
<img src="https://img.shields.io/badge/zero-telemetry-emerald" alt="Zero Telemetry" />
|
|
41
|
+
<img src="https://img.shields.io/badge/dependencies-0-brightgreen" alt="Zero Dependencies" />
|
|
42
|
+
</p>
|
|
43
|
+
|
|
44
|
+
<p align="center">
|
|
45
|
+
<strong>English</strong> | <a href="README.id.md">Bahasa Indonesia</a>
|
|
46
|
+
</p>
|
|
47
|
+
|
|
48
|
+
---
|
|
49
|
+
|
|
50
|
+
## Overview
|
|
51
|
+
|
|
52
|
+
Unauthorized LLM resellers and reverse-proxy providers often engage in model spoofing:
|
|
53
|
+
- Re-routing expensive requests (e.g. `claude-3-5-sonnet`, `gpt-4o`) to cheaper open-weights models (`qwen-2.5-72b`, `deepseek-v3`, or quantized mini variants).
|
|
54
|
+
- Stripping system instructions and inserting hidden proxy jailbreaks.
|
|
55
|
+
- Fabricating fake model IDs in `/v1/models` catalogs.
|
|
56
|
+
|
|
57
|
+
ModelProof provides developers, QA engineers, and consumers with a verifiable forensic audit trail and an authenticity confidence score (0-100%) before trusting an endpoint in production.
|
|
58
|
+
|
|
59
|
+
---
|
|
60
|
+
|
|
61
|
+
## Why This Matters
|
|
62
|
+
|
|
63
|
+
1. **Financial Arbitrage (Paying Flagship, Receiving Commodity)**:
|
|
64
|
+
Resellers charge premium rates for flagship models while silently routing traffic to commodity models costing a fraction of the price. Users overpay up to 15x for degraded intelligence.
|
|
65
|
+
|
|
66
|
+
2. **Silent Production Breakage**:
|
|
67
|
+
Substituted models fail on nuanced reasoning, strict JSON Schema compliance, tool calling, and high-order logic, causing silent corruption in autonomous agents and backend pipelines.
|
|
68
|
+
|
|
69
|
+
3. **Data Exfiltration and Prompt Manipulation**:
|
|
70
|
+
Untrusted reverse-proxy layers may log raw prompts, inject hidden system preambles, or manipulate token streams before forwarding to unvetted upstream endpoints.
|
|
71
|
+
|
|
72
|
+
4. **Actionable Dispute Evidence**:
|
|
73
|
+
ModelProof generates deterministic, timestamped forensic audit reports that can be used directly as technical proof for refund requests or merchant disputes.
|
|
74
|
+
|
|
75
|
+
---
|
|
76
|
+
|
|
77
|
+
## Distribution Modes
|
|
78
|
+
|
|
79
|
+
| Platform | Distribution | Execution Command | Zero Dependencies |
|
|
80
|
+
| :--- | :--- | :--- | :---: |
|
|
81
|
+
| **Terminal (Node.js)** | NPM Registry | `npx modelproof` | Yes |
|
|
82
|
+
| **Terminal (Python)** | PyPI Package | `pip install modelproof` | Yes |
|
|
83
|
+
| **Browser (Web Sandbox)** | Static Client | `index.html` (Local / Static Host) | Yes |
|
|
84
|
+
|
|
85
|
+
---
|
|
86
|
+
|
|
87
|
+
## 10-Vector Forensic Detection Matrix
|
|
88
|
+
|
|
89
|
+
ModelProof executes 10 adversarial probes calibrated against official model weight checkpoints:
|
|
90
|
+
|
|
91
|
+
| # | Forensic Vector | Target Vulnerability / Anomaly | Mode |
|
|
92
|
+
| :-: | :--- | :--- | :-: |
|
|
93
|
+
| **01** | **Spatial Logic & Character Horizon** | Obfuscated character counting (`'s-t-r-a-w-b-e-r-r-y'`) + arithmetic runtime trap | FAST |
|
|
94
|
+
| **02** | **Tokenizer Usage & BPE Precision** | Multi-byte Unicode sequence testing token discrepancy and upstream token stripping | FAST |
|
|
95
|
+
| **03** | **System Instruction & Identity Leak** | Adversarial system prompt breakout probing true base weights and vendor confessions | FAST |
|
|
96
|
+
| **04** | **Hardware Telemetry & TPS Profile** | Real-time SSE stream parser detecting hyper-fast LPU hardware (>220 TPS SambaNova/Groq) | FAST |
|
|
97
|
+
| **05** | **Negative Constraint Compliance** | Rigid formatting constraints without conversational fluff or apology preambles | FAST |
|
|
98
|
+
| **06** | **Strict Schema / Constrained Decoding** | Enforces native JSON Schema strict parsing (crashes weak proxy engines) | DEEP |
|
|
99
|
+
| **07** | **Glitched Token Embedding Anomaly** | Unspeakable tokens (`SolidGoldMagikarp`) tokenizer behavior checks | FAST |
|
|
100
|
+
| **08** | **Temporal Cutoff Horizon (2024-H2)** | Probes post-training cutoff events (Python 3.13, Nobel Oct 2024) | FAST |
|
|
101
|
+
| **09** | **Reasoning CoT & Delimiter Trap** | Detects OpenAI `o1`/`o3` masked as DeepSeek-R1 via `<think>` token analysis | DEEP |
|
|
102
|
+
| **10** | **Type-Level Memory & Lifetime Logic** | Compiler-level Rust borrow checker & HRTB (`for<'a>`) lifetime reasoning | DEEP |
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## CLI Quickstart
|
|
107
|
+
|
|
108
|
+
Both the Node.js and Python CLIs are 100% zero-dependency, running directly with native system runtimes (`fetch` and standard library `urllib`).
|
|
109
|
+
|
|
110
|
+
### 1. Node.js (NPX)
|
|
111
|
+
|
|
112
|
+
Run directly without installing:
|
|
113
|
+
|
|
114
|
+
```bash
|
|
115
|
+
# Basic scan against custom reverse proxy
|
|
116
|
+
npx modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "claude-3-5-sonnet-20241022"
|
|
117
|
+
|
|
118
|
+
# Deep audit running all 10 vectors with JSON output (CI/CD ready)
|
|
119
|
+
npx modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "gpt-4o" --all --json
|
|
120
|
+
|
|
121
|
+
# Inspect available models from upstream catalog
|
|
122
|
+
npx modelproof -u "https://my-proxy.com/v1" -k "sk-..." --models-only
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
### 2. Python (PIP)
|
|
126
|
+
|
|
127
|
+
Install from PyPI:
|
|
128
|
+
|
|
129
|
+
```bash
|
|
130
|
+
pip install modelproof
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Run audit:
|
|
134
|
+
|
|
135
|
+
```bash
|
|
136
|
+
# Audit model authenticity
|
|
137
|
+
modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "gpt-4o"
|
|
138
|
+
|
|
139
|
+
# Run in Indonesian language
|
|
140
|
+
modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "claude-3-5-sonnet-20241022" --lang id
|
|
141
|
+
|
|
142
|
+
# Run all 10 deep vectors
|
|
143
|
+
modelproof -u "https://my-proxy.com/v1" -k "sk-..." --all
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
Or execute without global install:
|
|
147
|
+
```bash
|
|
148
|
+
python -m modelproof.cli -u "https://my-proxy.com/v1" -k "sk-..." -m "gpt-4o"
|
|
149
|
+
```
|
|
150
|
+
|
|
151
|
+
---
|
|
152
|
+
|
|
153
|
+
## CLI Options & Flags
|
|
154
|
+
|
|
155
|
+
```text
|
|
156
|
+
Usage: modelproof [options]
|
|
157
|
+
|
|
158
|
+
Options:
|
|
159
|
+
-u, --base-url <url> Reverse proxy base URL (default: "https://api.openai.com/v1")
|
|
160
|
+
-k, --key <token> API Token / Key (or set OPENAI_API_KEY env var)
|
|
161
|
+
-m, --model <id> Claimed target model ID (default: "claude-3-5-sonnet-20241022")
|
|
162
|
+
-p, --protocol <proto> Protocol wire schema: auto, openai, anthropic (default: "auto")
|
|
163
|
+
-a, --all Run all 10 deep vectors (default: 8 fast vectors)
|
|
164
|
+
--models-only Audit upstream /v1/models catalog only and exit
|
|
165
|
+
--lang <lang> Output language: en, id (default: "en")
|
|
166
|
+
--json Output pure machine-readable JSON report
|
|
167
|
+
--timeout <sec> Per-request timeout in seconds (default: 30)
|
|
168
|
+
-v, --version Show version number
|
|
169
|
+
-h, --help Display help message
|
|
170
|
+
```
|
|
171
|
+
|
|
172
|
+
---
|
|
173
|
+
|
|
174
|
+
## Sample Terminal Output
|
|
175
|
+
|
|
176
|
+
```text
|
|
177
|
+
================================================================================
|
|
178
|
+
MODELPROOF CLI // LLM Proxy & Masking Forensic Scanner (v1.0.1)
|
|
179
|
+
Target: claude-3-5-sonnet-20241022 @ https://my-custom-proxy.com/v1
|
|
180
|
+
================================================================================
|
|
181
|
+
[*] Protocol Wire Schema: ANTHROPIC
|
|
182
|
+
[*] Catalog Audit: 42 models retrieved (Clean naming).
|
|
183
|
+
|
|
184
|
+
[+] RUNNING 8 FORENSIC VECTORS:
|
|
185
|
+
--------------------------------------------------------------------------------
|
|
186
|
+
[01] Spatial Logic & Character Horizon [PASS] r=3, math=324
|
|
187
|
+
[02] Tokenizer Usage & BPE Precision [PASS] Usage discrepancy within tolerance
|
|
188
|
+
[03] System Instruction & Identity Leak [PASS] Responded as official Claude
|
|
189
|
+
[04] Hardware Telemetry & TPS Profile [PASS] 64 TPS (Realistic datacenter profile)
|
|
190
|
+
[05] Negative Constraint Compliance [PASS] Zero preamble constraint honored
|
|
191
|
+
[06] Strict JSON Schema Decoding [PASS] Schema enforced natively
|
|
192
|
+
[07] Glitched Token Embedding [PASS] Token boundary verified
|
|
193
|
+
[08] Temporal Cutoff Horizon (2024-H2) [PASS] Verified post-2024H2 event awareness
|
|
194
|
+
--------------------------------------------------------------------------------
|
|
195
|
+
|
|
196
|
+
============================= FORENSIC VERDICT =================================
|
|
197
|
+
AUTHENTICITY SCORE : 100%
|
|
198
|
+
VERDICT : VERIFIED GENUINE
|
|
199
|
+
DETECTED VENDOR : Anthropic Claude
|
|
200
|
+
RISK LEVEL : SAFE
|
|
201
|
+
|
|
202
|
+
[+] CONCLUSION: Target verified consistent with genuine foundational weights.
|
|
203
|
+
================================================================================
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
---
|
|
207
|
+
|
|
208
|
+
## Security, Privacy & Compliance
|
|
209
|
+
|
|
210
|
+
- **Zero Telemetry Retention**: ModelProof never logs, collects, or transmits API keys, base URLs, or payload data to any external server.
|
|
211
|
+
- **Client-Side Sandbox**: In the web interface, network traffic travels directly from your browser to your designated proxy endpoint.
|
|
212
|
+
- **In-Memory Volatility**: Credentials exist strictly in runtime volatile memory and are cleared upon termination or reload.
|
|
213
|
+
|
|
214
|
+
---
|
|
215
|
+
|
|
216
|
+
## License
|
|
217
|
+
|
|
218
|
+
Distributed under the [MIT License](LICENSE).
|
|
@@ -0,0 +1,194 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="favicon.svg" width="64" height="64" alt="ModelProof Logo" />
|
|
3
|
+
</p>
|
|
4
|
+
|
|
5
|
+
<h1 align="center">ModelProof</h1>
|
|
6
|
+
|
|
7
|
+
<p align="center">
|
|
8
|
+
<strong>Alat audit forensik berbasis klien dan CLI tanpa persistensi data untuk mendeteksi pemalsuan model LLM, rekayasa proxy (masking), dan downgrade diam-diam.</strong>
|
|
9
|
+
</p>
|
|
10
|
+
|
|
11
|
+
<p align="center">
|
|
12
|
+
<a href="https://www.npmjs.com/package/modelproof"><img src="https://img.shields.io/npm/v/modelproof.svg?color=10b981&label=npm" alt="npm version" /></a>
|
|
13
|
+
<a href="https://pypi.org/project/modelproof/"><img src="https://img.shields.io/pypi/v/modelproof.svg?color=10b981&label=pypi" alt="pypi version" /></a>
|
|
14
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License" /></a>
|
|
15
|
+
<img src="https://img.shields.io/badge/zero-telemetry-emerald" alt="Zero Telemetry" />
|
|
16
|
+
<img src="https://img.shields.io/badge/dependencies-0-brightgreen" alt="Zero Dependencies" />
|
|
17
|
+
</p>
|
|
18
|
+
|
|
19
|
+
<p align="center">
|
|
20
|
+
<a href="README.md">English</a> | <strong>Bahasa Indonesia</strong>
|
|
21
|
+
</p>
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Ringkasan
|
|
26
|
+
|
|
27
|
+
Penjual API murah pihak ketiga, akun sharing, dan penyedia reverse-proxy sering melakukan **model spoofing / masking**:
|
|
28
|
+
- Mengalihkan request model mahal (seperti `claude-3-5-sonnet`, `gpt-4o`) ke model open-weights yang jauh lebih murah (`qwen-2.5-72b`, `deepseek-v3`, atau varian mini terkuantisasi).
|
|
29
|
+
- Menghapus sistem instruksi asli dan menyuntikkan jailbreak/kuota rahasia ke dalam prompt.
|
|
30
|
+
- Memanipulasi katalog `/v1/models` dengan nama-nama model fiktif.
|
|
31
|
+
|
|
32
|
+
**ModelProof** memberikan audit teknis independen dan **Skor Keaslian (0-100%)** yang dapat diverifikasi secara objektif sebelum Anda mempercayai suatu endpoint API untuk aplikasi produksi.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Mengapa Ini Penting?
|
|
37
|
+
|
|
38
|
+
1. **Kerugian Finansial (Beli Flagship, Dapat Model Murahan)**:
|
|
39
|
+
Penjual mematok harga untuk model tier atas resmi, namun secara diam-diam me-routing traffic ke model berbiaya sangat rendah. Pengguna membayar berkali-kali lipat lebih mahal untuk performa kecerdasan yang diturunkan drastis.
|
|
40
|
+
|
|
41
|
+
2. **Kerusakan Pipeline Aplikasi**:
|
|
42
|
+
Model tiruan kerap gagal saat menangani penalaran rumit, validasi JSON Schema ketat, eksekusi function/tool calling, serta context window panjang. Hal ini memicu silent bugs pada agen otonom dan sistem backend.
|
|
43
|
+
|
|
44
|
+
3. **Risiko Kebocoran Data dan Manipulasi Prompt**:
|
|
45
|
+
Server reverse-proxy perantara dapat mencatat isi prompt mentah Anda, menyuntikkan instruksi tersembunyi, atau merekayasa alur token sebelum diteruskan ke server upstream.
|
|
46
|
+
|
|
47
|
+
4. **Bukti Valid untuk Komplain & Refund**:
|
|
48
|
+
ModelProof menyusun bukti forensik berstempel waktu yang dapat disalin langsung sebagai lampiran teknis saat mengajukan komplain atau pengembalian dana kepada penjual.
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## Mode Distribusi
|
|
53
|
+
|
|
54
|
+
| Platform | Distribusi | Perintah Eksekusi | Bebas Dependensi |
|
|
55
|
+
| :--- | :--- | :--- | :---: |
|
|
56
|
+
| **Terminal (Node.js)** | NPM Registry | `npx modelproof` | Ya |
|
|
57
|
+
| **Terminal (Python)** | PyPI Package | `pip install modelproof` | Ya |
|
|
58
|
+
| **Browser (Web Sandbox)** | Klien Statis | `index.html` (Lokal / Hosting Statis) | Ya |
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## Matriks Deteksi 10 Vektor Forensik
|
|
63
|
+
|
|
64
|
+
ModelProof mengeksekusi 10 probe adversarial yang dikalibrasi terhadap bobot arsitektur model resmi:
|
|
65
|
+
|
|
66
|
+
| # | Vektor Forensik | Kerentanan / Anomali Target | Mode |
|
|
67
|
+
| :-: | :--- | :--- | :-: |
|
|
68
|
+
| **01** | **Logika Spasial & Batas Karakter** | Jebakan hitung huruf terselubung (`'s-t-r-a-w-b-e-r-r-y'`) + aritmatika runtime | CEPAT |
|
|
69
|
+
| **02** | **Penggunaan Tokenizer & Presisi BPE** | Sekuens multi-byte Unicode untuk menguji diskrepansi token prompt dan pemotongan upstream | CEPAT |
|
|
70
|
+
| **03** | **Instruksi Sistem & Kebocoran Identitas** | Bypass prompt adversarial untuk mengungkap bobot model asli dan pengakuan vendor | CEPAT |
|
|
71
|
+
| **04** | **Telemetri Hardware & Profil TPS** | Parser SSE streaming real-time guna mendeteksi hardware LPU hiper-cepat (>220 TPS SambaNova/Groq) | CEPAT |
|
|
72
|
+
| **05** | **Kepatuhan Batasan Negatif** | Batasan format kaku tanpa basa-basi pembuka atau kalimat permintaan maaf | CEPAT |
|
|
73
|
+
| **06** | **Skema Ketat / Decoding Terbatas** | Penerapan JSON Schema strict native (gagal pada engine proxy abal-abal) | MENDALAM |
|
|
74
|
+
| **07** | **Anomali Glitched Token Embedding** | Uji perilaku tokenizer pada token tak terucapkan (`SolidGoldMagikarp`) | CEPAT |
|
|
75
|
+
| **08** | **Horizon Cutoff Temporal (2024-H2)** | Uji pengetahuan peristiwa pasca-cutoff 2024 (Python 3.13, Nobel Oktober 2024) | CEPAT |
|
|
76
|
+
| **09** | **Alur CoT Penalaran & Delimiter** | Deteksi OpenAI `o1`/`o3` yang dimasking ke DeepSeek-R1 via analisis token `<think>` | MENDALAM |
|
|
77
|
+
| **10** | **Logika Tipe Tingkat Tinggi & Lifetime** | Borrow checker Rust level-kompiler & penalaran lifetime HRTB (`for<'a>`) | MENDALAM |
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## Panduan Cepat CLI
|
|
82
|
+
|
|
83
|
+
Kedua versi CLI (Node.js dan Python) beroperasi **100% tanpa dependensi eksternal**, memanfaatkan native runtime (`fetch` dan standard library `urllib`).
|
|
84
|
+
|
|
85
|
+
### 1. Node.js (NPX)
|
|
86
|
+
|
|
87
|
+
Dapat dijalankan langsung tanpa instalasi:
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
# Scan standar ke endpoint reverse proxy
|
|
91
|
+
npx modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "claude-3-5-sonnet-20241022"
|
|
92
|
+
|
|
93
|
+
# Audit mendalam (10 vektor) dengan output JSON
|
|
94
|
+
npx modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "gpt-4o" --all --json
|
|
95
|
+
|
|
96
|
+
# Periksa katalog model yang disediakan upstream
|
|
97
|
+
npx modelproof -u "https://my-proxy.com/v1" -k "sk-..." --models-only
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
### 2. Python (PIP)
|
|
101
|
+
|
|
102
|
+
Instal dari PyPI:
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
pip install modelproof
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Jalankan audit:
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
# Audit keaslian model
|
|
112
|
+
modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "gpt-4o"
|
|
113
|
+
|
|
114
|
+
# Jalankan dengan laporan Bahasa Indonesia
|
|
115
|
+
modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "claude-3-5-sonnet-20241022" --lang id
|
|
116
|
+
|
|
117
|
+
# Eksekusi seluruh 10 vektor mendalam
|
|
118
|
+
modelproof -u "https://my-proxy.com/v1" -k "sk-..." --all
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Atau jalankan langsung tanpa instalasi global:
|
|
122
|
+
```bash
|
|
123
|
+
python -m modelproof.cli -u "https://my-proxy.com/v1" -k "sk-..." -m "gpt-4o"
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
---
|
|
127
|
+
|
|
128
|
+
## Parameter & Opsi CLI
|
|
129
|
+
|
|
130
|
+
```text
|
|
131
|
+
Penggunaan: modelproof [opsi]
|
|
132
|
+
|
|
133
|
+
Opsi:
|
|
134
|
+
-u, --base-url <url> Alamat reverse proxy target (default: "https://api.openai.com/v1")
|
|
135
|
+
-k, --key <token> API Token / Key (atau set env var OPENAI_API_KEY)
|
|
136
|
+
-m, --model <id> Model ID target yang diklaim (default: "claude-3-5-sonnet-20241022")
|
|
137
|
+
-p, --protocol <proto> Skema wire protocol: auto, openai, anthropic (default: "auto")
|
|
138
|
+
-a, --all Jalankan seluruh 10 vektor (default: 8 vektor cepat)
|
|
139
|
+
--models-only Audit katalog /v1/models saja lalu keluar
|
|
140
|
+
--lang <lang> Bahasa laporan: en, id (default: "en")
|
|
141
|
+
--json Tampilkan output JSON murni untuk integrasi CI/CD
|
|
142
|
+
--timeout <detik> Batas waktu per request dalam detik (default: 30)
|
|
143
|
+
-v, --version Tampilkan nomor versi
|
|
144
|
+
-h, --help Tampilkan bantuan
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## Contoh Tampilan Terminal
|
|
150
|
+
|
|
151
|
+
```text
|
|
152
|
+
================================================================================
|
|
153
|
+
MODELPROOF CLI // LLM Proxy & Masking Forensic Scanner (v1.0.1)
|
|
154
|
+
Target: claude-3-5-sonnet-20241022 @ https://my-custom-proxy.com/v1
|
|
155
|
+
================================================================================
|
|
156
|
+
[*] Protocol Wire Schema: ANTHROPIC
|
|
157
|
+
[*] Catalog Audit: 42 models retrieved (Clean naming).
|
|
158
|
+
|
|
159
|
+
[+] RUNNING 8 FORENSIC VECTORS:
|
|
160
|
+
--------------------------------------------------------------------------------
|
|
161
|
+
[01] Spatial Logic & Character Horizon [PASS] r=3, math=324
|
|
162
|
+
[02] Tokenizer Usage & BPE Precision [PASS] Usage discrepancy within tolerance
|
|
163
|
+
[03] System Instruction & Identity Leak [PASS] Responded as official Claude
|
|
164
|
+
[04] Hardware Telemetry & TPS Profile [PASS] 64 TPS (Realistic datacenter profile)
|
|
165
|
+
[05] Negative Constraint Compliance [PASS] Zero preamble constraint honored
|
|
166
|
+
[06] Strict JSON Schema Decoding [PASS] Schema enforced natively
|
|
167
|
+
[07] Glitched Token Embedding [PASS] Token boundary verified
|
|
168
|
+
[08] Temporal Cutoff Horizon (2024-H2) [PASS] Verified post-2024H2 event awareness
|
|
169
|
+
--------------------------------------------------------------------------------
|
|
170
|
+
|
|
171
|
+
============================= FORENSIC VERDICT =================================
|
|
172
|
+
SKOR KEASLIAN : 100%
|
|
173
|
+
HASIL DIAGNOSTIK : TERVERIFIKASI ASLI (GENUINE)
|
|
174
|
+
VENDOR ASLI : Anthropic Claude
|
|
175
|
+
TINGKAT RISIKO : AMAN
|
|
176
|
+
|
|
177
|
+
[+] KESIMPULAN: Target terverifikasi konsisten dengan bobot model resmi.
|
|
178
|
+
================================================================================
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
---
|
|
182
|
+
|
|
183
|
+
## Keamanan & Privasi
|
|
184
|
+
|
|
185
|
+
- **Nol Retensi Telemetri**: ModelProof tidak pernah menyimpan, mengumpulkan, atau mengirimkan token API, base URL, atau percakapan Anda ke server eksternal mana pun.
|
|
186
|
+
- **Klien Sandbox Browser**: Pada antarmuka web, seluruh lalu lintas jaringan berjalan langsung dari browser lokal ke endpoint tujuan Anda.
|
|
187
|
+
- **Volatilitas Memori**: Kredensial hanya berada pada memori sementara aplikasi saat proses berjalan dan langsung dibersihkan saat jendela ditutup atau dimuat ulang.
|
|
188
|
+
|
|
189
|
+
---
|
|
190
|
+
|
|
191
|
+
## Lisensi
|
|
192
|
+
|
|
193
|
+
Didistribusikan di bawah [Lisensi MIT](LICENSE).
|
|
194
|
+
Bebas digunakan, diaudit, serta dimodifikasi untuk kebutuhan komersial maupun non-komersial.
|
|
@@ -0,0 +1,193 @@
|
|
|
1
|
+
<p align="center">
|
|
2
|
+
<img src="favicon.svg" width="64" height="64" alt="ModelProof Logo" />
|
|
3
|
+
</p>
|
|
4
|
+
|
|
5
|
+
<h1 align="center">ModelProof</h1>
|
|
6
|
+
|
|
7
|
+
<p align="center">
|
|
8
|
+
<strong>Zero-persistence client-side and CLI forensic scanner to detect LLM model spoofing, reverse-proxy masking, and silent downgrades.</strong>
|
|
9
|
+
</p>
|
|
10
|
+
|
|
11
|
+
<p align="center">
|
|
12
|
+
<a href="https://www.npmjs.com/package/modelproof"><img src="https://img.shields.io/npm/v/modelproof.svg?color=10b981&label=npm" alt="npm version" /></a>
|
|
13
|
+
<a href="https://pypi.org/project/modelproof/"><img src="https://img.shields.io/pypi/v/modelproof.svg?color=10b981&label=pypi" alt="pypi version" /></a>
|
|
14
|
+
<a href="LICENSE"><img src="https://img.shields.io/badge/license-MIT-blue.svg" alt="License" /></a>
|
|
15
|
+
<img src="https://img.shields.io/badge/zero-telemetry-emerald" alt="Zero Telemetry" />
|
|
16
|
+
<img src="https://img.shields.io/badge/dependencies-0-brightgreen" alt="Zero Dependencies" />
|
|
17
|
+
</p>
|
|
18
|
+
|
|
19
|
+
<p align="center">
|
|
20
|
+
<strong>English</strong> | <a href="README.id.md">Bahasa Indonesia</a>
|
|
21
|
+
</p>
|
|
22
|
+
|
|
23
|
+
---
|
|
24
|
+
|
|
25
|
+
## Overview
|
|
26
|
+
|
|
27
|
+
Unauthorized LLM resellers and reverse-proxy providers often engage in model spoofing:
|
|
28
|
+
- Re-routing expensive requests (e.g. `claude-3-5-sonnet`, `gpt-4o`) to cheaper open-weights models (`qwen-2.5-72b`, `deepseek-v3`, or quantized mini variants).
|
|
29
|
+
- Stripping system instructions and inserting hidden proxy jailbreaks.
|
|
30
|
+
- Fabricating fake model IDs in `/v1/models` catalogs.
|
|
31
|
+
|
|
32
|
+
ModelProof provides developers, QA engineers, and consumers with a verifiable forensic audit trail and an authenticity confidence score (0-100%) before trusting an endpoint in production.
|
|
33
|
+
|
|
34
|
+
---
|
|
35
|
+
|
|
36
|
+
## Why This Matters
|
|
37
|
+
|
|
38
|
+
1. **Financial Arbitrage (Paying Flagship, Receiving Commodity)**:
|
|
39
|
+
Resellers charge premium rates for flagship models while silently routing traffic to commodity models costing a fraction of the price. Users overpay up to 15x for degraded intelligence.
|
|
40
|
+
|
|
41
|
+
2. **Silent Production Breakage**:
|
|
42
|
+
Substituted models fail on nuanced reasoning, strict JSON Schema compliance, tool calling, and high-order logic, causing silent corruption in autonomous agents and backend pipelines.
|
|
43
|
+
|
|
44
|
+
3. **Data Exfiltration and Prompt Manipulation**:
|
|
45
|
+
Untrusted reverse-proxy layers may log raw prompts, inject hidden system preambles, or manipulate token streams before forwarding to unvetted upstream endpoints.
|
|
46
|
+
|
|
47
|
+
4. **Actionable Dispute Evidence**:
|
|
48
|
+
ModelProof generates deterministic, timestamped forensic audit reports that can be used directly as technical proof for refund requests or merchant disputes.
|
|
49
|
+
|
|
50
|
+
---
|
|
51
|
+
|
|
52
|
+
## Distribution Modes
|
|
53
|
+
|
|
54
|
+
| Platform | Distribution | Execution Command | Zero Dependencies |
|
|
55
|
+
| :--- | :--- | :--- | :---: |
|
|
56
|
+
| **Terminal (Node.js)** | NPM Registry | `npx modelproof` | Yes |
|
|
57
|
+
| **Terminal (Python)** | PyPI Package | `pip install modelproof` | Yes |
|
|
58
|
+
| **Browser (Web Sandbox)** | Static Client | `index.html` (Local / Static Host) | Yes |
|
|
59
|
+
|
|
60
|
+
---
|
|
61
|
+
|
|
62
|
+
## 10-Vector Forensic Detection Matrix
|
|
63
|
+
|
|
64
|
+
ModelProof executes 10 adversarial probes calibrated against official model weight checkpoints:
|
|
65
|
+
|
|
66
|
+
| # | Forensic Vector | Target Vulnerability / Anomaly | Mode |
|
|
67
|
+
| :-: | :--- | :--- | :-: |
|
|
68
|
+
| **01** | **Spatial Logic & Character Horizon** | Obfuscated character counting (`'s-t-r-a-w-b-e-r-r-y'`) + arithmetic runtime trap | FAST |
|
|
69
|
+
| **02** | **Tokenizer Usage & BPE Precision** | Multi-byte Unicode sequence testing token discrepancy and upstream token stripping | FAST |
|
|
70
|
+
| **03** | **System Instruction & Identity Leak** | Adversarial system prompt breakout probing true base weights and vendor confessions | FAST |
|
|
71
|
+
| **04** | **Hardware Telemetry & TPS Profile** | Real-time SSE stream parser detecting hyper-fast LPU hardware (>220 TPS SambaNova/Groq) | FAST |
|
|
72
|
+
| **05** | **Negative Constraint Compliance** | Rigid formatting constraints without conversational fluff or apology preambles | FAST |
|
|
73
|
+
| **06** | **Strict Schema / Constrained Decoding** | Enforces native JSON Schema strict parsing (crashes weak proxy engines) | DEEP |
|
|
74
|
+
| **07** | **Glitched Token Embedding Anomaly** | Unspeakable tokens (`SolidGoldMagikarp`) tokenizer behavior checks | FAST |
|
|
75
|
+
| **08** | **Temporal Cutoff Horizon (2024-H2)** | Probes post-training cutoff events (Python 3.13, Nobel Oct 2024) | FAST |
|
|
76
|
+
| **09** | **Reasoning CoT & Delimiter Trap** | Detects OpenAI `o1`/`o3` masked as DeepSeek-R1 via `<think>` token analysis | DEEP |
|
|
77
|
+
| **10** | **Type-Level Memory & Lifetime Logic** | Compiler-level Rust borrow checker & HRTB (`for<'a>`) lifetime reasoning | DEEP |
|
|
78
|
+
|
|
79
|
+
---
|
|
80
|
+
|
|
81
|
+
## CLI Quickstart
|
|
82
|
+
|
|
83
|
+
Both the Node.js and Python CLIs are 100% zero-dependency, running directly with native system runtimes (`fetch` and standard library `urllib`).
|
|
84
|
+
|
|
85
|
+
### 1. Node.js (NPX)
|
|
86
|
+
|
|
87
|
+
Run directly without installing:
|
|
88
|
+
|
|
89
|
+
```bash
|
|
90
|
+
# Basic scan against custom reverse proxy
|
|
91
|
+
npx modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "claude-3-5-sonnet-20241022"
|
|
92
|
+
|
|
93
|
+
# Deep audit running all 10 vectors with JSON output (CI/CD ready)
|
|
94
|
+
npx modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "gpt-4o" --all --json
|
|
95
|
+
|
|
96
|
+
# Inspect available models from upstream catalog
|
|
97
|
+
npx modelproof -u "https://my-proxy.com/v1" -k "sk-..." --models-only
|
|
98
|
+
```
|
|
99
|
+
|
|
100
|
+
### 2. Python (PIP)
|
|
101
|
+
|
|
102
|
+
Install from PyPI:
|
|
103
|
+
|
|
104
|
+
```bash
|
|
105
|
+
pip install modelproof
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
Run audit:
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
# Audit model authenticity
|
|
112
|
+
modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "gpt-4o"
|
|
113
|
+
|
|
114
|
+
# Run in Indonesian language
|
|
115
|
+
modelproof -u "https://my-proxy.com/v1" -k "sk-..." -m "claude-3-5-sonnet-20241022" --lang id
|
|
116
|
+
|
|
117
|
+
# Run all 10 deep vectors
|
|
118
|
+
modelproof -u "https://my-proxy.com/v1" -k "sk-..." --all
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Or execute without global install:
|
|
122
|
+
```bash
|
|
123
|
+
python -m modelproof.cli -u "https://my-proxy.com/v1" -k "sk-..." -m "gpt-4o"
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
---
|
|
127
|
+
|
|
128
|
+
## CLI Options & Flags
|
|
129
|
+
|
|
130
|
+
```text
|
|
131
|
+
Usage: modelproof [options]
|
|
132
|
+
|
|
133
|
+
Options:
|
|
134
|
+
-u, --base-url <url> Reverse proxy base URL (default: "https://api.openai.com/v1")
|
|
135
|
+
-k, --key <token> API Token / Key (or set OPENAI_API_KEY env var)
|
|
136
|
+
-m, --model <id> Claimed target model ID (default: "claude-3-5-sonnet-20241022")
|
|
137
|
+
-p, --protocol <proto> Protocol wire schema: auto, openai, anthropic (default: "auto")
|
|
138
|
+
-a, --all Run all 10 deep vectors (default: 8 fast vectors)
|
|
139
|
+
--models-only Audit upstream /v1/models catalog only and exit
|
|
140
|
+
--lang <lang> Output language: en, id (default: "en")
|
|
141
|
+
--json Output pure machine-readable JSON report
|
|
142
|
+
--timeout <sec> Per-request timeout in seconds (default: 30)
|
|
143
|
+
-v, --version Show version number
|
|
144
|
+
-h, --help Display help message
|
|
145
|
+
```
|
|
146
|
+
|
|
147
|
+
---
|
|
148
|
+
|
|
149
|
+
## Sample Terminal Output
|
|
150
|
+
|
|
151
|
+
```text
|
|
152
|
+
================================================================================
|
|
153
|
+
MODELPROOF CLI // LLM Proxy & Masking Forensic Scanner (v1.0.1)
|
|
154
|
+
Target: claude-3-5-sonnet-20241022 @ https://my-custom-proxy.com/v1
|
|
155
|
+
================================================================================
|
|
156
|
+
[*] Protocol Wire Schema: ANTHROPIC
|
|
157
|
+
[*] Catalog Audit: 42 models retrieved (Clean naming).
|
|
158
|
+
|
|
159
|
+
[+] RUNNING 8 FORENSIC VECTORS:
|
|
160
|
+
--------------------------------------------------------------------------------
|
|
161
|
+
[01] Spatial Logic & Character Horizon [PASS] r=3, math=324
|
|
162
|
+
[02] Tokenizer Usage & BPE Precision [PASS] Usage discrepancy within tolerance
|
|
163
|
+
[03] System Instruction & Identity Leak [PASS] Responded as official Claude
|
|
164
|
+
[04] Hardware Telemetry & TPS Profile [PASS] 64 TPS (Realistic datacenter profile)
|
|
165
|
+
[05] Negative Constraint Compliance [PASS] Zero preamble constraint honored
|
|
166
|
+
[06] Strict JSON Schema Decoding [PASS] Schema enforced natively
|
|
167
|
+
[07] Glitched Token Embedding [PASS] Token boundary verified
|
|
168
|
+
[08] Temporal Cutoff Horizon (2024-H2) [PASS] Verified post-2024H2 event awareness
|
|
169
|
+
--------------------------------------------------------------------------------
|
|
170
|
+
|
|
171
|
+
============================= FORENSIC VERDICT =================================
|
|
172
|
+
AUTHENTICITY SCORE : 100%
|
|
173
|
+
VERDICT : VERIFIED GENUINE
|
|
174
|
+
DETECTED VENDOR : Anthropic Claude
|
|
175
|
+
RISK LEVEL : SAFE
|
|
176
|
+
|
|
177
|
+
[+] CONCLUSION: Target verified consistent with genuine foundational weights.
|
|
178
|
+
================================================================================
|
|
179
|
+
```
|
|
180
|
+
|
|
181
|
+
---
|
|
182
|
+
|
|
183
|
+
## Security, Privacy & Compliance
|
|
184
|
+
|
|
185
|
+
- **Zero Telemetry Retention**: ModelProof never logs, collects, or transmits API keys, base URLs, or payload data to any external server.
|
|
186
|
+
- **Client-Side Sandbox**: In the web interface, network traffic travels directly from your browser to your designated proxy endpoint.
|
|
187
|
+
- **In-Memory Volatility**: Credentials exist strictly in runtime volatile memory and are cleared upon termination or reload.
|
|
188
|
+
|
|
189
|
+
---
|
|
190
|
+
|
|
191
|
+
## License
|
|
192
|
+
|
|
193
|
+
Distributed under the [MIT License](LICENSE).
|
|
@@ -543,17 +543,17 @@ function renderTestCheckboxes() {
|
|
|
543
543
|
<div class="checkbox-box ${checkboxBoxClass}">
|
|
544
544
|
${checkmarkSvg}
|
|
545
545
|
</div>
|
|
546
|
-
<div class="space-y-0.5 flex-1 pointer-events-none">
|
|
547
|
-
<div class="flex items-center justify-between">
|
|
548
|
-
<div class="flex items-center gap-
|
|
549
|
-
<span class="${isChecked ? 'text-emerald-400' : 'text-zinc-500'} transition-colors">${t.svg}</span>
|
|
550
|
-
<span class="text-xs font-mono font-medium ${isChecked ? 'text-zinc-100' : 'text-zinc-300'} transition-colors">${String(t.id).padStart(2, '0')}. ${info.name}</span>
|
|
546
|
+
<div class="space-y-0.5 flex-1 pointer-events-none min-w-0">
|
|
547
|
+
<div class="flex items-center justify-between gap-2">
|
|
548
|
+
<div class="flex items-center gap-1.5 min-w-0">
|
|
549
|
+
<span class="${isChecked ? 'text-emerald-400' : 'text-zinc-500'} transition-colors flex items-center justify-center shrink-0 w-4 h-4">${t.svg}</span>
|
|
550
|
+
<span class="text-xs font-mono font-medium ${isChecked ? 'text-zinc-100' : 'text-zinc-300'} transition-colors truncate">${String(t.id).padStart(2, '0')}. ${info.name}</span>
|
|
551
551
|
</div>
|
|
552
552
|
${t.quick
|
|
553
|
-
? '<span class="text-[9px] px-1.5 py-0.2 rounded bg-zinc-900 text-zinc-400 border border-zinc-800 font-mono">FAST</span>'
|
|
554
|
-
: '<span class="text-[9px] px-1.5 py-0.2 rounded bg-purple-950/50 text-purple-300 border border-purple-800/40 font-mono">DEEP</span>'}
|
|
553
|
+
? '<span class="text-[9px] px-1.5 py-0.2 rounded bg-zinc-900 text-zinc-400 border border-zinc-800 font-mono shrink-0">FAST</span>'
|
|
554
|
+
: '<span class="text-[9px] px-1.5 py-0.2 rounded bg-purple-950/50 text-purple-300 border border-purple-800/40 font-mono shrink-0">DEEP</span>'}
|
|
555
555
|
</div>
|
|
556
|
-
<p class="text-[11px] text-zinc-500 font-sans leading-tight pl-5">${info.desc}</p>
|
|
556
|
+
<p class="text-[11px] text-zinc-500 font-sans leading-tight pl-5.5">${info.desc}</p>
|
|
557
557
|
</div>
|
|
558
558
|
`;
|
|
559
559
|
|
|
@@ -599,10 +599,10 @@ function renderPipelineRows() {
|
|
|
599
599
|
row.innerHTML = `
|
|
600
600
|
<div class="space-y-0.5 min-w-0 flex-1">
|
|
601
601
|
<div class="flex items-center gap-2">
|
|
602
|
-
<span class="test-icon text-xs text-zinc-600 font-mono shrink-0"><i class="fa-regular fa-circle"></i></span>
|
|
602
|
+
<span class="test-icon text-xs text-zinc-600 font-mono shrink-0 flex items-center justify-center w-4 h-4"><i class="fa-regular fa-circle"></i></span>
|
|
603
603
|
<span class="text-xs font-mono font-medium text-zinc-300 truncate">${String(t.id).padStart(2, '0')}. ${info.name}</span>
|
|
604
604
|
</div>
|
|
605
|
-
<p class="text-[11px] text-zinc-500 test-detail font-sans pl-
|
|
605
|
+
<p class="text-[11px] text-zinc-500 test-detail font-sans pl-6 break-words">${info.desc}</p>
|
|
606
606
|
</div>
|
|
607
607
|
<span class="test-status font-mono text-[9px] sm:text-[10px] px-1.5 sm:px-2 py-0.5 rounded bg-zinc-900 text-zinc-400 border border-zinc-800 uppercase tracking-wider shrink-0 mt-0.5">${initialStatus}</span>
|
|
608
608
|
`;
|
|
@@ -895,37 +895,37 @@ function initUI() {
|
|
|
895
895
|
}
|
|
896
896
|
|
|
897
897
|
function updateProtocolUI() {
|
|
898
|
-
const activeClass = 'py-1.5 px-2 rounded-md transition-all flex items-center justify-center gap-1.5 font-medium bg-emerald-950/40 text-emerald-400 border border-emerald-500/50 shadow-sm';
|
|
899
|
-
const inactiveClass = 'py-1.5 px-2 rounded-md transition-all flex items-center justify-center gap-1.5 font-medium text-zinc-500 hover:text-zinc-300 hover:bg-zinc-900/40 border border-transparent';
|
|
898
|
+
const activeClass = 'py-1.5 px-1 sm:px-2 rounded-md transition-all flex items-center justify-center gap-1 sm:gap-1.5 font-medium bg-emerald-950/40 text-emerald-400 border border-emerald-500/50 shadow-sm leading-none';
|
|
899
|
+
const inactiveClass = 'py-1.5 px-1 sm:px-2 rounded-md transition-all flex items-center justify-center gap-1 sm:gap-1.5 font-medium text-zinc-500 hover:text-zinc-300 hover:bg-zinc-900/40 border border-transparent leading-none';
|
|
900
900
|
|
|
901
|
-
const checkSvg = `<svg class="w-3.5 h-3.5 text-emerald-400 stroke-[3] shrink-0" fill="none" viewBox="0 0 24 24" stroke="currentColor">
|
|
901
|
+
const checkSvg = `<svg class="w-3.5 h-3.5 text-emerald-400 stroke-[3] shrink-0 inline-block align-middle" fill="none" viewBox="0 0 24 24" stroke="currentColor">
|
|
902
902
|
<path stroke-linecap="round" stroke-linejoin="round" d="M4.5 12.75l6 6 9-13.5" />
|
|
903
903
|
</svg>`;
|
|
904
904
|
|
|
905
|
-
const openaiLogo = `<svg class="w-3 h-3 fill-current shrink-0" viewBox="0 0 24 24"><path d="M22.2819 9.8211a5.9847 5.9847 0 0 0-.5157-4.9108 6.0462 6.0462 0 0 0-6.5098-2.9A6.0651 6.0651 0 0 0 4.9807 4.1818a5.9847 5.9847 0 0 0-3.9977 2.9 6.0462 6.0462 0 0 0 .7427 7.0966 5.98 5.98 0 0 0 .511 4.9107 6.051 6.051 0 0 0 6.5146 2.9001A5.9847 5.9847 0 0 0 13.2599 24a6.0557 6.0557 0 0 0 5.7718-4.2058 5.9894 5.9894 0 0 0 3.9977-2.9001 6.0557 6.0557 0 0 0-.7475-7.0729zm-9.022 12.6081a4.4755 4.4755 0 0 1-2.8764-1.0408l.1419-.0804 4.7783-2.7582a.7948.7948 0 0 0 .3927-.6813v-6.7369l2.02 1.1683a.071.071 0 0 1 .038.052v5.5826a4.504 4.504 0 0 1-4.4945 4.4947zm-9.6607-4.1254a4.4708 4.4708 0 0 1-.5346-3.0137l.142.0852 4.783 2.7582a.7712.7712 0 0 0 .7806 0l5.8428-3.3685v2.3324a.0804.0804 0 0 1-.0332.0615L9.74 19.9502a4.4992 4.4992 0 0 1-6.1408-1.6464zM2.3408 7.8956a4.485 4.485 0 0 1 2.3655-1.9728V11.6a.7664.7664 0 0 0 .3879.6765l5.8144 3.3543-2.0201 1.1683a.0757.0757 0 0 1-.071 0l-4.8303-2.7866A4.504 4.504 0 0 1 2.3408 7.872zm16.5963 3.8558L13.1038 8.364 15.1192 7.2a.0757.0757 0 0 1 .071 0l4.8303 2.7913a4.4944 4.4944 0 0 1-.6765 8.1042v-5.6772a.79.79 0 0 0-.407-.6668zm2.0107-3.0231l-.142-.0852-4.7735-2.7818a.7759.7759 0 0 0-.7854 0L9.409 9.2297V6.8974a.0662.0662 0 0 1 .0284-.0615l4.8303-2.7866a4.4992 4.4992 0 0 1 6.6802 4.66zM8.3065 12.863l-2.02-1.1635a.0804.0804 0 0 1-.038-.0567V6.0742a4.4992 4.4992 0 0 1 7.3757-3.4537l-.142.0805L8.704 5.459a.7948.7948 0 0 0-.3927.6813zm1.0976-2.3654l2.602-1.4998 2.6069 1.4998v2.9994l-2.5974 1.4997-2.6067-1.4997Z"/></svg>`;
|
|
905
|
+
const openaiLogo = `<svg class="w-3.5 h-3.5 fill-current shrink-0 inline-block align-middle" viewBox="0 0 24 24"><path d="M22.2819 9.8211a5.9847 5.9847 0 0 0-.5157-4.9108 6.0462 6.0462 0 0 0-6.5098-2.9A6.0651 6.0651 0 0 0 4.9807 4.1818a5.9847 5.9847 0 0 0-3.9977 2.9 6.0462 6.0462 0 0 0 .7427 7.0966 5.98 5.98 0 0 0 .511 4.9107 6.051 6.051 0 0 0 6.5146 2.9001A5.9847 5.9847 0 0 0 13.2599 24a6.0557 6.0557 0 0 0 5.7718-4.2058 5.9894 5.9894 0 0 0 3.9977-2.9001 6.0557 6.0557 0 0 0-.7475-7.0729zm-9.022 12.6081a4.4755 4.4755 0 0 1-2.8764-1.0408l.1419-.0804 4.7783-2.7582a.7948.7948 0 0 0 .3927-.6813v-6.7369l2.02 1.1683a.071.071 0 0 1 .038.052v5.5826a4.504 4.504 0 0 1-4.4945 4.4947zm-9.6607-4.1254a4.4708 4.4708 0 0 1-.5346-3.0137l.142.0852 4.783 2.7582a.7712.7712 0 0 0 .7806 0l5.8428-3.3685v2.3324a.0804.0804 0 0 1-.0332.0615L9.74 19.9502a4.4992 4.4992 0 0 1-6.1408-1.6464zM2.3408 7.8956a4.485 4.485 0 0 1 2.3655-1.9728V11.6a.7664.7664 0 0 0 .3879.6765l5.8144 3.3543-2.0201 1.1683a.0757.0757 0 0 1-.071 0l-4.8303-2.7866A4.504 4.504 0 0 1 2.3408 7.872zm16.5963 3.8558L13.1038 8.364 15.1192 7.2a.0757.0757 0 0 1 .071 0l4.8303 2.7913a4.4944 4.4944 0 0 1-.6765 8.1042v-5.6772a.79.79 0 0 0-.407-.6668zm2.0107-3.0231l-.142-.0852-4.7735-2.7818a.7759.7759 0 0 0-.7854 0L9.409 9.2297V6.8974a.0662.0662 0 0 1 .0284-.0615l4.8303-2.7866a4.4992 4.4992 0 0 1 6.6802 4.66zM8.3065 12.863l-2.02-1.1635a.0804.0804 0 0 1-.038-.0567V6.0742a4.4992 4.4992 0 0 1 7.3757-3.4537l-.142.0805L8.704 5.459a.7948.7948 0 0 0-.3927.6813zm1.0976-2.3654l2.602-1.4998 2.6069 1.4998v2.9994l-2.5974 1.4997-2.6067-1.4997Z"/></svg>`;
|
|
906
906
|
|
|
907
|
-
const anthropicLogo = `<svg class="w-3 h-3 fill-current shrink-0" viewBox="0 0 24 24"><path d="M17.
|
|
907
|
+
const anthropicLogo = `<svg class="w-3.5 h-3.5 fill-current shrink-0 inline-block align-middle" viewBox="0 0 24 24"><path d="M17.3041 3.541h-3.6718l6.696 16.918H24Zm-10.6082 0L0 20.459h3.7442l1.3693-3.5527h7.0052l1.3693 3.5528h3.7442L10.5363 3.5409Zm-.3712 10.2232 2.2914-5.9456 2.2914 5.9456Z"/></svg>`;
|
|
908
908
|
|
|
909
909
|
if (state.protocolMode === 'auto') {
|
|
910
910
|
el.protoAuto.className = activeClass;
|
|
911
|
-
el.protoAuto.innerHTML = `${checkSvg}<span>Auto</span>`;
|
|
911
|
+
el.protoAuto.innerHTML = `${checkSvg}<span class="inline-block leading-none">Auto</span>`;
|
|
912
912
|
|
|
913
913
|
el.protoOpenai.className = inactiveClass;
|
|
914
|
-
el.protoOpenai.innerHTML = `${openaiLogo}<span>OpenAI</span>`;
|
|
914
|
+
el.protoOpenai.innerHTML = `${openaiLogo}<span class="inline-block leading-none">OpenAI</span>`;
|
|
915
915
|
|
|
916
916
|
el.protoAnthropic.className = inactiveClass;
|
|
917
|
-
el.protoAnthropic.innerHTML = `${anthropicLogo}<span>Anthropic</span>`;
|
|
917
|
+
el.protoAnthropic.innerHTML = `${anthropicLogo}<span class="inline-block leading-none">Anthropic</span>`;
|
|
918
918
|
|
|
919
919
|
updateProtocolBadge('AUTO: STANDBY', false);
|
|
920
920
|
} else if (state.protocolMode === 'openai') {
|
|
921
921
|
el.protoAuto.className = inactiveClass;
|
|
922
|
-
el.protoAuto.innerHTML = `<span>Auto</span>`;
|
|
922
|
+
el.protoAuto.innerHTML = `<span class="inline-block leading-none">Auto</span>`;
|
|
923
923
|
|
|
924
924
|
el.protoOpenai.className = activeClass;
|
|
925
|
-
el.protoOpenai.innerHTML = `${checkSvg}<span>OpenAI</span>`;
|
|
925
|
+
el.protoOpenai.innerHTML = `${checkSvg}<span class="inline-block leading-none">OpenAI</span>`;
|
|
926
926
|
|
|
927
927
|
el.protoAnthropic.className = inactiveClass;
|
|
928
|
-
el.protoAnthropic.innerHTML = `${anthropicLogo}<span>Anthropic</span>`;
|
|
928
|
+
el.protoAnthropic.innerHTML = `${anthropicLogo}<span class="inline-block leading-none">Anthropic</span>`;
|
|
929
929
|
|
|
930
930
|
state.detectedProtocol = 'openai';
|
|
931
931
|
updateProtocolBadge('MANUAL: OPENAI', false);
|
|
@@ -934,13 +934,13 @@ function updateProtocolUI() {
|
|
|
934
934
|
}
|
|
935
935
|
} else {
|
|
936
936
|
el.protoAuto.className = inactiveClass;
|
|
937
|
-
el.protoAuto.innerHTML = `<span>Auto</span>`;
|
|
937
|
+
el.protoAuto.innerHTML = `<span class="inline-block leading-none">Auto</span>`;
|
|
938
938
|
|
|
939
939
|
el.protoOpenai.className = inactiveClass;
|
|
940
|
-
el.protoOpenai.innerHTML = `${openaiLogo}<span>OpenAI</span>`;
|
|
940
|
+
el.protoOpenai.innerHTML = `${openaiLogo}<span class="inline-block leading-none">OpenAI</span>`;
|
|
941
941
|
|
|
942
942
|
el.protoAnthropic.className = activeClass;
|
|
943
|
-
el.protoAnthropic.innerHTML = `${checkSvg}<span>Anthropic</span>`;
|
|
943
|
+
el.protoAnthropic.innerHTML = `${checkSvg}<span class="inline-block leading-none">Anthropic</span>`;
|
|
944
944
|
|
|
945
945
|
state.detectedProtocol = 'anthropic';
|
|
946
946
|
updateProtocolBadge('MANUAL: ANTHROPIC', false);
|
|
@@ -1323,22 +1323,22 @@ function updateTestRow(testId, status, detailText = null) {
|
|
|
1323
1323
|
if (detailText) detail.textContent = detailText;
|
|
1324
1324
|
|
|
1325
1325
|
if (status === 'RUNNING') {
|
|
1326
|
-
icon.innerHTML = `<svg class="w-3.5 h-3.5 text-cyan-400 animate-spin-fast shrink-0 inline-block" viewBox="0 0 50 50">
|
|
1326
|
+
icon.innerHTML = `<svg class="w-3.5 h-3.5 text-cyan-400 animate-spin-fast shrink-0 inline-block align-middle" viewBox="0 0 50 50">
|
|
1327
1327
|
<circle class="opacity-20" cx="25" cy="25" r="20" fill="none" stroke="currentColor" stroke-width="4"></circle>
|
|
1328
1328
|
<circle class="spinner-circle-morph" cx="25" cy="25" r="20" fill="none" stroke="currentColor" stroke-width="4"></circle>
|
|
1329
1329
|
</svg>`;
|
|
1330
1330
|
statusBadge.textContent = state.lang === 'en' ? 'RUNNING' : 'PROSES';
|
|
1331
1331
|
statusBadge.className = 'test-status font-mono text-[10px] px-2 py-0.5 rounded bg-cyan-950 text-cyan-400 border border-cyan-800 uppercase';
|
|
1332
1332
|
} else if (status === 'PASSED') {
|
|
1333
|
-
icon.innerHTML = `<svg class="w-3.5 h-3.5 text-emerald-400 shrink-0 inline-block" fill="none" viewBox="0 0 24 24" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12l2 2 4-4m6 2a9 9 0 11-18 0 9 9 0 0118 0z" /></svg>`;
|
|
1333
|
+
icon.innerHTML = `<svg class="w-3.5 h-3.5 text-emerald-400 shrink-0 inline-block align-middle" fill="none" viewBox="0 0 24 24" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M9 12l2 2 4-4m6 2a9 9 0 11-18 0 9 9 0 0118 0z" /></svg>`;
|
|
1334
1334
|
statusBadge.textContent = state.lang === 'en' ? 'PASSED' : 'LOLOS';
|
|
1335
1335
|
statusBadge.className = 'test-status font-mono text-[10px] px-2 py-0.5 rounded bg-emerald-950 text-emerald-400 border border-emerald-800 uppercase';
|
|
1336
1336
|
} else if (status === 'WARNING') {
|
|
1337
|
-
icon.innerHTML = `<svg class="w-3.5 h-3.5 text-yellow-400 shrink-0 inline-block" fill="none" viewBox="0 0 24 24" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-3L13.732 4c-.77-1.333-2.694-1.333-3.464 0L3.34 16c-.77 1.333.192 3 1.732 3z" /></svg>`;
|
|
1337
|
+
icon.innerHTML = `<svg class="w-3.5 h-3.5 text-yellow-400 shrink-0 inline-block align-middle" fill="none" viewBox="0 0 24 24" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 9v2m0 4h.01m-6.938 4h13.856c1.54 0 2.502-1.667 1.732-3L13.732 4c-.77-1.333-2.694-1.333-3.464 0L3.34 16c-.77 1.333.192 3 1.732 3z" /></svg>`;
|
|
1338
1338
|
statusBadge.textContent = state.lang === 'en' ? 'SUSPICIOUS' : 'CURIGA';
|
|
1339
1339
|
statusBadge.className = 'test-status font-mono text-[10px] px-2 py-0.5 rounded bg-yellow-950 text-yellow-400 border border-yellow-800 uppercase';
|
|
1340
1340
|
} else if (status === 'FAILED') {
|
|
1341
|
-
icon.innerHTML = `<svg class="w-3.5 h-3.5 text-rose-400 shrink-0 inline-block" fill="none" viewBox="0 0 24 24" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M10 14l2-2m0 0l2-2m-2 2l-2-2m2 2l2 2m7-2a9 9 0 11-18 0 9 9 0 0118 0z" /></svg>`;
|
|
1341
|
+
icon.innerHTML = `<svg class="w-3.5 h-3.5 text-rose-400 shrink-0 inline-block align-middle" fill="none" viewBox="0 0 24 24" stroke="currentColor"><path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M10 14l2-2m0 0l2-2m-2 2l-2-2m2 2l2 2m7-2a9 9 0 11-18 0 9 9 0 0118 0z" /></svg>`;
|
|
1342
1342
|
statusBadge.textContent = state.lang === 'en' ? 'FAILED' : 'GAGAL';
|
|
1343
1343
|
statusBadge.className = 'test-status font-mono text-[10px] px-2 py-0.5 rounded bg-rose-950 text-rose-400 border border-rose-800 uppercase';
|
|
1344
1344
|
} else if (status === 'PENDING' || status === 'READY') {
|
|
@@ -28,7 +28,7 @@ const c = {
|
|
|
28
28
|
// Help text
|
|
29
29
|
function printHelp() {
|
|
30
30
|
console.log(`
|
|
31
|
-
${c.bold}${c.cyan}MODELPROOF CLI // LLM Proxy & Masking Forensic Scanner (v1.0.
|
|
31
|
+
${c.bold}${c.cyan}MODELPROOF CLI // LLM Proxy & Masking Forensic Scanner (v1.0.1)${c.reset}
|
|
32
32
|
Detect model spoofing, masking, and downgrading on reverse proxies.
|
|
33
33
|
|
|
34
34
|
${c.bold}USAGE:${c.reset}
|
|
@@ -75,7 +75,7 @@ function parseArgs() {
|
|
|
75
75
|
const a = args[i];
|
|
76
76
|
if (a === '-h' || a === '--help') printHelp();
|
|
77
77
|
if (a === '-v' || a === '--version') {
|
|
78
|
-
console.log('1.0.
|
|
78
|
+
console.log('1.0.1');
|
|
79
79
|
process.exit(0);
|
|
80
80
|
}
|
|
81
81
|
if ((a === '-u' || a === '--base-url') && args[i + 1]) opts.baseUrl = args[++i];
|
|
@@ -501,7 +501,7 @@ async function main() {
|
|
|
501
501
|
// Friendly fallback if user runs without args and no API key is set
|
|
502
502
|
if (!opts.apiKey) {
|
|
503
503
|
console.log(`\n${c.bold}================================================================================${c.reset}`);
|
|
504
|
-
console.log(` ${c.bold}${c.cyan}MODELPROOF CLI${c.reset} // LLM Proxy & Masking Forensic Scanner (v1.0.
|
|
504
|
+
console.log(` ${c.bold}${c.cyan}MODELPROOF CLI${c.reset} // LLM Proxy & Masking Forensic Scanner (v1.0.1)`);
|
|
505
505
|
console.log(`${c.bold}================================================================================${c.reset}`);
|
|
506
506
|
console.log(`Zero-persistence scanner to detect model spoofing, masking, and proxy downgrades.\n`);
|
|
507
507
|
console.log(`${c.bold}QUICKSTART:${c.reset}`);
|
|
@@ -517,7 +517,7 @@ async function main() {
|
|
|
517
517
|
|
|
518
518
|
if (!opts.json) {
|
|
519
519
|
console.log(`\n${c.bold}================================================================================${c.reset}`);
|
|
520
|
-
console.log(` ${c.bold}${c.cyan}MODELPROOF CLI${c.reset} // LLM Proxy & Masking Forensic Scanner (v1.0.
|
|
520
|
+
console.log(` ${c.bold}${c.cyan}MODELPROOF CLI${c.reset} // LLM Proxy & Masking Forensic Scanner (v1.0.1)`);
|
|
521
521
|
console.log(` Target: ${c.bold}${opts.model}${c.reset} @ ${opts.baseUrl}`);
|
|
522
522
|
console.log(`${c.bold}================================================================================${c.reset}`);
|
|
523
523
|
}
|
|
@@ -0,0 +1,5 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 32 32">
|
|
2
|
+
<rect width="32" height="32" rx="8" fill="#09090b" stroke="#27272a" stroke-width="1.5"/>
|
|
3
|
+
<path d="M16 4L6 8.5V16C6 22.2 10.3 27.9 16 29.5C21.7 27.9 26 22.2 26 16V8.5L16 4Z" fill="#064e3b" stroke="#10b981" stroke-width="1.75" stroke-linejoin="round"/>
|
|
4
|
+
<path d="M11 16L14.5 19.5L21 12.5" fill="none" stroke="#34d399" stroke-width="2.2" stroke-linecap="round" stroke-linejoin="round"/>
|
|
5
|
+
</svg>
|
|
@@ -3,7 +3,27 @@
|
|
|
3
3
|
<head>
|
|
4
4
|
<meta charset="UTF-8">
|
|
5
5
|
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
|
6
|
-
<title>ModelProof
|
|
6
|
+
<title>ModelProof — LLM Forensic Scanner</title>
|
|
7
|
+
<meta name="description" content="Zero-persistence forensic scanner to audit LLM proxy endpoints and detect model spoofing, masking, and silent downgrades.">
|
|
8
|
+
<meta name="keywords" content="llm, ai, model spoofing, reverse proxy, audit, benchmark, openai, anthropic, deepseek, forensics">
|
|
9
|
+
<meta name="author" content="ModelProof Contributors">
|
|
10
|
+
<meta name="theme-color" content="#09090b">
|
|
11
|
+
|
|
12
|
+
<!-- Favicon -->
|
|
13
|
+
<link rel="icon" type="image/svg+xml" href="favicon.svg">
|
|
14
|
+
<link rel="apple-touch-icon" href="favicon.svg">
|
|
15
|
+
|
|
16
|
+
<!-- Open Graph / Facebook / LinkedIn -->
|
|
17
|
+
<meta property="og:type" content="website">
|
|
18
|
+
<meta property="og:title" content="ModelProof — LLM Forensic Scanner">
|
|
19
|
+
<meta property="og:description" content="Zero-persistence forensic scanner to audit LLM proxy endpoints and detect model spoofing, masking, and silent downgrades.">
|
|
20
|
+
<meta property="og:image" content="favicon.svg">
|
|
21
|
+
|
|
22
|
+
<!-- Twitter Card -->
|
|
23
|
+
<meta name="twitter:card" content="summary">
|
|
24
|
+
<meta name="twitter:title" content="ModelProof — LLM Forensic Scanner">
|
|
25
|
+
<meta name="twitter:description" content="Zero-persistence forensic scanner to audit LLM proxy endpoints and detect model spoofing, masking, and silent downgrades.">
|
|
26
|
+
<meta name="twitter:image" content="favicon.svg">
|
|
7
27
|
|
|
8
28
|
<!-- Fonts -->
|
|
9
29
|
<link rel="preconnect" href="https://fonts.googleapis.com">
|
|
@@ -110,7 +130,7 @@
|
|
|
110
130
|
<span class="inline-block w-2 h-2 rounded-full bg-emerald-400"></span>
|
|
111
131
|
<span class="font-mono font-semibold tracking-tight text-xs sm:text-sm text-zinc-100">MODEL<span class="text-emerald-400">PROOF</span></span>
|
|
112
132
|
<span class="text-zinc-600 font-mono text-xs">/</span>
|
|
113
|
-
<span class="text-[10px] sm:text-[11px] font-mono text-zinc-400 bg-zinc-900 border border-zinc-800 px-1.5 sm:px-2 py-0.5 rounded">v1.0.
|
|
133
|
+
<span class="text-[10px] sm:text-[11px] font-mono text-zinc-400 bg-zinc-900 border border-zinc-800 px-1.5 sm:px-2 py-0.5 rounded">v1.0.1</span>
|
|
114
134
|
</div>
|
|
115
135
|
</div>
|
|
116
136
|
|
|
@@ -160,23 +180,23 @@
|
|
|
160
180
|
</span>
|
|
161
181
|
</div>
|
|
162
182
|
<div class="grid grid-cols-3 p-1 bg-zinc-950 rounded-lg border border-zinc-800/80 text-[11px] sm:text-xs font-mono gap-1">
|
|
163
|
-
<button id="proto-auto" type="button" class="py-
|
|
164
|
-
<svg class="w-3 h-3 text-emerald-400 shrink-0" fill="none" viewBox="0 0 24 24" stroke="currentColor">
|
|
183
|
+
<button id="proto-auto" type="button" class="py-1.5 px-1 sm:px-2 rounded-md transition-all flex items-center justify-center gap-1 sm:gap-1.5 font-medium bg-zinc-850 text-emerald-400 border border-emerald-500/40 shadow-sm leading-none">
|
|
184
|
+
<svg class="w-3.5 h-3.5 text-emerald-400 shrink-0 inline-block align-middle" fill="none" viewBox="0 0 24 24" stroke="currentColor">
|
|
165
185
|
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M13 10V3L4 14h7v7l9-11h-7z" />
|
|
166
186
|
</svg>
|
|
167
|
-
<span>Auto</span>
|
|
187
|
+
<span class="inline-block leading-none">Auto</span>
|
|
168
188
|
</button>
|
|
169
|
-
<button id="proto-openai" type="button" class="py-
|
|
170
|
-
<svg class="w-3 h-3 fill-current text-zinc-500 shrink-0" viewBox="0 0 24 24">
|
|
189
|
+
<button id="proto-openai" type="button" class="py-1.5 px-1 sm:px-2 rounded-md transition-all flex items-center justify-center gap-1 sm:gap-1.5 font-medium text-zinc-500 hover:text-zinc-300 hover:bg-zinc-900/40 border border-transparent leading-none">
|
|
190
|
+
<svg class="w-3.5 h-3.5 fill-current text-zinc-500 shrink-0 inline-block align-middle" viewBox="0 0 24 24">
|
|
171
191
|
<path d="M22.2819 9.8211a5.9847 5.9847 0 0 0-.5157-4.9108 6.0462 6.0462 0 0 0-6.5098-2.9A6.0651 6.0651 0 0 0 4.9807 4.1818a5.9847 5.9847 0 0 0-3.9977 2.9 6.0462 6.0462 0 0 0 .7427 7.0966 5.98 5.98 0 0 0 .511 4.9107 6.051 6.051 0 0 0 6.5146 2.9001A5.9847 5.9847 0 0 0 13.2599 24a6.0557 6.0557 0 0 0 5.7718-4.2058 5.9894 5.9894 0 0 0 3.9977-2.9001 6.0557 6.0557 0 0 0-.7475-7.0729zm-9.022 12.6081a4.4755 4.4755 0 0 1-2.8764-1.0408l.1419-.0804 4.7783-2.7582a.7948.7948 0 0 0 .3927-.6813v-6.7369l2.02 1.1683a.071.071 0 0 1 .038.052v5.5826a4.504 4.504 0 0 1-4.4945 4.4947zm-9.6607-4.1254a4.4708 4.4708 0 0 1-.5346-3.0137l.142.0852 4.783 2.7582a.7712.7712 0 0 0 .7806 0l5.8428-3.3685v2.3324a.0804.0804 0 0 1-.0332.0615L9.74 19.9502a4.4992 4.4992 0 0 1-6.1408-1.6464zM2.3408 7.8956a4.485 4.485 0 0 1 2.3655-1.9728V11.6a.7664.7664 0 0 0 .3879.6765l5.8144 3.3543-2.0201 1.1683a.0757.0757 0 0 1-.071 0l-4.8303-2.7866A4.504 4.504 0 0 1 2.3408 7.872zm16.5963 3.8558L13.1038 8.364 15.1192 7.2a.0757.0757 0 0 1 .071 0l4.8303 2.7913a4.4944 4.4944 0 0 1-.6765 8.1042v-5.6772a.79.79 0 0 0-.407-.6668zm2.0107-3.0231l-.142-.0852-4.7735-2.7818a.7759.7759 0 0 0-.7854 0L9.409 9.2297V6.8974a.0662.0662 0 0 1 .0284-.0615l4.8303-2.7866a4.4992 4.4992 0 0 1 6.6802 4.66zM8.3065 12.863l-2.02-1.1635a.0804.0804 0 0 1-.038-.0567V6.0742a4.4992 4.4992 0 0 1 7.3757-3.4537l-.142.0805L8.704 5.459a.7948.7948 0 0 0-.3927.6813zm1.0976-2.3654l2.602-1.4998 2.6069 1.4998v2.9994l-2.5974 1.4997-2.6067-1.4997Z"/>
|
|
172
192
|
</svg>
|
|
173
|
-
<span>OpenAI</span>
|
|
193
|
+
<span class="inline-block leading-none">OpenAI</span>
|
|
174
194
|
</button>
|
|
175
|
-
<button id="proto-anthropic" type="button" class="py-
|
|
176
|
-
<svg class="w-3 h-3 fill-current text-zinc-500 shrink-0" viewBox="0 0 24 24">
|
|
177
|
-
<path d="M17.
|
|
195
|
+
<button id="proto-anthropic" type="button" class="py-1.5 px-1 sm:px-2 rounded-md transition-all flex items-center justify-center gap-1 sm:gap-1.5 font-medium text-zinc-500 hover:text-zinc-300 hover:bg-zinc-900/40 border border-transparent leading-none">
|
|
196
|
+
<svg class="w-3.5 h-3.5 fill-current text-zinc-500 shrink-0 inline-block align-middle" viewBox="0 0 24 24">
|
|
197
|
+
<path d="M17.3041 3.541h-3.6718l6.696 16.918H24Zm-10.6082 0L0 20.459h3.7442l1.3693-3.5527h7.0052l1.3693 3.5528h3.7442L10.5363 3.5409Zm-.3712 10.2232 2.2914-5.9456 2.2914 5.9456Z"/>
|
|
178
198
|
</svg>
|
|
179
|
-
<span>Anthropic</span>
|
|
199
|
+
<span class="inline-block leading-none">Anthropic</span>
|
|
180
200
|
</button>
|
|
181
201
|
</div>
|
|
182
202
|
</div>
|
|
@@ -424,13 +424,13 @@ Examples:
|
|
|
424
424
|
parser.add_argument("--lang", default="en", choices=["en", "id"], help="Output language (default: en)")
|
|
425
425
|
parser.add_argument("--json", action="store_true", help="Output pure JSON report for CI/CD pipelines")
|
|
426
426
|
parser.add_argument("--timeout", type=int, default=30, help="Per-request timeout in seconds (default: 30)")
|
|
427
|
-
parser.add_argument("-v", "--version", action="version", version="1.0.
|
|
427
|
+
parser.add_argument("-v", "--version", action="version", version="1.0.1")
|
|
428
428
|
|
|
429
429
|
args = parser.parse_args()
|
|
430
430
|
|
|
431
431
|
if not args.key:
|
|
432
432
|
print(f"\n{C_BOLD}================================================================================{C_RESET}")
|
|
433
|
-
print(f" {C_BOLD}{C_CYAN}MODELPROOF CLI{C_RESET} // LLM Proxy & Masking Forensic Scanner (v1.0.
|
|
433
|
+
print(f" {C_BOLD}{C_CYAN}MODELPROOF CLI{C_RESET} // LLM Proxy & Masking Forensic Scanner (v1.0.1)")
|
|
434
434
|
print(f"{C_BOLD}================================================================================{C_RESET}")
|
|
435
435
|
print("Zero-persistence scanner to detect model spoofing, masking, and proxy downgrades.\n")
|
|
436
436
|
print(f"{C_BOLD}QUICKSTART:{C_RESET}")
|
|
@@ -446,7 +446,7 @@ Examples:
|
|
|
446
446
|
|
|
447
447
|
if not args.json:
|
|
448
448
|
print(f"\n{C_BOLD}================================================================================{C_RESET}")
|
|
449
|
-
print(f" {C_BOLD}{C_CYAN}MODELPROOF CLI{C_RESET} // LLM Proxy & Masking Forensic Scanner (v1.0.
|
|
449
|
+
print(f" {C_BOLD}{C_CYAN}MODELPROOF CLI{C_RESET} // LLM Proxy & Masking Forensic Scanner (v1.0.1)")
|
|
450
450
|
print(f" Target: {C_BOLD}{args.model}{C_RESET} @ {args.base_url}")
|
|
451
451
|
print(f"{C_BOLD}================================================================================{C_RESET}")
|
|
452
452
|
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "modelproof",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.1",
|
|
4
4
|
"description": "Zero-persistence client-side & CLI forensic scanner to detect LLM model spoofing, masking, and downgrading.",
|
|
5
5
|
"main": "bin/modelproof.js",
|
|
6
6
|
"bin": {
|
|
@@ -27,7 +27,8 @@
|
|
|
27
27
|
"files": [
|
|
28
28
|
"bin/",
|
|
29
29
|
"LICENSE",
|
|
30
|
-
"README.md"
|
|
30
|
+
"README.md",
|
|
31
|
+
"README.id.md"
|
|
31
32
|
],
|
|
32
33
|
"repository": {
|
|
33
34
|
"type": "git",
|
|
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
|
|
|
4
4
|
|
|
5
5
|
[project]
|
|
6
6
|
name = "modelproof"
|
|
7
|
-
version = "1.0.
|
|
7
|
+
version = "1.0.1"
|
|
8
8
|
description = "Zero-persistence client-side and CLI forensic scanner to detect LLM model spoofing, masking, and downgrading."
|
|
9
9
|
readme = "README.md"
|
|
10
10
|
requires-python = ">=3.8"
|
modelproof-1.0.0/PKG-INFO
DELETED
|
@@ -1,111 +0,0 @@
|
|
|
1
|
-
Metadata-Version: 2.5
|
|
2
|
-
Name: modelproof
|
|
3
|
-
Version: 1.0.0
|
|
4
|
-
Summary: Zero-persistence client-side and CLI forensic scanner to detect LLM model spoofing, masking, and downgrading.
|
|
5
|
-
Project-URL: Homepage, https://github.com/siumik/ai-model-mask-checker
|
|
6
|
-
Project-URL: Repository, https://github.com/siumik/ai-model-mask-checker
|
|
7
|
-
Author: ModelProof Contributors
|
|
8
|
-
License-Expression: MIT
|
|
9
|
-
License-File: LICENSE
|
|
10
|
-
Keywords: ai,anthropic,benchmark,deepseek,forensics,llm,model-spoofing,openai,qwen,reverse-proxy,security
|
|
11
|
-
Classifier: Development Status :: 5 - Production/Stable
|
|
12
|
-
Classifier: Intended Audience :: Developers
|
|
13
|
-
Classifier: License :: OSI Approved :: MIT License
|
|
14
|
-
Classifier: Programming Language :: Python :: 3
|
|
15
|
-
Classifier: Programming Language :: Python :: 3.8
|
|
16
|
-
Classifier: Programming Language :: Python :: 3.9
|
|
17
|
-
Classifier: Programming Language :: Python :: 3.10
|
|
18
|
-
Classifier: Programming Language :: Python :: 3.11
|
|
19
|
-
Classifier: Programming Language :: Python :: 3.12
|
|
20
|
-
Classifier: Programming Language :: Python :: 3.13
|
|
21
|
-
Classifier: Topic :: Scientific/Engineering :: Artificial Intelligence
|
|
22
|
-
Classifier: Topic :: Security
|
|
23
|
-
Requires-Python: >=3.8
|
|
24
|
-
Description-Content-Type: text/markdown
|
|
25
|
-
|
|
26
|
-
# ModelProof 🛡️ (v1.0.0)
|
|
27
|
-
> Zero-persistence client-side and CLI forensic scanner to detect LLM model spoofing, proxy masking, and silent downgrades.
|
|
28
|
-
|
|
29
|
-
Available as:
|
|
30
|
-
- **Web App**: 100% Client-side sandbox deployable on GitHub Pages.
|
|
31
|
-
- **Node.js CLI**: Run instantly via `npx modelproof`.
|
|
32
|
-
- **Python CLI**: Run via `pip install modelproof` or `python -m modelproof.cli`.
|
|
33
|
-
|
|
34
|
-
---
|
|
35
|
-
|
|
36
|
-
## ⚡ 10-Vector Detection Matrix
|
|
37
|
-
|
|
38
|
-
1. **Spatial Logic & Character Horizon**: Obfuscated character counting trap (`'s-t-r-a-w-b-e-r-r-y'`) + runtime multiplication trap.
|
|
39
|
-
2. **Tokenizer Usage & BPE Precision**: Multi-byte Unicode sequence testing token discrepancy and upstream token stripping.
|
|
40
|
-
3. **Internal System Identity & Vendor Breakout**: Adversarial prompts probing foundational weights and vendor disavowal.
|
|
41
|
-
4. **Streaming Telemetry & Speed Profiling (TTFT & TPS)**: Real-time SSE stream parser measuring Time-to-First-Token and Tokens-per-Second.
|
|
42
|
-
5. **Negative Constraint Compliance**: Enforces strict negative constraints (zero fluff, raw CSV/SVG).
|
|
43
|
-
6. **Strict Schema / Constrained Decoding**: Tests native JSON Schema strict parsing (crashes weak proxy engines).
|
|
44
|
-
7. **Glitched Token Embedding Anomaly**: Probes unspeakable tokens (`SolidGoldMagikarp`) tokenizer behavior.
|
|
45
|
-
8. **Temporal Cutoff Horizon (2024-H2)**: Validates late-2024 events (Nobel October 2024, Python 3.13).
|
|
46
|
-
9. **Reasoning CoT & Delimiter Trap**: Checks reasoning tokens vs `<think>` tags (detects DeepSeek-R1 masked as OpenAI o1).
|
|
47
|
-
10. **High-Order Type Logic**: Tests compiler-level Rust lifetime borrow checker & HRTB syntax.
|
|
48
|
-
|
|
49
|
-
---
|
|
50
|
-
|
|
51
|
-
## 💻 CLI Quickstart
|
|
52
|
-
|
|
53
|
-
### Option A: NPX (No installation required)
|
|
54
|
-
```bash
|
|
55
|
-
# Instant audit via npx
|
|
56
|
-
npx modelproof -u "https://my-custom-proxy.com/v1" -k "sk-..." -m "qwen-2.5-72b-instruct"
|
|
57
|
-
|
|
58
|
-
# Deep audit with all 10 vectors + JSON output
|
|
59
|
-
npx modelproof -u "https://api.openai.com/v1" -k "$OPENAI_API_KEY" -m "gpt-4o" --all --json
|
|
60
|
-
```
|
|
61
|
-
|
|
62
|
-
### Option B: Python (PIP)
|
|
63
|
-
```bash
|
|
64
|
-
# Install package
|
|
65
|
-
pip install modelproof
|
|
66
|
-
|
|
67
|
-
# Run audit
|
|
68
|
-
modelproof -u "https://my-custom-proxy.com/v1" -k "sk-..." -m "claude-3-5-sonnet-20241022"
|
|
69
|
-
|
|
70
|
-
# Audit catalog only
|
|
71
|
-
modelproof -u "https://my-custom-proxy.com/v1" -k "sk-..." --models-only
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
### CLI Options:
|
|
75
|
-
| Flag | Description | Default |
|
|
76
|
-
| :--- | :--- | :--- |
|
|
77
|
-
| `-u, --base-url` | Reverse proxy base URL | `https://api.openai.com/v1` |
|
|
78
|
-
| `-k, --key` | API Token / Key | `$OPENAI_API_KEY` |
|
|
79
|
-
| `-m, --model` | Claimed model profile | `claude-3-5-sonnet-20241022` |
|
|
80
|
-
| `-p, --protocol` | Protocol wire schema (`auto`, `openai`, `anthropic`) | `auto` |
|
|
81
|
-
| `-a, --all` | Run all 10 vectors (default: 8 fast vectors) | `false` |
|
|
82
|
-
| `--models-only` | Audit `/v1/models` catalog only | `false` |
|
|
83
|
-
| `--lang` | Report language (`en`, `id`) | `en` |
|
|
84
|
-
| `--json` | Output pure JSON for CI/CD | `false` |
|
|
85
|
-
| `--timeout` | Request timeout in seconds | `30` |
|
|
86
|
-
|
|
87
|
-
---
|
|
88
|
-
|
|
89
|
-
## 🌐 Web App Deployment (GitHub Pages)
|
|
90
|
-
|
|
91
|
-
1. Push this directory to your GitHub repository:
|
|
92
|
-
```bash
|
|
93
|
-
git add .
|
|
94
|
-
git commit -m "feat: initial release"
|
|
95
|
-
git branch -M main
|
|
96
|
-
git push -u origin main
|
|
97
|
-
```
|
|
98
|
-
2. In your GitHub repo:
|
|
99
|
-
- Go to **Settings** → **Pages**.
|
|
100
|
-
- Under **Build and deployment** > **Source**, choose **Deploy from a branch**.
|
|
101
|
-
- Select branch `main` and folder `/ (root)`.
|
|
102
|
-
- Click **Save**.
|
|
103
|
-
3. Your app is live at `https://<your-username>.github.io/<your-repo-name>/`.
|
|
104
|
-
|
|
105
|
-
---
|
|
106
|
-
|
|
107
|
-
## 🔒 Privacy & Security
|
|
108
|
-
|
|
109
|
-
- **Zero Telemetry**: All requests travel strictly between your client and your designated proxy.
|
|
110
|
-
- **In-Memory**: API keys are never persisted or shared.
|
|
111
|
-
- **Open Source**: Full code inspection available under the MIT License.
|
modelproof-1.0.0/README.md
DELETED
|
@@ -1,86 +0,0 @@
|
|
|
1
|
-
# ModelProof 🛡️ (v1.0.0)
|
|
2
|
-
> Zero-persistence client-side and CLI forensic scanner to detect LLM model spoofing, proxy masking, and silent downgrades.
|
|
3
|
-
|
|
4
|
-
Available as:
|
|
5
|
-
- **Web App**: 100% Client-side sandbox deployable on GitHub Pages.
|
|
6
|
-
- **Node.js CLI**: Run instantly via `npx modelproof`.
|
|
7
|
-
- **Python CLI**: Run via `pip install modelproof` or `python -m modelproof.cli`.
|
|
8
|
-
|
|
9
|
-
---
|
|
10
|
-
|
|
11
|
-
## ⚡ 10-Vector Detection Matrix
|
|
12
|
-
|
|
13
|
-
1. **Spatial Logic & Character Horizon**: Obfuscated character counting trap (`'s-t-r-a-w-b-e-r-r-y'`) + runtime multiplication trap.
|
|
14
|
-
2. **Tokenizer Usage & BPE Precision**: Multi-byte Unicode sequence testing token discrepancy and upstream token stripping.
|
|
15
|
-
3. **Internal System Identity & Vendor Breakout**: Adversarial prompts probing foundational weights and vendor disavowal.
|
|
16
|
-
4. **Streaming Telemetry & Speed Profiling (TTFT & TPS)**: Real-time SSE stream parser measuring Time-to-First-Token and Tokens-per-Second.
|
|
17
|
-
5. **Negative Constraint Compliance**: Enforces strict negative constraints (zero fluff, raw CSV/SVG).
|
|
18
|
-
6. **Strict Schema / Constrained Decoding**: Tests native JSON Schema strict parsing (crashes weak proxy engines).
|
|
19
|
-
7. **Glitched Token Embedding Anomaly**: Probes unspeakable tokens (`SolidGoldMagikarp`) tokenizer behavior.
|
|
20
|
-
8. **Temporal Cutoff Horizon (2024-H2)**: Validates late-2024 events (Nobel October 2024, Python 3.13).
|
|
21
|
-
9. **Reasoning CoT & Delimiter Trap**: Checks reasoning tokens vs `<think>` tags (detects DeepSeek-R1 masked as OpenAI o1).
|
|
22
|
-
10. **High-Order Type Logic**: Tests compiler-level Rust lifetime borrow checker & HRTB syntax.
|
|
23
|
-
|
|
24
|
-
---
|
|
25
|
-
|
|
26
|
-
## 💻 CLI Quickstart
|
|
27
|
-
|
|
28
|
-
### Option A: NPX (No installation required)
|
|
29
|
-
```bash
|
|
30
|
-
# Instant audit via npx
|
|
31
|
-
npx modelproof -u "https://my-custom-proxy.com/v1" -k "sk-..." -m "qwen-2.5-72b-instruct"
|
|
32
|
-
|
|
33
|
-
# Deep audit with all 10 vectors + JSON output
|
|
34
|
-
npx modelproof -u "https://api.openai.com/v1" -k "$OPENAI_API_KEY" -m "gpt-4o" --all --json
|
|
35
|
-
```
|
|
36
|
-
|
|
37
|
-
### Option B: Python (PIP)
|
|
38
|
-
```bash
|
|
39
|
-
# Install package
|
|
40
|
-
pip install modelproof
|
|
41
|
-
|
|
42
|
-
# Run audit
|
|
43
|
-
modelproof -u "https://my-custom-proxy.com/v1" -k "sk-..." -m "claude-3-5-sonnet-20241022"
|
|
44
|
-
|
|
45
|
-
# Audit catalog only
|
|
46
|
-
modelproof -u "https://my-custom-proxy.com/v1" -k "sk-..." --models-only
|
|
47
|
-
```
|
|
48
|
-
|
|
49
|
-
### CLI Options:
|
|
50
|
-
| Flag | Description | Default |
|
|
51
|
-
| :--- | :--- | :--- |
|
|
52
|
-
| `-u, --base-url` | Reverse proxy base URL | `https://api.openai.com/v1` |
|
|
53
|
-
| `-k, --key` | API Token / Key | `$OPENAI_API_KEY` |
|
|
54
|
-
| `-m, --model` | Claimed model profile | `claude-3-5-sonnet-20241022` |
|
|
55
|
-
| `-p, --protocol` | Protocol wire schema (`auto`, `openai`, `anthropic`) | `auto` |
|
|
56
|
-
| `-a, --all` | Run all 10 vectors (default: 8 fast vectors) | `false` |
|
|
57
|
-
| `--models-only` | Audit `/v1/models` catalog only | `false` |
|
|
58
|
-
| `--lang` | Report language (`en`, `id`) | `en` |
|
|
59
|
-
| `--json` | Output pure JSON for CI/CD | `false` |
|
|
60
|
-
| `--timeout` | Request timeout in seconds | `30` |
|
|
61
|
-
|
|
62
|
-
---
|
|
63
|
-
|
|
64
|
-
## 🌐 Web App Deployment (GitHub Pages)
|
|
65
|
-
|
|
66
|
-
1. Push this directory to your GitHub repository:
|
|
67
|
-
```bash
|
|
68
|
-
git add .
|
|
69
|
-
git commit -m "feat: initial release"
|
|
70
|
-
git branch -M main
|
|
71
|
-
git push -u origin main
|
|
72
|
-
```
|
|
73
|
-
2. In your GitHub repo:
|
|
74
|
-
- Go to **Settings** → **Pages**.
|
|
75
|
-
- Under **Build and deployment** > **Source**, choose **Deploy from a branch**.
|
|
76
|
-
- Select branch `main` and folder `/ (root)`.
|
|
77
|
-
- Click **Save**.
|
|
78
|
-
3. Your app is live at `https://<your-username>.github.io/<your-repo-name>/`.
|
|
79
|
-
|
|
80
|
-
---
|
|
81
|
-
|
|
82
|
-
## 🔒 Privacy & Security
|
|
83
|
-
|
|
84
|
-
- **Zero Telemetry**: All requests travel strictly between your client and your designated proxy.
|
|
85
|
-
- **In-Memory**: API keys are never persisted or shared.
|
|
86
|
-
- **Open Source**: Full code inspection available under the MIT License.
|
|
File without changes
|
|
File without changes
|
|
File without changes
|