tokenrace 1.0.0__tar.gz

tokenrace-1.0.0/.gitignore

@@ -0,0 +1,18 @@
+# Ambiente virtuale e cache Python
+.venv/
+venv/
+__pycache__/
+*.pyc
+
+# Pacchetti generati da uv build
+dist/
+
+# Risultati esportati
+*.csv
+
+# Configurazioni locali di editor e strumenti
+.claude/
+.vscode/
+.idea/
+.DS_Store
+.streamlit/secrets.toml

tokenrace-1.0.0/.streamlit/config.toml

@@ -0,0 +1,11 @@
+# Nessuna statistica d'uso inviata a Streamlit: l'app resta tutta locale.
+[browser]
+gatherUsageStats = false
+
+[client]
+# Nasconde il pulsante "Deploy" e lascia solo il menu essenziale.
+toolbarMode = "minimal"
+
+[theme]
+primaryColor = "#FF5A1F"
+baseRadius = "0.6rem"

tokenrace-1.0.0/CHANGELOG.md

@@ -0,0 +1,18 @@
+# Changelog
+
+## 1.0.0 — 2026-11-10
+
+First public release.
+
+### Added
+
+- Install and run with `uvx tokenrace`, with `--port` and `--lang` options.
+- Speed test for local OpenAI-compatible LLM inference servers, with the same client-side measuring code for all: TTFT, generation and effective tokens per second, total time, input and output tokens, estimated prompt speed, system RAM.
+- 15 preconfigured servers, including Ollama, LM Studio, Rapid-MLX, oMLX, llama.cpp and vLLM, with editable URL and optional API key.
+- Three preset tests: Short, Long (≈8,000 tokens of context) and Cache.
+- Reasoning and answer tokens shown separately, and a "No reasoning" switch with the right parameter for each server.
+- Streaming response and live metrics, with the status of each run.
+- Session summary with median, min and max (cold run excluded from the median), chart per server and model, CSV export.
+- English and Italian interface; servers, defaults, prompts and texts in editable TOML files.
+- TokenRace logo and mascot.
+- Form to share results in GitHub Discussions.

tokenrace-1.0.0/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 ernext
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

tokenrace-1.0.0/PKG-INFO

@@ -0,0 +1,203 @@
+Metadata-Version: 2.5
+Name: tokenrace
+Version: 1.0.0
+Summary: LLM inference speed test: compare local inference servers on your own hardware, with the same measuring code for all.
+Project-URL: Homepage, https://github.com/ernext/tokenrace
+Project-URL: Repository, https://github.com/ernext/tokenrace
+Project-URL: Issues, https://github.com/ernext/tokenrace/issues
+Project-URL: Changelog, https://github.com/ernext/tokenrace/blob/main/CHANGELOG.md
+License-Expression: MIT
+License-File: LICENSE
+Keywords: benchmark,inference,llm,lm-studio,local-llm,mlx,ollama,speed-test,streamlit
+Requires-Python: >=3.11
+Requires-Dist: pandas==3.1.0
+Requires-Dist: psutil==7.2.2
+Requires-Dist: requests==2.34.2
+Requires-Dist: streamlit==1.66.1
+Description-Content-Type: text/markdown
+
+<p align="center"><img src="https://raw.githubusercontent.com/ernext/tokenrace/main/assets/logo/tokenrace-mascot.png" alt="TokenRace mascot: a running token" width="190"></p>
+
+# TokenRace — Local LLM Speed Test
+
+[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org)
+[![Streamlit](https://img.shields.io/badge/Streamlit-app-FF4B4B?logo=streamlit&logoColor=white)](https://streamlit.io)
+[![License MIT](https://img.shields.io/badge/license-MIT-green)](LICENSE)
+[![Servers](https://img.shields.io/badge/servers-15-FF5A1F)](#supported-servers)
+[![Languages](https://img.shields.io/badge/UI-English%20%7C%20Italiano-lightgrey)](#languages)
+
+**English** · [Italiano](README.it.md)
+
+An **LLM inference speed test**: compare local inference servers (Ollama, LM Studio, Rapid-MLX, oMLX, llama.cpp, vLLM and more) on your own hardware, with **the exact same measuring code** for all of them.
+
+Speed depends on the combination of model, inference server and hardware: there is no universal winner, and numbers measured elsewhere, with other rules, rarely apply to your machine. **Measure first, then choose.**
+
+A small [Streamlit](https://streamlit.io) app: pick a server and a model, start the test and watch TTFT, tokens per second, total time and RAM live, with a summary, a chart and CSV export.
+
+<p align="center">
+  <img src="https://raw.githubusercontent.com/ernext/tokenrace/main/docs/tokenrace-en.png" alt="TokenRace: LFM3 1.7B at 8-bit on Rapid-MLX, oMLX and Ollama, with streaming response, live metrics and summary" width="900">
+</p>
+<p align="center"><sub>LFM3 1.7B at 8-bit on Rapid-MLX and oMLX (same MLX weights) and on Ollama (mxfp8), "Short" prompt, no reasoning, 3 runs on one Mac. A single example, not a ranking.</sub></p>
+
+## Features
+
+- Same method for every server: streaming `POST /chat/completions`, timed on the client with `time.perf_counter()`, never with the servers' own stats.
+- Three preset tests: **Short** (long answer), **Long** (≈8,000 tokens of context) and **Cache** (identical prompt on every run).
+- Streaming response and live metrics.
+- Reasoning models timed correctly, with reasoning and answer tokens shown separately, plus a **No reasoning** switch.
+- First run marked "cold" and excluded from the median.
+- Summary with median, min and max, a chart per server and model, CSV export.
+- English and Italian interface; servers, defaults and texts in easy-to-edit TOML files.
+- Fully local: no database, accounts or telemetry.
+
+## Example results
+
+LFM3 1.7B at 8-bit on an Apple M5 Pro (24 GB), "Short" prompt, 512 tokens, median of 3 warm runs. Effective speed in tok/s.
+
+| Server | IT, reasoning on | IT, no reasoning | EN, reasoning on | EN, no reasoning |
+|---|---:|---:|---:|---:|
+| oMLX | **131.8** | **129.1** | 123.1 | **117.2** |
+| Rapid-MLX | 128.8 | 126.8 | **127.0** | 103.2 |
+| Ollama (mxfp8) | 118.0 | 113.9 | 101.1 | 104.4 |
+
+- oMLX and Rapid-MLX use the same MLX weights (`LiquidAI/LFM3-1.7B-MLX-8bit`).
+- With reasoning on, oMLX and Ollama spend all 512 tokens reasoning, while Rapid-MLX does not reason with this model: those columns are not a like-for-like comparison.
+- The same test can vary by 10–20% from one minute to the next on the same Mac: treat small gaps as a tie and run your own tests.
+
+Got different numbers on your hardware? **[Share your results](https://github.com/ernext/tokenrace/discussions/categories/share-your-results)**.
+
+## Requirements
+
+- Python 3.11 or newer
+- At least one local OpenAI-compatible LLM server, already running
+
+Built for Apple Silicon Macs, works on Linux and Windows too.
+
+## Install
+
+The quickest way needs only [uv](https://docs.astral.sh/uv/getting-started/installation/) (macOS/Linux: `curl -LsSf https://astral.sh/uv/install.sh | sh`):
+
+```bash
+uvx tokenrace
+```
+
+The app opens at http://localhost:8501. Options: `uvx tokenrace --port 8600 --lang it`.
+
+Alternatively, from the source code:
+
+```bash
+git clone https://github.com/ernext/tokenrace.git
+cd tokenrace
+python3 -m venv .venv
+source .venv/bin/activate        # Windows: .venv\Scripts\activate
+pip install -r requirements.txt
+```
+
+## Run
+
+If you installed from the source code:
+
+```bash
+streamlit run app.py
+```
+
+Then open http://localhost:8501. The interface opens in English; switch language with the selector at the top right, or open http://localhost:8501/?lang=it.
+
+## Supported servers
+
+TokenRace never installs or starts servers: start them yourself. They all speak the same OpenAI-compatible API, and the app shows the start command of the selected server.
+
+If your server is not listed but is OpenAI-compatible, pick any entry and change the base URL, or add it to `config/servers.toml` (see below).
+
+| Server | Default URL | Start | Notes |
+|---|---|---|---|
+| <img src="https://github.com/ollama.png?size=40" width="20" alt="">&nbsp;[Ollama](https://ollama.com) | `http://localhost:11434/v1` | `ollama serve` | Mac, Linux, Windows. |
+| <img src="https://github.com/lmstudio-ai.png?size=40" width="20" alt="">&nbsp;[LM Studio](https://lmstudio.ai) | `http://localhost:1234/v1` | `lms server start && lms load <model>` | Mac, Linux, Windows. |
+| <img src="https://raw.githubusercontent.com/raullenchai/Rapid-MLX/e2114cf9e7c16dfa3c668832ea6780c02779ca89/docs/assets/logo.png" width="20" alt="">&nbsp;[Rapid-MLX](https://github.com/raullenchai/Rapid-MLX) | `http://localhost:7659/v1` | `rapid-mlx serve <model> --port 8000` | Mac only. The Rapid-MLX Desktop app uses port 7659 and requires the API key; the CLI uses 8000. |
+| <img src="https://raw.githubusercontent.com/jundot/omlx/2165e75726ba83d9651dbaced61c97728da77bba/apps/omlx-web/omlx_web/static/favicon.svg" width="20" alt="">&nbsp;[oMLX](https://github.com/jundot/omlx) | `http://localhost:8000/v1` | — | Mac only. Start it from the oMLX app: it uses port 8000 and requires the API key set in the app. |
+| <img src="https://github.com/ml-explore.png?size=40" width="20" alt="">&nbsp;[mlx_lm.server](https://github.com/ml-explore/mlx-lm) | `http://localhost:8080/v1` | `mlx_lm.server --model <model> --port 8080` | Mac only. The reference MLX server, with no extra optimizations. |
+| <img src="https://github.com/ggml-org.png?size=40" width="20" alt="">&nbsp;[llama.cpp (llama-server)](https://github.com/ggml-org/llama.cpp) | `http://localhost:8080/v1` | `llama-server -m <file.gguf> --port 8080` | Mac, Linux, Windows. The engine used by Ollama and LM Studio. |
+| <img src="https://github.com/vllm-project.png?size=40" width="20" alt="">&nbsp;[vLLM](https://github.com/vllm-project/vllm) | `http://localhost:8000/v1` | `vllm serve <model>` | Linux with NVIDIA/AMD GPU. The most widely used in production. |
+| <img src="https://github.com/sgl-project.png?size=40" width="20" alt="">&nbsp;[SGLang](https://github.com/sgl-project/sglang) | `http://localhost:30000/v1` | `python -m sglang.launch_server --model-path <model>` | Linux with NVIDIA/AMD GPU. |
+| <img src="https://github.com/lemonade-sdk.png?size=40" width="20" alt="">&nbsp;[Lemonade](https://github.com/lemonade-sdk/lemonade) | `http://localhost:13305/api/v1` | `lemonade-server serve` | Windows/Linux, optimized for AMD Ryzen AI. Older versions use port 8000. |
+| <img src="https://github.com/docker.png?size=40" width="20" alt="">&nbsp;[Docker Model Runner](https://docs.docker.com/ai/model-runner/) | `http://localhost:12434/engines/v1` | `docker desktop enable model-runner --tcp 12434` | Mac, Linux, Windows. Host-side TCP access must be enabled. |
+| <img src="https://github.com/unslothai.png?size=40" width="20" alt="">&nbsp;[Unsloth Studio](https://github.com/unslothai/unsloth) | `http://localhost:8888/v1` | `unsloth run --model <model>` | Port 8888 comes from examples: «unsloth run» prints the real one at startup. Needs the sk-unsloth-… API key (Studio → Settings → API). |
+| <img src="https://github.com/go-skynet.png?size=40" width="20" alt="">&nbsp;[LocalAI](https://github.com/mudler/LocalAI) | `http://localhost:8080/v1` | `local-ai run <model>` | Mac, Linux, Windows. |
+| <img src="https://github.com/janhq.png?size=40" width="20" alt="">&nbsp;[Jan](https://github.com/janhq/jan) | `http://localhost:1337/v1` | — | Mac, Linux, Windows. In Jan: Settings → Local API Server → Start. Requires the API key set in Jan. |
+| <img src="https://raw.githubusercontent.com/LostRuins/koboldcpp/6d2fedf8624e5f05926af3191f9efdcb13c4133c/niko.ico" width="20" alt="">&nbsp;[KoboldCpp](https://github.com/LostRuins/koboldcpp) | `http://localhost:5001/v1` | `koboldcpp --model <file.gguf>` | Mac, Linux, Windows. |
+| <img src="https://raw.githubusercontent.com/oobabooga/textgen/c93f8871239550de2ccfe1e95d469aa82616f07e/css/icon.png" width="20" alt="">&nbsp;[text-generation-webui](https://github.com/oobabooga/textgen) | `http://localhost:5000/v1` | — | Mac, Linux, Windows. Start it with --api. On Mac port 5000 is taken by AirPlay Receiver: use --api-port 5005 and change the URL. |
+
+Notes:
+- Several servers share port 8000 or 8080: run one at a time, or change the port with `--port` and update the URL in the app.
+- The **API key** field is optional: it is needed for Rapid-MLX Desktop, oMLX, Jan, Unsloth Studio or vLLM started with `--api-key`. It is never saved or exported.
+
+## Metrics
+
+- **TTFT**: time from sending the request to the first non-empty chunk (content or reasoning).
+- **Generation (tok/s)**: (output tokens − 1) / (last token − first token).
+- **Effective speed (tok/s)**: output tokens / (last token − request sent).
+- **Prompt processing (estimate)**: input tokens / TTFT.
+- **RAM**: memory in use on the whole system, sampled every 0.5 s (value before the test and peak).
+
+Tokens come from the `usage` field. If the server does not send it, chunks are counted and an "estimated count" warning appears.
+
+## Presets
+
+- **Short**: short prompt, long answer.
+- **Long**: about 8,000 tokens of context, short answer.
+- **Cache**: the same long prompt, identical on every run.
+
+In Short and Long every run starts with a unique ID, so prompt caching does not skew the result. Prompts exist in both languages: compare only runs made with the same prompt language (the CSV records it).
+
+The **No reasoning** switch sends a different parameter depending on the server (✅ = tested):
+- Ollama: `reasoning_effort: "none"` ✅
+- LM Studio: `reasoning_effort: "none"` plus `chat_template_kwargs` ✅
+- oMLX: `chat_template_kwargs: {"enable_thinking": false}` ✅
+- text-generation-webui: `enable_thinking: false`
+- all others (llama.cpp, MLX, vLLM, SGLang…): `chat_template_kwargs: {"enable_thinking": false}`
+
+Some models reason anyway: the app shows a warning in that case.
+
+## Project structure
+
+```
+app.py               Streamlit interface
+tokenrace/
+  bench.py           measuring logic (streaming, timing, RAM, token counts), no Streamlit
+  cli.py             the `tokenrace` command
+  config.py          reads the TOML files
+  i18n.py            translations
+config/
+  servers.toml       preconfigured servers
+  defaults.toml      default parameters, timeouts, languages, colours
+  prompts.toml       preset prompt texts, per language
+locales/
+  en.toml, it.toml   interface texts
+assets/style.css     visual style
+assets/logo/         mascot, icon and favicon (SVG + PNG)
+tools/make_logo.py   regenerates the logo: python tools/make_logo.py --png
+```
+
+**Add a server**: add a `[[server]]` block to `config/servers.toml` and its note under `[server_notes]` in each `locales/*.toml` file.
+
+<a id="languages"></a>**Add a language**: copy `locales/en.toml` to `locales/<code>.toml`, translate it, add the prompt texts in `config/prompts.toml` and the code to `languages` in `config/defaults.toml`.
+
+## Limitations
+
+- RAM is the whole system's, not just the server's.
+- It measures speed, not the quality of the answers.
+- Prompt processing speed is an estimate (input tokens / TTFT) and includes network latency and any model loading.
+- Default ports for Unsloth Studio and text-generation-webui come from examples in their docs: check them when you start the server.
+
+## Contributing
+
+Issues and pull requests are welcome, especially:
+- confirmations or fixes for ports and "no reasoning" parameters of servers not yet tested;
+- new OpenAI-compatible servers;
+- new translations.
+
+If TokenRace is useful to you, a ⭐ on GitHub is always appreciated.
+
+## License
+
+[MIT](LICENSE). Server logos belong to their respective projects and are shown only to identify them.

tokenrace-1.0.0/README.it.md

@@ -0,0 +1,185 @@
+<p align="center"><img src="assets/logo/tokenrace-mascot.svg" alt="Mascotte di TokenRace: un gettone che corre" width="190"></p>
+
+# TokenRace — Local LLM Speed Test
+
+[![Python 3.11+](https://img.shields.io/badge/python-3.11%2B-blue)](https://www.python.org)
+[![Streamlit](https://img.shields.io/badge/Streamlit-app-FF4B4B?logo=streamlit&logoColor=white)](https://streamlit.io)
+[![Licenza MIT](https://img.shields.io/badge/licenza-MIT-green)](LICENSE)
+[![Servers](https://img.shields.io/badge/servers-15-FF5A1F)](#server-supportati)
+[![Languages](https://img.shields.io/badge/UI-English%20%7C%20Italiano-lightgrey)](#lingue)
+
+[English](README.md) · **Italiano**
+
+Un **LLM inference speed test**: confronta gli inference server locali (Ollama, LM Studio, Rapid-MLX, oMLX, llama.cpp, vLLM e altri) sul tuo hardware, con **lo stesso identico codice di misura** per tutti.
+
+La velocità dipende dalla combinazione di modello, inference server e hardware: non c'è un vincitore universale, e i numeri misurati altrove, con altre regole, raramente valgono per la tua macchina. **Prima misuri, poi scegli.**
+
+Una piccola app [Streamlit](https://streamlit.io): scegli server e modello, avvii il test e vedi in tempo reale TTFT, token al secondo, tempo totale e RAM, con riepilogo, grafico ed export CSV.
+
+<p align="center">
+  <img src="docs/tokenrace-it.png" alt="TokenRace: LFM3 1.7B a 8 bit su Rapid-MLX, oMLX e Ollama, con risposta in streaming, misure e riepilogo" width="900">
+</p>
+<p align="center"><sub>LFM3 1.7B a 8 bit su Rapid-MLX e oMLX (stessi pesi MLX) e su Ollama (mxfp8), prompt "Breve", senza ragionamento, 3 ripetizioni su un Mac. Un singolo esempio, non una classifica.</sub></p>
+
+## Caratteristiche
+
+- Stesso metodo per tutti i server: `POST /chat/completions` in streaming, cronometrato lato client con `time.perf_counter()`, mai con le statistiche dei server.
+- Tre prove pronte: **Breve** (risposta lunga), **Lungo** (≈8.000 token di contesto) e **Cache** (prompt identico a ogni ripetizione).
+- Risposta in streaming e misure in tempo reale.
+- Modelli con ragionamento cronometrati correttamente, con token di ragionamento e di risposta separati, e interruttore **Senza ragionamento**.
+- Prima ripetizione segnata "a freddo" ed esclusa dalla mediana.
+- Riepilogo con mediana, minimo e massimo, grafico per server e modello ed export CSV.
+- Interfaccia in inglese e italiano; server, valori predefiniti e testi in file TOML facili da modificare.
+- Tutto locale: niente database, account o telemetria.
+
+## Risultati di esempio
+
+LFM3 1.7B a 8 bit su Apple M5 Pro (24 GB), prompt "Breve", 512 token, mediana di 3 ripetizioni a caldo. Velocità effettiva in tok/s.
+
+| Server | IT, ragionamento attivo | IT, senza ragionamento | EN, ragionamento attivo | EN, senza ragionamento |
+|---|---:|---:|---:|---:|
+| oMLX | **131,8** | **129,1** | 123,1 | **117,2** |
+| Rapid-MLX | 128,8 | 126,8 | **127,0** | 103,2 |
+| Ollama (mxfp8) | 118,0 | 113,9 | 101,1 | 104,4 |
+
+- oMLX e Rapid-MLX usano gli stessi pesi MLX (`LiquidAI/LFM3-1.7B-MLX-8bit`).
+- Con il ragionamento attivo oMLX e Ollama passano tutti i 512 token a ragionare, mentre Rapid-MLX con questo modello non ragiona: quelle colonne non sono un confronto alla pari.
+- Lo stesso test può variare del 10–20% da un minuto all'altro sullo stesso Mac: considera pari le differenze piccole e fai le tue prove.
+
+Hai ottenuto numeri diversi sul tuo hardware? **[Share your results](https://github.com/ernext/tokenrace/discussions/categories/share-your-results)**.
+
+## Requisiti
+
+- Python 3.11 o superiore
+- Almeno un server LLM locale compatibile OpenAI, già avviato
+
+Pensata per Mac con Apple Silicon, funziona anche su Linux e Windows.
+
+## Installazione
+
+Il modo più rapido richiede solo [uv](https://docs.astral.sh/uv/getting-started/installation/) (macOS/Linux: `curl -LsSf https://astral.sh/uv/install.sh | sh`):
+
+```bash
+uvx tokenrace
+```
+
+L'app si apre su http://localhost:8501. Opzioni: `uvx tokenrace --port 8600 --lang it`.
+
+In alternativa, dal codice sorgente:
+
+```bash
+git clone https://github.com/ernext/tokenrace.git
+cd tokenrace
+python3 -m venv .venv
+source .venv/bin/activate        # Windows: .venv\Scripts\activate
+pip install -r requirements.txt
+```
+
+## Avvio
+
+Se hai installato dal codice sorgente:
+
+```bash
+streamlit run app.py
+```
+
+Poi apri http://localhost:8501. L'interfaccia si apre in inglese: cambia lingua con il selettore in alto a destra, oppure apri direttamente http://localhost:8501/?lang=it.
+
+## Server supportati
+
+TokenRace non installa né avvia i server: vanno avviati a parte. Tutti parlano la stessa API compatibile OpenAI, e l'app mostra il comando di avvio del server scelto.
+
+Se il tuo server non è in elenco ma è compatibile OpenAI, scegline uno qualsiasi e cambia l'URL base, oppure aggiungilo in `config/servers.toml` (vedi sotto).
+
+| Server | URL predefinito | Avvio | Note |
+|---|---|---|---|
+| <img src="https://github.com/ollama.png?size=40" width="20" alt="">&nbsp;[Ollama](https://ollama.com) | `http://localhost:11434/v1` | `ollama serve` | Mac, Linux, Windows. |
+| <img src="https://github.com/lmstudio-ai.png?size=40" width="20" alt="">&nbsp;[LM Studio](https://lmstudio.ai) | `http://localhost:1234/v1` | `lms server start && lms load <model>` | Mac, Linux, Windows. |
+| <img src="https://raw.githubusercontent.com/raullenchai/Rapid-MLX/e2114cf9e7c16dfa3c668832ea6780c02779ca89/docs/assets/logo.png" width="20" alt="">&nbsp;[Rapid-MLX](https://github.com/raullenchai/Rapid-MLX) | `http://localhost:7659/v1` | `rapid-mlx serve <model> --port 8000` | Solo Mac. L'app Rapid-MLX Desktop usa la porta 7659 e richiede la API key; la CLI usa la 8000. |
+| <img src="https://raw.githubusercontent.com/jundot/omlx/2165e75726ba83d9651dbaced61c97728da77bba/apps/omlx-web/omlx_web/static/favicon.svg" width="20" alt="">&nbsp;[oMLX](https://github.com/jundot/omlx) | `http://localhost:8000/v1` | — | Solo Mac. Avvialo dall'app oMLX: usa la porta 8000 e richiede la API key impostata nell'app. |
+| <img src="https://github.com/ml-explore.png?size=40" width="20" alt="">&nbsp;[mlx_lm.server](https://github.com/ml-explore/mlx-lm) | `http://localhost:8080/v1` | `mlx_lm.server --model <model> --port 8080` | Solo Mac. Server di riferimento MLX, senza ottimizzazioni extra. |
+| <img src="https://github.com/ggml-org.png?size=40" width="20" alt="">&nbsp;[llama.cpp (llama-server)](https://github.com/ggml-org/llama.cpp) | `http://localhost:8080/v1` | `llama-server -m <file.gguf> --port 8080` | Mac, Linux, Windows. Il motore usato da Ollama e LM Studio. |
+| <img src="https://github.com/vllm-project.png?size=40" width="20" alt="">&nbsp;[vLLM](https://github.com/vllm-project/vllm) | `http://localhost:8000/v1` | `vllm serve <model>` | Linux con GPU NVIDIA/AMD. Il più usato in produzione. |
+| <img src="https://github.com/sgl-project.png?size=40" width="20" alt="">&nbsp;[SGLang](https://github.com/sgl-project/sglang) | `http://localhost:30000/v1` | `python -m sglang.launch_server --model-path <model>` | Linux con GPU NVIDIA/AMD. |
+| <img src="https://github.com/lemonade-sdk.png?size=40" width="20" alt="">&nbsp;[Lemonade](https://github.com/lemonade-sdk/lemonade) | `http://localhost:13305/api/v1` | `lemonade-server serve` | Windows/Linux, ottimizzato per AMD Ryzen AI. Le versioni vecchie usano la porta 8000. |
+| <img src="https://github.com/docker.png?size=40" width="20" alt="">&nbsp;[Docker Model Runner](https://docs.docker.com/ai/model-runner/) | `http://localhost:12434/engines/v1` | `docker desktop enable model-runner --tcp 12434` | Mac, Linux, Windows. Serve l'accesso TCP dall'host attivo. |
+| <img src="https://github.com/unslothai.png?size=40" width="20" alt="">&nbsp;[Unsloth Studio](https://github.com/unslothai/unsloth) | `http://localhost:8888/v1` | `unsloth run --model <model>` | Porta 8888 presa dagli esempi: quella vera la stampa «unsloth run» all'avvio. Serve la API key sk-unsloth-… (Studio → Impostazioni → API). |
+| <img src="https://github.com/go-skynet.png?size=40" width="20" alt="">&nbsp;[LocalAI](https://github.com/mudler/LocalAI) | `http://localhost:8080/v1` | `local-ai run <model>` | Mac, Linux, Windows. |
+| <img src="https://github.com/janhq.png?size=40" width="20" alt="">&nbsp;[Jan](https://github.com/janhq/jan) | `http://localhost:1337/v1` | — | Mac, Linux, Windows. In Jan: Impostazioni → Local API Server → Avvia. Richiede la API key impostata in Jan. |
+| <img src="https://raw.githubusercontent.com/LostRuins/koboldcpp/6d2fedf8624e5f05926af3191f9efdcb13c4133c/niko.ico" width="20" alt="">&nbsp;[KoboldCpp](https://github.com/LostRuins/koboldcpp) | `http://localhost:5001/v1` | `koboldcpp --model <file.gguf>` | Mac, Linux, Windows. |
+| <img src="https://raw.githubusercontent.com/oobabooga/textgen/c93f8871239550de2ccfe1e95d469aa82616f07e/css/icon.png" width="20" alt="">&nbsp;[text-generation-webui](https://github.com/oobabooga/textgen) | `http://localhost:5000/v1` | — | Mac, Linux, Windows. Avvialo con --api. Su Mac la porta 5000 è occupata dal Ricevitore AirPlay: usa --api-port 5005 e cambia l'URL. |
+
+Note:
+- Diversi server condividono la porta 8000 o 8080: avviane uno alla volta, oppure cambia porta con `--port` e aggiorna l'URL nell'app.
+- Il campo **API key** è facoltativo: serve per Rapid-MLX Desktop, oMLX, Jan, Unsloth Studio o vLLM avviato con `--api-key`. Non viene mai salvato né esportato.
+
+## Misure
+
+- **TTFT**: tempo dall'invio della richiesta al primo chunk non vuoto (contenuto o ragionamento).
+- **Generazione (tok/s)**: (token output − 1) / (ultimo token − primo token).
+- **Velocità effettiva (tok/s)**: token output / (ultimo token − invio richiesta).
+- **Elaborazione prompt (stima)**: token input / TTFT.
+- **RAM**: memoria in uso dell'intero sistema, campionata ogni 0,5 s (valore prima del test e picco).
+
+I token arrivano dal campo `usage`. Se il server non lo fornisce, vengono contati i chunk e compare l'avviso "conteggio stimato".
+
+## Prompt pronti
+
+- **Breve**: prompt corto, risposta lunga.
+- **Lungo**: circa 8.000 token di contesto, risposta breve.
+- **Cache**: lo stesso prompt lungo, identico a ogni ripetizione.
+
+In Breve e Lungo ogni ripetizione inizia con un ID univoco, così la cache del prompt non altera il risultato. I prompt esistono in entrambe le lingue: confronta solo prove fatte con la stessa lingua del prompt (il CSV la registra).
+
+L'interruttore **Senza ragionamento** invia un parametro diverso a seconda del server (✅ = provato):
+- Ollama: `reasoning_effort: "none"` ✅
+- LM Studio: `reasoning_effort: "none"` più `chat_template_kwargs` ✅
+- oMLX: `chat_template_kwargs: {"enable_thinking": false}` ✅
+- text-generation-webui: `enable_thinking: false`
+- tutti gli altri (llama.cpp, MLX, vLLM, SGLang…): `chat_template_kwargs: {"enable_thinking": false}`
+
+Alcuni modelli ragionano comunque: in quel caso l'app mostra un avviso.
+
+## Struttura del progetto
+
+```
+app.py               interfaccia Streamlit
+tokenrace/
+  bench.py           logica di misura (streaming, cronometro, RAM, token), senza Streamlit
+  cli.py             il comando `tokenrace`
+  config.py          lettura dei file TOML
+  i18n.py            traduzioni
+config/
+  servers.toml       server preconfigurati
+  defaults.toml      parametri predefiniti, timeout, lingue, colori
+  prompts.toml       testi dei prompt pronti, per lingua
+locales/
+  en.toml, it.toml   testi dell'interfaccia
+assets/style.css     stile grafico
+assets/logo/         mascotte, icona e favicon (SVG + PNG)
+tools/make_logo.py   rigenera il logo: python tools/make_logo.py --png
+```
+
+**Aggiungere un server**: un blocco `[[server]]` in `config/servers.toml` e la sua nota sotto `[server_notes]` in ogni file `locales/*.toml`.
+
+<a id="lingue"></a>**Aggiungere una lingua**: copia `locales/en.toml` come `locales/<codice>.toml`, traducilo, aggiungi i testi dei prompt in `config/prompts.toml` e il codice in `languages` dentro `config/defaults.toml`.
+
+## Limiti
+
+- La RAM è quella dell'intero sistema, non del solo server.
+- Misura la velocità, non la qualità delle risposte.
+- La velocità di elaborazione del prompt è una stima (token input / TTFT) e include la latenza di rete e l'eventuale caricamento del modello.
+- Le porte predefinite di Unsloth Studio e text-generation-webui vengono da esempi nella documentazione: verificale all'avvio del server.
+
+## Contribuire
+
+Segnalazioni e pull request sono benvenute, in particolare:
+- conferme o correzioni su porte e parametri "senza ragionamento" dei server non ancora provati;
+- nuovi server compatibili OpenAI;
+- nuove traduzioni.
+
+Se TokenRace ti è utile, una ⭐ su GitHub fa sempre piacere.
+
+## Licenza
+
+[MIT](LICENSE). I loghi dei server appartengono ai rispettivi progetti e sono usati solo per identificarli.