@tokcalc/mcp-server 0.1.2 → 0.1.4

package/tool-results/bash_1789888175925_49c53ba3c61b.txt

@@ -1,191 +0,0 @@
-- **Caveats:** 6ND is an approximation; attention, sequence length, activation checkpointing, optimizer, packing, MoE routing, and communication alter real cost. LoRA ratios are not universal.
-
-## 12. Build-vs-buy calculator
-- **Definition:** Compare self-hosted GPU cost/latency/capacity against API pricing for the same input/output/workload distribution.
-- **User value:** Directly answers whether utilization, latency/SLO, privacy, or operational complexity justifies owning/renting GPUs.
-- **Complexity / priority:** Medium / Must-have.
-- **Formula implications:** Self-host cost/request = `(GPU_hourly_cost × wall_time + CPU/RAM/egress/ops amortization)/requests`; per-million-token cost = hourly_cost / effective_tokens_per_hour × 1e6. API cost = uncached input×input_rate + cached input×cached_rate + output×output_rate + tool/media charges. Break-even volume solves self-host fixed+variable = API variable.
-- **Required knobs:** utilization, concurrency, TTFT/TPOT SLO, prompt/output distributions, cache hit rate, quantization, replicas/HA, region/egress, idle/cold-start time, and API batch discounts.
-- **Caveats:** Prices change quickly and throughput benchmarks are not apples-to-apples; link every provider/model price and timestamp it. No universal demand statistic was verified, but this feature follows directly from the cost equations and existing tokcalc scope.
-
-## 13. Energy / carbon per million tokens
-- **Definition:** Attribute measured/estimated electricity and regional carbon intensity to prefill/decode work, then normalize by tokens.
-- **User value:** Sustainability reporting and energy-aware model/GPU selection; helps expose long-context/reasoning cost not visible in dollars.
-- **Complexity / priority:** Medium–High / Nice.
-- **Quantitative evidence:** A 2025 infrastructure-aware benchmark gives `Carbon (kgCO2e) = E_query × CIF` and separates prompt-level operational footprint; it reports ~1.14 gCO2e per 400-token query for Mistral Large 2 as aligned with Mistral's LCA, but this is workload/region-specific. [Hungry AI paper](https://arxiv.org/html/2505.09598v6) [Mistral LCA reference is linked in paper]
-- CodeCarbon measures/estimates CPU, GPU, and RAM electricity and applies regional carbon intensity. [CodeCarbon](https://github.com/mlco2/codecarbon)
-- **Formula implications:** `E_Wh = ∫ power_W dt / 3600`; `gCO2e = E_Wh × carbon_intensity_g/Wh × PUE`; report `gCO2e / 1e6 output_tokens` and optionally input/output separately. Include facility overhead and region as explicit fields.
-- **Caveats:** API providers rarely publish per-request power; GPU TDP is not measured draw; utilization, batching, PUE, cooling, and grid mix dominate. I found no stable universal gCO2e/M-token constant.
-
-## 14. Serverless GPU cold starts
-- **Definition:** Time from scale-to-zero request to a ready model worker/first token, including container/image pull, library init, weight transfer, CUDA graph capture, and model warmup.
-- **User value:** Quantify tail latency and economics for bursty workloads; compare warm pool, snapshot, quantization, and scale-to-zero strategies.
-- **Complexity / priority:** Medium / Nice.
-- **Quantitative evidence:** Modal says model weights can load from a Volume at 1–2 GB/s, roughly nearly 1 second of cold-start latency per GB of weights; its high-performance guide reports GPU memory snapshots can yield 10x cold-start reductions. [Modal high-performance LLM inference](https://modal.com/docs/guide/high-performance-llm-inference)
-- Modal's Ministral 3 test reports median cold start falling from ~118s to ~12s with GPU snapshotting. [Modal blog](https://modal.com/blog/mistral-3)
-- RunPod's vLLM cold-start guide says uncached image pulls can take 30–90s; official RunPod materials advertise sub-200ms FlashBoot for active endpoints, which should be treated as a warm/snapshot path, not a generic model boot. [RunPod guide](https://www.runpod.io/blog/cut-vllm-cold-starts-runpod-serverless) [RunPod product](https://www.runpod.io/product/serverless)
-- **Formula implications:** `cold_start ≈ image_pull + container_init + weight_load(model_GB / effective_GBps) + GPU_load + compile/capture + warmup`; amortize over requests per warm interval. Add p50/p95/p99, warm-worker fraction, snapshot support, model size, and provider.
-- **Caveats:** Marketing numbers are platform/configuration-specific; exact cold-start distributions for Replicate/RunPod/Modal vary by region, cached image/model, GPU availability, and model. Treat every provider value as timestamped benchmark input, not a constant.
-
-## Cross-cutting implementation schema
-Store per feature/model/provider: source URL, benchmark date, hardware, engine/version, model/quantization, prompt/output distributions, batch/concurrency, QPS, metric (TTFT/TPOT/ITL/throughput/cost/energy), and confidence. Keep measured benchmark multipliers separate from analytical estimates.
-
-## Gaps / failed attempts
-- vLLM docs and vendor pages were readable, but several arXiv abstract pages and older dynamic docs timed out in extraction; equivalent official pages or arXiv results were used where content was available.
-- No authoritative universal chunked-prefill speedup, agent tool-call latency multiplier, long-context 1M throughput multiplier, embedding market-demand figure, or universal energy/token constant was verified.
-- OpenAI's public marketing pricing page returned 403 in direct fetch; the official developer pricing/prompt-caching docs were available and should be used for current model rates.
-
-
----
-
-# Tokcalc Section B — implementation-ready model catalog
-
-**Cutoff:** 2026-09-20. **Status:** best effort within the declared HF/API universe; not exhaustive of every community checkpoint. Architecture fields are taken from direct config JSON when readable. HF `createdAt` is a repository-date proxy, not necessarily announcement date. `n/a` means not disclosed or inaccessible; it must remain null in the calculator.
-
-## Open LLM, coding, and reasoning rows
-
-|Official HF model|Family|Total / active params|Layers|Hidden|Q / KV heads|Head dim|Vocab|Context|Release/date proxy|Sources|
-|---|---|---:|---:|---:|---:|---:|---:|---:|---|---|
-|eta-llama/Llama-4-Scout-17B-16E-Instruct](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct)|Llama 4|[~109B / 17B](https://huggingface.co/blog/llama4-release)|n/a|n/a|n/a / n/a|n/a|n/a|[10M advertised](https://developer.meta.com/ai/models/llama-4/)|[2025-04-05](https://huggingface.co/api/models/meta-llama/Llama-4-Scout-17B-16E-Instruct)|[card](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct) · [config](https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct/raw/main/config.json)|
-|eta-llama/Llama-4-Maverick-17B-128E-Instruct](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct)|Llama 4|[~400B / 17B](https://huggingface.co/blog/llama4-release)|n/a|n/a|n/a / n/a|n/a|n/a|[1M advertised](https://developer.meta.com/ai/models/llama-4/)|[2025-04-05](https://huggingface.co/api/models/meta-llama/Llama-4-Maverick-17B-128E-Instruct)|[card](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct) · [config](https://huggingface.co/meta-llama/Llama-4-Maverick-17B-128E-Instruct/raw/main/config.json)|
-|eta-llama/Llama-4-Behemoth-17B-16E-Instruct](https://huggingface.co/meta-llama/Llama-4-Behemoth-17B-16E-Instruct)|Llama 4|[~2T / 288B](https://ai.meta.com/blog/llama-4-multimodal-intelligence)|n/a|n/a|n/a / n/a|n/a|n/a|n/a|[2025-04-05 announced](https://huggingface.co/api/models/meta-llama/Llama-4-Behemoth-17B-16E-Instruct)|[card](https://huggingface.co/meta-llama/Llama-4-Behemoth-17B-16E-Instruct) · [config](https://huggingface.co/meta-llama/Llama-4-Behemoth-17B-16E-Instruct/raw/main/config.json)|
-|[Qwen/Qwen3-0.6B](https://huggingface.co/Qwen/Qwen3-0.6B)|Qwen 3|[0.6B](https://huggingface.co/api/models/Qwen/Qwen3-0.6B)|[28](https://huggingface.co/Qwen/Qwen3-0.6B/raw/main/config.json)|[1024](https://huggingface.co/Qwen/Qwen3-0.6B/raw/main/config.json)|[16](https://huggingface.co/Qwen/Qwen3-0.6B/raw/main/config.json) / [8](https://huggingface.co/Qwen/Qwen3-0.6B/raw/main/config.json)|[128](https://huggingface.co/Qwen/Qwen3-0.6B/raw/main/config.json)|[151936](https://huggingface.co/Qwen/Qwen3-0.6B/raw/main/config.json)|[40960](https://huggingface.co/Qwen/Qwen3-0.6B/raw/main/config.json)|[2025-04-27](https://huggingface.co/api/models/Qwen/Qwen3-0.6B)|[card](https://huggingface.co/Qwen/Qwen3-0.6B) · [config](https://huggingface.co/Qwen/Qwen3-0.6B/raw/main/config.json)|
-|[Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B)|Qwen 3|[4B](https://huggingface.co/api/models/Qwen/Qwen3-4B)|[36](https://huggingface.co/Qwen/Qwen3-4B/raw/main/config.json)|[2560](https://huggingface.co/Qwen/Qwen3-4B/raw/main/config.json)|[32](https://huggingface.co/Qwen/Qwen3-4B/raw/main/config.json) / [8](https://huggingface.co/Qwen/Qwen3-4B/raw/main/config.json)|[128](https://huggingface.co/Qwen/Qwen3-4B/raw/main/config.json)|[151936](https://huggingface.co/Qwen/Qwen3-4B/raw/main/config.json)|[40960](https://huggingface.co/Qwen/Qwen3-4B/raw/main/config.json)|[2025-04-27](https://huggingface.co/api/models/Qwen/Qwen3-4B)|[card](https://huggingface.co/Qwen/Qwen3-4B) · [config](https://huggingface.co/Qwen/Qwen3-4B/raw/main/config.json)|
-|[Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)|Qwen 3|[8B](https://huggingface.co/api/models/Qwen/Qwen3-8B)|[36](https://huggingface.co/Qwen/Qwen3-8B/raw/main/config.json)|[4096](https://huggingface.co/Qwen/Qwen3-8B/raw/main/config.json)|[32](https://huggingface.co/Qwen/Qwen3-8B/raw/main/config.json) / [8](https://huggingface.co/Qwen/Qwen3-8B/raw/main/config.json)|[128](https://huggingface.co/Qwen/Qwen3-8B/raw/main/config.json)|[151936](https://huggingface.co/Qwen/Qwen3-8B/raw/main/config.json)|[40960](https://huggingface.co/Qwen/Qwen3-8B/raw/main/config.json)|[2025-04-27](https://huggingface.co/api/models/Qwen/Qwen3-8B)|[card](https://huggingface.co/Qwen/Qwen3-8B) · [config](https://huggingface.co/Qwen/Qwen3-8B/raw/main/config.json)|
-|[Qwen/Qwen3-14B](https://huggingface.co/Qwen/Qwen3-14B)|Qwen 3|[14B](https://huggingface.co/api/models/Qwen/Qwen3-14B)|[40](https://huggingface.co/Qwen/Qwen3-14B/raw/main/config.json)|[5120](https://huggingface.co/Qwen/Qwen3-14B/raw/main/config.json)|[40](https://huggingface.co/Qwen/Qwen3-14B/raw/main/config.json) / [8](https://huggingface.co/Qwen/Qwen3-14B/raw/main/config.json)|[128](https://huggingface.co/Qwen/Qwen3-14B/raw/main/config.json)|[151936](https://huggingface.co/Qwen/Qwen3-14B/raw/main/config.json)|[40960](https://huggingface.co/Qwen/Qwen3-14B/raw/main/config.json)|[2025-04-27](https://huggingface.co/api/models/Qwen/Qwen3-14B)|[card](https://huggingface.co/Qwen/Qwen3-14B) · [config](https://huggingface.co/Qwen/Qwen3-14B/raw/main/config.json)|
-|[Qwen/Qwen3-32B](https://huggingface.co/Qwen/Qwen3-32B)|Qwen 3|[32B](https://huggingface.co/api/models/Qwen/Qwen3-32B)|[64](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json)|[5120](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json)|[64](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json) / [8](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json)|[128](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json)|[151936](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json)|[40960](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json)|[2025-04-27](https://huggingface.co/api/models/Qwen/Qwen3-32B)|[card](https://huggingface.co/Qwen/Qwen3-32B) · [config](https://huggingface.co/Qwen/Qwen3-32B/raw/main/config.json)|
-|[Qwen/Qwen3-30B-A3B](https://huggingface.co/Qwen/Qwen3-30B-A3B)|Qwen 3|[30B](https://huggingface.co/api/models/Qwen/Qwen3-30B-A3B)|[48](https://huggingface.co/Qwen/Qwen3-30B-A3B/raw/main/config.json)|[2048](https://huggingface.co/Qwen/Qwen3-30B-A3B/raw/main/config.json)|[32](https://huggingface.co/Qwen/Qwen3-30B-A3B/raw/main/config.json) / [4](https://huggingface.co/Qwen/Qwen3-30B-A3B/raw/main/config.json)|[128](https://huggingface.co/Qwen/Qwen3-30B-A3B/raw/main/config.json)|[151936](https://huggingface.co/Qwen/Qwen3-30B-A3B/raw/main/config.json)|[40960](https://huggingface.co/Qwen/Qwen3-30B-A3B/raw/main/config.json)|[2025-04-27](https://huggingface.co/api/models/Qwen/Qwen3-30B-A3B)|[card](https://huggingface.co/Qwen/Qwen3-30B-A3B) · [config](https://huggingface.co/Qwen/Qwen3-30B-A3B/raw/main/config.json)|
-|[Qwen/Qwen3-235B-A22B](https://huggingface.co/Qwen/Qwen3-235B-A22B)|Qwen 3|[235B](https://huggingface.co/api/models/Qwen/Qwen3-235B-A22B)|[94](https://huggingface.co/Qwen/Qwen3-235B-A22B/raw/main/config.json)|[4096](https://huggingface.co/Qwen/Qwen3-235B-A22B/raw/main/config.json)|[64](https://huggingface.co/Qwen/Qwen3-235B-A22B/raw/main/config.json) / [4](https://huggingface.co/Qwen/Qwen3-235B-A22B/raw/main/config.json)|[128](https://huggingface.co/Qwen/Qwen3-235B-A22B/raw/main/config.json)|[151936](https://huggingface.co/Qwen/Qwen3-235B-A22B/raw/main/config.json)|[40960](https://huggingface.co/Qwen/Qwen3-235B-A22B/raw/main/config.json)|[2025-04-27](https://huggingface.co/api/models/Qwen/Qwen3-235B-A22B)|[card](https://huggingface.co/Qwen/Qwen3-235B-A22B) · [config](https://huggingface.co/Qwen/Qwen3-235B-A22B/raw/main/config.json)|
-|[Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct)|Qwen 2.5-VL|[7B](https://huggingface.co/api/models/Qwen/Qwen2.5-VL-7B-Instruct)|[28](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|[3584](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|[28](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json) / [4](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|n/a|[152064](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|[128000](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|[2025-01-26](https://huggingface.co/api/models/Qwen/Qwen2.5-VL-7B-Instruct)|[card](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) · [config](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|
-|[Qwen/Qwen2.5-VL-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct)|Qwen 2.5-VL|[32B](https://huggingface.co/api/models/Qwen/Qwen2.5-VL-32B-Instruct)|[64](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|[5120](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|[40](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json) / [8](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|n/a|[152064](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|[128000](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|[2025-03-21](https://huggingface.co/api/models/Qwen/Qwen2.5-VL-32B-Instruct)|[card](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct) · [config](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|
-|[Qwen/Qwen2.5-VL-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct)|Qwen 2.5-VL|[72B](https://huggingface.co/api/models/Qwen/Qwen2.5-VL-72B-Instruct)|[80](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|[8192](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|[64](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json) / [8](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|n/a|[152064](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|[128000](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|[2025-01-27](https://huggingface.co/api/models/Qwen/Qwen2.5-VL-72B-Instruct)|[card](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct) · [config](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|
-|[Qwen/Qwen2.5-MoE-A2.7B](https://huggingface.co/Qwen/Qwen2.5-MoE-A2.7B)|Other|[n/a](https://huggingface.co/api/models/Qwen/Qwen2.5-MoE-A2.7B)|n/a|n/a|n/a / n/a|n/a|n/a|n/a|n/a|[card](https://huggingface.co/Qwen/Qwen2.5-MoE-A2.7B) · [config](https://huggingface.co/Qwen/Qwen2.5-MoE-A2.7B/raw/main/config.json)|
-|[Qwen/Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct)|Qwen 2.5 Coder|[7B](https://huggingface.co/api/models/Qwen/Qwen2.5-Coder-7B-Instruct)|[28](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/raw/main/config.json)|[3584](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/raw/main/config.json)|[28](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/raw/main/config.json) / [4](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/raw/main/config.json)|n/a|[152064](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/raw/main/config.json)|[32768](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/raw/main/config.json)|[2024-09-17](https://huggingface.co/api/models/Qwen/Qwen2.5-Coder-7B-Instruct)|[card](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct) · [config](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct/raw/main/config.json)|
-|[Qwen/Qwen2.5-Coder-14B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct)|Qwen 2.5 Coder|[14B](https://huggingface.co/api/models/Qwen/Qwen2.5-Coder-14B-Instruct)|[48](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct/raw/main/config.json)|[5120](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct/raw/main/config.json)|[40](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct/raw/main/config.json) / [8](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct/raw/main/config.json)|n/a|[152064](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct/raw/main/config.json)|[32768](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct/raw/main/config.json)|[2024-11-06](https://huggingface.co/api/models/Qwen/Qwen2.5-Coder-14B-Instruct)|[card](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct) · [config](https://huggingface.co/Qwen/Qwen2.5-Coder-14B-Instruct/raw/main/config.json)|
-|[Qwen/Qwen2.5-Coder-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct)|Qwen 2.5 Coder|[32B](https://huggingface.co/api/models/Qwen/Qwen2.5-Coder-32B-Instruct)|[64](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct/raw/main/config.json)|[5120](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct/raw/main/config.json)|[40](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct/raw/main/config.json) / [8](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct/raw/main/config.json)|n/a|[152064](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct/raw/main/config.json)|[32768](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct/raw/main/config.json)|[2024-11-06](https://huggingface.co/api/models/Qwen/Qwen2.5-Coder-32B-Instruct)|[card](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct) · [config](https://huggingface.co/Qwen/Qwen2.5-Coder-32B-Instruct/raw/main/config.json)|
-|[deepseek-ai/DeepSeek-V3](https://huggingface.co/deepseek-ai/DeepSeek-V3)|DeepSeek|[671B](https://huggingface.co/api/models/deepseek-ai/DeepSeek-V3)|[61](https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/config.json)|[7168](https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/config.json)|[128](https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/config.json) / [128](https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/config.json)|n/a|[129280](https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/config.json)|[163840](https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/config.json)|[2024-12-25](https://huggingface.co/api/models/deepseek-ai/DeepSeek-V3)|[card](https://huggingface.co/deepseek-ai/DeepSeek-V3) · [config](https://huggingface.co/deepseek-ai/DeepSeek-V3/raw/main/config.json)|
-|[deepseek-ai/DeepSeek-V3-0324](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324)|DeepSeek|[671B](https://huggingface.co/api/models/deepseek-ai/DeepSeek-V3-0324)|[61](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324/raw/main/config.json)|[7168](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324/raw/main/config.json)|[128](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324/raw/main/config.json) / [128](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324/raw/main/config.json)|n/a|[129280](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324/raw/main/config.json)|[163840](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324/raw/main/config.json)|[2025-03-24](https://huggingface.co/api/models/deepseek-ai/DeepSeek-V3-0324)|[card](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324) · [config](https://huggingface.co/deepseek-ai/DeepSeek-V3-0324/raw/main/config.json)|
-|[deepseek-ai/DeepSeek-R1](https://huggingface.co/deepseek-ai/DeepSeek-R1)|DeepSeek|[671B](https://huggingface.co/api/models/deepseek-ai/DeepSeek-R1)|[61](https://huggingface.co/deepseek-ai/DeepSeek-R1/raw/main/config.json)|[7168](https://huggingface.co/deepseek-ai/DeepSeek-R1/raw/main/config.json)|[128](https://huggingface.co/deepseek-ai/DeepSeek-R1/raw/main/config.json) / [128](https://huggingface.co/deepseek-ai/DeepSeek-R1/raw/main/config.json)|n/a|[129280](https://huggingface.co/deepseek-ai/DeepSeek-R1/raw/main/config.json)|[163840](https://huggingface.co/deepseek-ai/DeepSeek-R1/raw/main/config.json)|[2025-01-20](https://huggingface.co/api/models/deepseek-ai/DeepSeek-R1)|[card](https://huggingface.co/deepseek-ai/DeepSeek-R1) · [config](https://huggingface.co/deepseek-ai/DeepSeek-R1/raw/main/config.json)|
-|[deepseek-ai/DeepSeek-R1-0528](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528)|DeepSeek|[671B](https://huggingface.co/api/models/deepseek-ai/DeepSeek-R1-0528)|[61](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)|[7168](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)|[128](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json) / [128](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)|n/a|[129280](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)|[163840](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)|[2025-05-28](https://huggingface.co/api/models/deepseek-ai/DeepSeek-R1-0528)|[card](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528) · [config](https://huggingface.co/deepseek-ai/DeepSeek-R1-0528/raw/main/config.json)|
-|[deepseek-ai/DeepSeek-Coder-V2-Instruct](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct)|DeepSeek Coder V2|[236B](https://huggingface.co/api/models/deepseek-ai/DeepSeek-Coder-V2-Instruct)|[60](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct/raw/main/config.json)|[5120](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct/raw/main/config.json)|[128](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct/raw/main/config.json) / [128](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct/raw/main/config.json)|n/a|[102400](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct/raw/main/config.json)|[163840](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct/raw/main/config.json)|[2024-06-14](https://huggingface.co/api/models/deepseek-ai/DeepSeek-Coder-V2-Instruct)|[card](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct) · [config](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct/raw/main/config.json)|
-|istralai/Mistral-Large-Instruct-2411](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411)|Mistral|[122.6B](https://huggingface.co/api/models/mistralai/Mistral-Large-Instruct-2411)|[88](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411/raw/main/config.json)|[12288](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411/raw/main/config.json)|[96](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411/raw/main/config.json) / [8](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411/raw/main/config.json)|[128](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411/raw/main/config.json)|[32768](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411/raw/main/config.json)|[131072](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411/raw/main/config.json)|[2024-11-14](https://huggingface.co/api/models/mistralai/Mistral-Large-Instruct-2411)|[card](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411) · [config](https://huggingface.co/mistralai/Mistral-Large-Instruct-2411/raw/main/config.json)|
-|istralai/Codestral-22B-v0.1](https://huggingface.co/mistralai/Codestral-22B-v0.1)|Mistral|[22.2B](https://huggingface.co/api/models/mistralai/Codestral-22B-v0.1)|[56](https://huggingface.co/mistralai/Codestral-22B-v0.1/raw/main/config.json)|[6144](https://huggingface.co/mistralai/Codestral-22B-v0.1/raw/main/config.json)|[48](https://huggingface.co/mistralai/Codestral-22B-v0.1/raw/main/config.json) / [8](https://huggingface.co/mistralai/Codestral-22B-v0.1/raw/main/config.json)|n/a|[32768](https://huggingface.co/mistralai/Codestral-22B-v0.1/raw/main/config.json)|[32768](https://huggingface.co/mistralai/Codestral-22B-v0.1/raw/main/config.json)|[2024-05-29](https://huggingface.co/api/models/mistralai/Codestral-22B-v0.1)|[card](https://huggingface.co/mistralai/Codestral-22B-v0.1) · [config](https://huggingface.co/mistralai/Codestral-22B-v0.1/raw/main/config.json)|
-|istralai/Ministral-3B-Instruct-2410](https://huggingface.co/mistralai/Ministral-3B-Instruct-2410)|Mistral|[3B](https://huggingface.co/api/models/mistralai/Ministral-3B-Instruct-2410)|n/a|n/a|n/a / n/a|n/a|n/a|n/a|[2024-10; repo 401](https://huggingface.co/api/models/mistralai/Ministral-3B-Instruct-2410)|[card](https://huggingface.co/mistralai/Ministral-3B-Instruct-2410) · [config](https://huggingface.co/mistralai/Ministral-3B-Instruct-2410/raw/main/config.json)|
-|istralai/Ministral-8B-Instruct-2410](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410)|Mistral|[8.0B](https://huggingface.co/api/models/mistralai/Ministral-8B-Instruct-2410)|[36](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410/raw/main/config.json)|[4096](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410/raw/main/config.json)|[32](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410/raw/main/config.json) / [8](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410/raw/main/config.json)|[128](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410/raw/main/config.json)|[131072](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410/raw/main/config.json)|[32768](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410/raw/main/config.json)|[2024-10-15](https://huggingface.co/api/models/mistralai/Ministral-8B-Instruct-2410)|[card](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410) · [config](https://huggingface.co/mistralai/Ministral-8B-Instruct-2410/raw/main/config.json)|
-|istralai/Pixtral-12B-2409](https://huggingface.co/mistralai/Pixtral-12B-2409)|Pixtral|[12B](https://huggingface.co/api/models/mistralai/Pixtral-12B-2409)|n/a|n/a|n/a / n/a|n/a|n/a|[128K advertised](https://huggingface.co/mistralai/Pixtral-12B-2409/raw/main/config.json)|[2024-09](https://huggingface.co/api/models/mistralai/Pixtral-12B-2409)|[card](https://huggingface.co/mistralai/Pixtral-12B-2409) · [config](https://huggingface.co/mistralai/Pixtral-12B-2409/raw/main/config.json)|
-|[google/gemma-3-1b-it](https://huggingface.co/google/gemma-3-1b-it)|Gemma 3|[1B](https://huggingface.co/api/models/google/gemma-3-1b-it)|n/a|n/a|n/a / n/a|n/a|n/a|[32K](https://huggingface.co/google/gemma-3-1b-it/raw/main/config.json)|[2025-03](https://huggingface.co/api/models/google/gemma-3-1b-it)|[card](https://huggingface.co/google/gemma-3-1b-it) · [config](https://huggingface.co/google/gemma-3-1b-it/raw/main/config.json)|
-|[google/gemma-3-4b-it](https://huggingface.co/google/gemma-3-4b-it)|Gemma 3|[4B](https://huggingface.co/api/models/google/gemma-3-4b-it)|n/a|n/a|n/a / n/a|n/a|n/a|[128K](https://huggingface.co/google/gemma-3-4b-it/raw/main/config.json)|[2025-03](https://huggingface.co/api/models/google/gemma-3-4b-it)|[card](https://huggingface.co/google/gemma-3-4b-it) · [config](https://huggingface.co/google/gemma-3-4b-it/raw/main/config.json)|
-|[google/gemma-3-12b-it](https://huggingface.co/google/gemma-3-12b-it)|Gemma 3|[12B](https://huggingface.co/api/models/google/gemma-3-12b-it)|n/a|n/a|n/a / n/a|n/a|n/a|[128K](https://huggingface.co/google/gemma-3-12b-it/raw/main/config.json)|[2025-03](https://huggingface.co/api/models/google/gemma-3-12b-it)|[card](https://huggingface.co/google/gemma-3-12b-it) · [config](https://huggingface.co/google/gemma-3-12b-it/raw/main/config.json)|
-|[google/gemma-3-27b-it](https://huggingface.co/google/gemma-3-27b-it)|Gemma 3|[27B](https://huggingface.co/api/models/google/gemma-3-27b-it)|n/a|n/a|n/a / n/a|n/a|n/a|[128K](https://huggingface.co/google/gemma-3-27b-it/raw/main/config.json)|[2025-03](https://huggingface.co/api/models/google/gemma-3-27b-it)|[card](https://huggingface.co/google/gemma-3-27b-it) · [config](https://huggingface.co/google/gemma-3-27b-it/raw/main/config.json)|
-|[google/gemma-2-2b-it](https://huggingface.co/google/gemma-2-2b-it)|Gemma 2|[2B](https://huggingface.co/api/models/google/gemma-2-2b-it)|n/a|n/a|n/a / n/a|n/a|n/a|[8K](https://huggingface.co/google/gemma-2-2b-it/raw/main/config.json)|[2024-07](https://huggingface.co/api/models/google/gemma-2-2b-it)|[card](https://huggingface.co/google/gemma-2-2b-it) · [config](https://huggingface.co/google/gemma-2-2b-it/raw/main/config.json)|
-|[google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it)|Gemma 2|[9B](https://huggingface.co/api/models/google/gemma-2-9b-it)|n/a|n/a|n/a / n/a|n/a|n/a|[8K](https://huggingface.co/google/gemma-2-9b-it/raw/main/config.json)|[2024-06](https://huggingface.co/api/models/google/gemma-2-9b-it)|[card](https://huggingface.co/google/gemma-2-9b-it) · [config](https://huggingface.co/google/gemma-2-9b-it/raw/main/config.json)|
-|[google/gemma-2-27b-it](https://huggingface.co/google/gemma-2-27b-it)|Gemma 2|[27B](https://huggingface.co/api/models/google/gemma-2-27b-it)|n/a|n/a|n/a / n/a|n/a|n/a|[8K](https://huggingface.co/google/gemma-2-27b-it/raw/main/config.json)|[2024-06](https://huggingface.co/api/models/google/gemma-2-27b-it)|[card](https://huggingface.co/google/gemma-2-27b-it) · [config](https://huggingface.co/google/gemma-2-27b-it/raw/main/config.json)|
-|icrosoft/phi-4](https://huggingface.co/microsoft/phi-4)|Phi|[14.7B](https://huggingface.co/api/models/microsoft/phi-4)|[40](https://huggingface.co/microsoft/phi-4/raw/main/config.json)|[5120](https://huggingface.co/microsoft/phi-4/raw/main/config.json)|[40](https://huggingface.co/microsoft/phi-4/raw/main/config.json) / [10](https://huggingface.co/microsoft/phi-4/raw/main/config.json)|n/a|[100352](https://huggingface.co/microsoft/phi-4/raw/main/config.json)|[16384](https://huggingface.co/microsoft/phi-4/raw/main/config.json)|[2024-12-11](https://huggingface.co/api/models/microsoft/phi-4)|[card](https://huggingface.co/microsoft/phi-4) · [config](https://huggingface.co/microsoft/phi-4/raw/main/config.json)|
-|icrosoft/Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct)|Phi|[3.8B](https://huggingface.co/api/models/microsoft/Phi-4-mini-instruct)|[32](https://huggingface.co/microsoft/Phi-4-mini-instruct/raw/main/config.json)|[3072](https://huggingface.co/microsoft/Phi-4-mini-instruct/raw/main/config.json)|[24](https://huggingface.co/microsoft/Phi-4-mini-instruct/raw/main/config.json) / [8](https://huggingface.co/microsoft/Phi-4-mini-instruct/raw/main/config.json)|n/a|[200064](https://huggingface.co/microsoft/Phi-4-mini-instruct/raw/main/config.json)|[131072](https://huggingface.co/microsoft/Phi-4-mini-instruct/raw/main/config.json)|[2025-02-19](https://huggingface.co/api/models/microsoft/Phi-4-mini-instruct)|[card](https://huggingface.co/microsoft/Phi-4-mini-instruct) · [config](https://huggingface.co/microsoft/Phi-4-mini-instruct/raw/main/config.json)|
-|icrosoft/Phi-3.5-mini-instruct](https://huggingface.co/microsoft/Phi-3.5-mini-instruct)|Phi|[3.8B](https://huggingface.co/api/models/microsoft/Phi-3.5-mini-instruct)|[32](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/raw/main/config.json)|[3072](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/raw/main/config.json)|[32](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/raw/main/config.json) / [32](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/raw/main/config.json)|n/a|[32064](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/raw/main/config.json)|[131072](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/raw/main/config.json)|[2024-08-16](https://huggingface.co/api/models/microsoft/Phi-3.5-mini-instruct)|[card](https://huggingface.co/microsoft/Phi-3.5-mini-instruct) · [config](https://huggingface.co/microsoft/Phi-3.5-mini-instruct/raw/main/config.json)|
-|icrosoft/Phi-3.5-MoE-instruct](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct)|Phi|[41.9B](https://huggingface.co/api/models/microsoft/Phi-3.5-MoE-instruct)|[32](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct/raw/main/config.json)|[4096](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct/raw/main/config.json)|[32](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct/raw/main/config.json) / [8](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct/raw/main/config.json)|n/a|[32064](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct/raw/main/config.json)|[131072](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct/raw/main/config.json)|[2024-08-17](https://huggingface.co/api/models/microsoft/Phi-3.5-MoE-instruct)|[card](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct) · [config](https://huggingface.co/microsoft/Phi-3.5-MoE-instruct/raw/main/config.json)|
-|[WizardLM/WizardLM-2-8x22B](https://huggingface.co/WizardLM/WizardLM-2-8x22B)|WizardLM|[141B](https://huggingface.co/api/models/WizardLM/WizardLM-2-8x22B)|n/a|n/a|n/a / n/a|n/a|n/a|n/a|[2024-04](https://huggingface.co/api/models/WizardLM/WizardLM-2-8x22B)|[card](https://huggingface.co/WizardLM/WizardLM-2-8x22B) · [config](https://huggingface.co/WizardLM/WizardLM-2-8x22B/raw/main/config.json)|
-|icrosoft/Orca-2-13b](https://huggingface.co/microsoft/Orca-2-13b)|Orca|[n/a](https://huggingface.co/api/models/microsoft/Orca-2-13b)|[40](https://huggingface.co/microsoft/Orca-2-13b/raw/main/config.json)|[5120](https://huggingface.co/microsoft/Orca-2-13b/raw/main/config.json)|[40](https://huggingface.co/microsoft/Orca-2-13b/raw/main/config.json) / [40](https://huggingface.co/microsoft/Orca-2-13b/raw/main/config.json)|n/a|[32003](https://huggingface.co/microsoft/Orca-2-13b/raw/main/config.json)|[4096](https://huggingface.co/microsoft/Orca-2-13b/raw/main/config.json)|[2023-11-14](https://huggingface.co/api/models/microsoft/Orca-2-13b)|[card](https://huggingface.co/microsoft/Orca-2-13b) · [config](https://huggingface.co/microsoft/Orca-2-13b/raw/main/config.json)|
-|icrosoft/BitNet-b1.58-2B-4T](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T)|BitNet|[850M](https://huggingface.co/api/models/microsoft/BitNet-b1.58-2B-4T)|[30](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T/raw/main/config.json)|[2560](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T/raw/main/config.json)|[20](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T/raw/main/config.json) / [5](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T/raw/main/config.json)|n/a|[128256](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T/raw/main/config.json)|[4096](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T/raw/main/config.json)|[2025-04-15](https://huggingface.co/api/models/microsoft/BitNet-b1.58-2B-4T)|[card](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T) · [config](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T/raw/main/config.json)|
-|[HuggingFaceTB/SmolLM2-135M](https://huggingface.co/HuggingFaceTB/SmolLM2-135M)|SmolLM2|[135M](https://huggingface.co/api/models/HuggingFaceTB/SmolLM2-135M)|[30](https://huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/config.json)|[576](https://huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/config.json)|[9](https://huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/config.json) / [3](https://huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/config.json)|n/a|[49152](https://huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/config.json)|[8192](https://huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/config.json)|[2024-10-31](https://huggingface.co/api/models/HuggingFaceTB/SmolLM2-135M)|[card](https://huggingface.co/HuggingFaceTB/SmolLM2-135M) · [config](https://huggingface.co/HuggingFaceTB/SmolLM2-135M/raw/main/config.json)|
-|[HuggingFaceTB/SmolLM2-360M](https://huggingface.co/HuggingFaceTB/SmolLM2-360M)|SmolLM2|[362M](https://huggingface.co/api/models/HuggingFaceTB/SmolLM2-360M)|[32](https://huggingface.co/HuggingFaceTB/SmolLM2-360M/raw/main/config.json)|[960](https://huggingface.co/HuggingFaceTB/SmolLM2-360M/raw/main/config.json)|[15](https://huggingface.co/HuggingFaceTB/SmolLM2-360M/raw/main/config.json) / [5](https://huggingface.co/HuggingFaceTB/SmolLM2-360M/raw/main/config.json)|n/a|[49152](https://huggingface.co/HuggingFaceTB/SmolLM2-360M/raw/main/config.json)|[8192](https://huggingface.co/HuggingFaceTB/SmolLM2-360M/raw/main/config.json)|[2024-10-31](https://huggingface.co/api/models/HuggingFaceTB/SmolLM2-360M)|[card](https://huggingface.co/HuggingFaceTB/SmolLM2-360M) · [config](https://huggingface.co/HuggingFaceTB/SmolLM2-360M/raw/main/config.json)|
-|[HuggingFaceTB/SmolLM2-1.7B](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B)|SmolLM2|[1.7B](https://huggingface.co/api/models/HuggingFaceTB/SmolLM2-1.7B)|[24](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B/raw/main/config.json)|[2048](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B/raw/main/config.json)|[32](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B/raw/main/config.json) / [32](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B/raw/main/config.json)|n/a|[49152](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B/raw/main/config.json)|[8192](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B/raw/main/config.json)|[2024-10-30](https://huggingface.co/api/models/HuggingFaceTB/SmolLM2-1.7B)|[card](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B) · [config](https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B/raw/main/config.json)|
-|[TinyLlama/TinyLlama-1.1B-Chat-v1.0](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0)|TinyLlama|[1.1B](https://huggingface.co/api/models/TinyLlama/TinyLlama-1.1B-Chat-v1.0)|[22](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0/raw/main/config.json)|[2048](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0/raw/main/config.json)|[32](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0/raw/main/config.json) / [4](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0/raw/main/config.json)|n/a|[32000](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0/raw/main/config.json)|[2048](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0/raw/main/config.json)|[2023-12-30](https://huggingface.co/api/models/TinyLlama/TinyLlama-1.1B-Chat-v1.0)|[card](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0) · [config](https://huggingface.co/TinyLlama/TinyLlama-1.1B-Chat-v1.0/raw/main/config.json)|
-|[allenai/OLMo-2-1124-7B-Instruct](https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct)|OLMo|[7.3B](https://huggingface.co/api/models/allenai/OLMo-2-1124-7B-Instruct)|[32](https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct/raw/main/config.json)|[4096](https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct/raw/main/config.json)|[32](https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct/raw/main/config.json) / [32](https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct/raw/main/config.json)|n/a|[100352](https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct/raw/main/config.json)|[4096](https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct/raw/main/config.json)|[2024-12-18](https://huggingface.co/api/models/allenai/OLMo-2-1124-7B-Instruct)|[card](https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct) · [config](https://huggingface.co/allenai/OLMo-2-1124-7B-Instruct/raw/main/config.json)|
-|[tiiuae/Falcon3-7B-Instruct](https://huggingface.co/tiiuae/Falcon3-7B-Instruct)|Falcon 3|[7.5B](https://huggingface.co/api/models/tiiuae/Falcon3-7B-Instruct)|[28](https://huggingface.co/tiiuae/Falcon3-7B-Instruct/raw/main/config.json)|[3072](https://huggingface.co/tiiuae/Falcon3-7B-Instruct/raw/main/config.json)|[12](https://huggingface.co/tiiuae/Falcon3-7B-Instruct/raw/main/config.json) / [4](https://huggingface.co/tiiuae/Falcon3-7B-Instruct/raw/main/config.json)|[256](https://huggingface.co/tiiuae/Falcon3-7B-Instruct/raw/main/config.json)|[131072](https://huggingface.co/tiiuae/Falcon3-7B-Instruct/raw/main/config.json)|[32768](https://huggingface.co/tiiuae/Falcon3-7B-Instruct/raw/main/config.json)|[2024-11-29](https://huggingface.co/api/models/tiiuae/Falcon3-7B-Instruct)|[card](https://huggingface.co/tiiuae/Falcon3-7B-Instruct) · [config](https://huggingface.co/tiiuae/Falcon3-7B-Instruct/raw/main/config.json)|
-|[01-ai/Yi-1.5-9B-Chat](https://huggingface.co/01-ai/Yi-1.5-9B-Chat)|Yi 1.5|[8.8B](https://huggingface.co/api/models/01-ai/Yi-1.5-9B-Chat)|[48](https://huggingface.co/01-ai/Yi-1.5-9B-Chat/raw/main/config.json)|[4096](https://huggingface.co/01-ai/Yi-1.5-9B-Chat/raw/main/config.json)|[32](https://huggingface.co/01-ai/Yi-1.5-9B-Chat/raw/main/config.json) / [4](https://huggingface.co/01-ai/Yi-1.5-9B-Chat/raw/main/config.json)|n/a|[64000](https://huggingface.co/01-ai/Yi-1.5-9B-Chat/raw/main/config.json)|[4096](https://huggingface.co/01-ai/Yi-1.5-9B-Chat/raw/main/config.json)|[2024-05-10](https://huggingface.co/api/models/01-ai/Yi-1.5-9B-Chat)|[card](https://huggingface.co/01-ai/Yi-1.5-9B-Chat) · [config](https://huggingface.co/01-ai/Yi-1.5-9B-Chat/raw/main/config.json)|
-|[internlm/internlm2_5-7b-chat](https://huggingface.co/internlm/internlm2_5-7b-chat)|InternLM 2.5|[7.7B](https://huggingface.co/api/models/internlm/internlm2_5-7b-chat)|[32](https://huggingface.co/internlm/internlm2_5-7b-chat/raw/main/config.json)|[4096](https://huggingface.co/internlm/internlm2_5-7b-chat/raw/main/config.json)|[32](https://huggingface.co/internlm/internlm2_5-7b-chat/raw/main/config.json) / [8](https://huggingface.co/internlm/internlm2_5-7b-chat/raw/main/config.json)|n/a|[92544](https://huggingface.co/internlm/internlm2_5-7b-chat/raw/main/config.json)|[32768](https://huggingface.co/internlm/internlm2_5-7b-chat/raw/main/config.json)|[2024-06-27](https://huggingface.co/api/models/internlm/internlm2_5-7b-chat)|[card](https://huggingface.co/internlm/internlm2_5-7b-chat) · [config](https://huggingface.co/internlm/internlm2_5-7b-chat/raw/main/config.json)|
-|[deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct)|DeepSeek Coder V2|[16B](https://huggingface.co/api/models/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct)|[27](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct/raw/main/config.json)|[2048](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct/raw/main/config.json)|[16](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct/raw/main/config.json) / [16](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct/raw/main/config.json)|n/a|[102400](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct/raw/main/config.json)|[163840](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct/raw/main/config.json)|[2024-06-14](https://huggingface.co/api/models/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct)|[card](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct) · [config](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Lite-Instruct/raw/main/config.json)|
-|[codellama/CodeLlama-7b-hf](https://huggingface.co/codellama/CodeLlama-7b-hf)|CodeLlama|[6.7B](https://huggingface.co/api/models/codellama/CodeLlama-7b-hf)|[32](https://huggingface.co/codellama/CodeLlama-7b-hf/raw/main/config.json)|[4096](https://huggingface.co/codellama/CodeLlama-7b-hf/raw/main/config.json)|[32](https://huggingface.co/codellama/CodeLlama-7b-hf/raw/main/config.json) / [32](https://huggingface.co/codellama/CodeLlama-7b-hf/raw/main/config.json)|n/a|[32016](https://huggingface.co/codellama/CodeLlama-7b-hf/raw/main/config.json)|[16384](https://huggingface.co/codellama/CodeLlama-7b-hf/raw/main/config.json)|[2023-08-24](https://huggingface.co/api/models/codellama/CodeLlama-7b-hf)|[card](https://huggingface.co/codellama/CodeLlama-7b-hf) · [config](https://huggingface.co/codellama/CodeLlama-7b-hf/raw/main/config.json)|
-|[codellama/CodeLlama-34b-Instruct-hf](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf)|CodeLlama|[33.7B](https://huggingface.co/api/models/codellama/CodeLlama-34b-Instruct-hf)|[48](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf/raw/main/config.json)|[8192](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf/raw/main/config.json)|[64](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf/raw/main/config.json) / [8](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf/raw/main/config.json)|n/a|[32000](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf/raw/main/config.json)|[16384](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf/raw/main/config.json)|[2023-08-24](https://huggingface.co/api/models/codellama/CodeLlama-34b-Instruct-hf)|[card](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf) · [config](https://huggingface.co/codellama/CodeLlama-34b-Instruct-hf/raw/main/config.json)|
-|[Qwen/QwQ-32B](https://huggingface.co/Qwen/QwQ-32B)|QwQ|[32.8B](https://huggingface.co/api/models/Qwen/QwQ-32B)|[64](https://huggingface.co/Qwen/QwQ-32B/raw/main/config.json)|[5120](https://huggingface.co/Qwen/QwQ-32B/raw/main/config.json)|[40](https://huggingface.co/Qwen/QwQ-32B/raw/main/config.json) / [8](https://huggingface.co/Qwen/QwQ-32B/raw/main/config.json)|n/a|[152064](https://huggingface.co/Qwen/QwQ-32B/raw/main/config.json)|[40960](https://huggingface.co/Qwen/QwQ-32B/raw/main/config.json)|[2025-03-05](https://huggingface.co/api/models/Qwen/QwQ-32B)|[card](https://huggingface.co/Qwen/QwQ-32B) · [config](https://huggingface.co/Qwen/QwQ-32B/raw/main/config.json)|
-
-## Embeddings
-
-|Model|Family|Params|Layers|Hidden|Heads|Vocab|Max sequence|Source|
-|---|---|---:|---:|---:|---:|---:|---:|---|
-|[BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3)|Embeddings|[n/a](https://huggingface.co/api/models/BAAI/bge-m3)|[24](https://huggingface.co/BAAI/bge-m3/raw/main/config.json)|[1024](https://huggingface.co/BAAI/bge-m3/raw/main/config.json)|[16](https://huggingface.co/BAAI/bge-m3/raw/main/config.json)|[250002](https://huggingface.co/BAAI/bge-m3/raw/main/config.json)|[8194](https://huggingface.co/BAAI/bge-m3/raw/main/config.json)|[card](https://huggingface.co/BAAI/bge-m3) · [config](https://huggingface.co/BAAI/bge-m3/raw/main/config.json)|
-|[intfloat/e5-large-v2](https://huggingface.co/intfloat/e5-large-v2)|Embeddings|[335M](https://huggingface.co/api/models/intfloat/e5-large-v2)|[24](https://huggingface.co/intfloat/e5-large-v2/raw/main/config.json)|[1024](https://huggingface.co/intfloat/e5-large-v2/raw/main/config.json)|[16](https://huggingface.co/intfloat/e5-large-v2/raw/main/config.json)|[30522](https://huggingface.co/intfloat/e5-large-v2/raw/main/config.json)|[512](https://huggingface.co/intfloat/e5-large-v2/raw/main/config.json)|[card](https://huggingface.co/intfloat/e5-large-v2) · [config](https://huggingface.co/intfloat/e5-large-v2/raw/main/config.json)|
-|[thenlper/gte-large](https://huggingface.co/thenlper/gte-large)|Embeddings|[335M](https://huggingface.co/api/models/thenlper/gte-large)|[24](https://huggingface.co/thenlper/gte-large/raw/main/config.json)|[1024](https://huggingface.co/thenlper/gte-large/raw/main/config.json)|[16](https://huggingface.co/thenlper/gte-large/raw/main/config.json)|[30522](https://huggingface.co/thenlper/gte-large/raw/main/config.json)|[512](https://huggingface.co/thenlper/gte-large/raw/main/config.json)|[card](https://huggingface.co/thenlper/gte-large) · [config](https://huggingface.co/thenlper/gte-large/raw/main/config.json)|
-|[nomic-ai/nomic-embed-text-v1.5](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5)|Embeddings|[137M](https://huggingface.co/api/models/nomic-ai/nomic-embed-text-v1.5)|[12](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5/raw/main/config.json)|[768](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5/raw/main/config.json)|[12](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5/raw/main/config.json)|[30528](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5/raw/main/config.json)|[2048](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5/raw/main/config.json)|[card](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5) · [config](https://huggingface.co/nomic-ai/nomic-embed-text-v1.5/raw/main/config.json)|
-|[jinaai/jina-embeddings-v3](https://huggingface.co/jinaai/jina-embeddings-v3)|Embeddings|[572M](https://huggingface.co/api/models/jinaai/jina-embeddings-v3)|[24](https://huggingface.co/jinaai/jina-embeddings-v3/raw/main/config.json)|[1024](https://huggingface.co/jinaai/jina-embeddings-v3/raw/main/config.json)|[16](https://huggingface.co/jinaai/jina-embeddings-v3/raw/main/config.json)|[250002](https://huggingface.co/jinaai/jina-embeddings-v3/raw/main/config.json)|[8194](https://huggingface.co/jinaai/jina-embeddings-v3/raw/main/config.json)|[card](https://huggingface.co/jinaai/jina-embeddings-v3) · [config](https://huggingface.co/jinaai/jina-embeddings-v3/raw/main/config.json)|
-
-## Vision-language
-
-|Model|Family|Total params|Architecture summary|Vocab|Context|Source|
-|---|---|---:|---|---:|---:|---|
-|istralai/Pixtral-12B-2409](https://huggingface.co/mistralai/Pixtral-12B-2409)|Pixtral|[12B](https://huggingface.co/api/models/mistralai/Pixtral-12B-2409)|[n/a / n/a / n/a](https://huggingface.co/mistralai/Pixtral-12B-2409/raw/main/config.json)|n/a|n/a|[card](https://huggingface.co/mistralai/Pixtral-12B-2409) · [config](https://huggingface.co/mistralai/Pixtral-12B-2409/raw/main/config.json)|
-|[Qwen/Qwen2.5-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct)|Qwen 2.5-VL|[7B](https://huggingface.co/api/models/Qwen/Qwen2.5-VL-7B-Instruct)|[28 / 3584 / 28 q](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|[152064](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|[128000](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|[card](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct) · [config](https://huggingface.co/Qwen/Qwen2.5-VL-7B-Instruct/raw/main/config.json)|
-|[Qwen/Qwen2.5-VL-32B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct)|Qwen 2.5-VL|[32B](https://huggingface.co/api/models/Qwen/Qwen2.5-VL-32B-Instruct)|[64 / 5120 / 40 q](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|[152064](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|[128000](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|[card](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct) · [config](https://huggingface.co/Qwen/Qwen2.5-VL-32B-Instruct/raw/main/config.json)|
-|[Qwen/Qwen2.5-VL-72B-Instruct](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct)|Qwen 2.5-VL|[72B](https://huggingface.co/api/models/Qwen/Qwen2.5-VL-72B-Instruct)|[80 / 8192 / 64 q](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|[152064](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|[128000](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|[card](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct) · [config](https://huggingface.co/Qwen/Qwen2.5-VL-72B-Instruct/raw/main/config.json)|
-|[OpenGVLab/InternVL2-8B](https://huggingface.co/OpenGVLab/InternVL2-8B)|Vision-language|[8B combined](https://huggingface.co/api/models/OpenGVLab/InternVL2-8B)|[32 LLM / 24 vision / 4096 LLM / 1024 vision / 32 LLM / 16 vision](https://huggingface.co/OpenGVLab/InternVL2-8B/raw/main/config.json)|n/a|[92553](https://huggingface.co/OpenGVLab/InternVL2-8B/raw/main/config.json)|[card](https://huggingface.co/OpenGVLab/InternVL2-8B) · [config](https://huggingface.co/OpenGVLab/InternVL2-8B/raw/main/config.json)|
-|[THUDM/cogvlm2-llama3-chat-19B](https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B)|Vision-language|[19B combined](https://huggingface.co/api/models/THUDM/cogvlm2-llama3-chat-19B)|[32 / 4096 / 32 / n/a](https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B/raw/main/config.json)|n/a|[128256](https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B/raw/main/config.json)|[card](https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B) · [config](https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B/raw/main/config.json)|
-
-## Audio ASR
-
-|Model|Params|Layers/hidden|Vocab|Source|
-|---|---:|---|---:|---|
-|[openai/whisper-large-v3-turbo](https://huggingface.co/openai/whisper-large-v3-turbo)|[809M](https://huggingface.co/api/models/openai/whisper-large-v3-turbo)|[n/a / 1280](https://huggingface.co/openai/whisper-large-v3-turbo/raw/main/config.json)|[51866](https://huggingface.co/openai/whisper-large-v3-turbo/raw/main/config.json)|[card](https://huggingface.co/openai/whisper-large-v3-turbo)|
-|[nvidia/parakeet-tdt-0.6b-v2](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2)|[0.6B](https://huggingface.co/api/models/nvidia/parakeet-tdt-0.6b-v2)|[n/a / n/a](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2/raw/main/config.json)|n/a|[card](https://huggingface.co/nvidia/parakeet-tdt-0.6b-v2)|
-|[distil-whisper/distil-large-v3](https://huggingface.co/distil-whisper/distil-large-v3)|[756M](https://huggingface.co/api/models/distil-whisper/distil-large-v3)|[32 / 1280](https://huggingface.co/distil-whisper/distil-large-v3/raw/main/config.json)|[51866](https://huggingface.co/distil-whisper/distil-large-v3/raw/main/config.json)|[card](https://huggingface.co/distil-whisper/distil-large-v3)|
-
-ASR throughput should be modeled as audio-seconds processed per wall-clock second (real-time factor), not text token/sec. Keep audio duration, sample rate, chunking, beam/decoder settings, and WER separate from decoder LLM token metrics. Parakeet’s card reports WER but no universal token/sec.
-
-## Release/status exceptions and verified facts
-
-- **Llama 4 Scout/Maverick:** Meta describes both as MoE with 17B active; HF release material reports approximately 109B total for Scout and 400B for Maverick. Behemoth is announced at nearly 2T total and 288B active, but the official HF path was inaccessible/weights unavailable in this check. [Meta announcement](https://ai.meta.com/blog/llama-4-multimodal-intelligence) · [HF release](https://huggingface.co/blog/llama4-release)
-- **DeepSeek:** V3 and R1 are 671B total / 37B active per the HF card; V3 card says 128K context, while its config exposes 163840—store both fields and show a conflict warning. Coder V2 full is 236B / 21B active and Lite is 16B / 2.4B active. [V3](https://huggingface.co/deepseek-ai/DeepSeek-V3) · [R1](https://huggingface.co/deepseek-ai/DeepSeek-R1) · [Coder V2](https://huggingface.co/deepseek-ai/DeepSeek-Coder-V2-Instruct)
-- **DeepSeek V3.5 / R2:** No verified official HF model card or primary release page under those names was found in the checked universe as of the cutoff. Keep them as `status=pending/unverified`, not as inference rows.
-- **Qwen MoE naming:** The public documented model is **Qwen1.5-MoE-A2.7B**, 14.3B total / 2.7B active; the requested “Qwen2.5-MoE-A2.7B” repo name was not found. [card](https://huggingface.co/Qwen/Qwen1.5-MoE-A2.7B)
-- **WizardLM-2 8x22B:** official repo access returned 401; an HF mirror identifies it as 141B MoE based on Mixtral-8x22B. Use 141B only with `source_confidence=mirror`; keep architecture null until official config access. irror](https://huggingface.co/alpindale/WizardLM-2-8x22B)
-- **Pixtral 12B:** Mistral marks Pixtral 12B as no longer maintained/replaced; retain as a historical compatibility row. [status](https://mistral.ai/news/pixtral-12b/)
-
-## Closed API models (pricing-only, detailed pricing deferred)
-
-Do not add local architecture/inference rows for GPT-4o/GPT-4o-mini/o1/o1-mini/o3-mini, Claude 3.5 Sonnet/Haiku/Claude 3 Opus, or Gemini 2.0 Flash/1.5 Pro/1.5 Flash. Add versioned pricing records with `provider`, `model_id`, `input_usd_per_million`, `output_usd_per_million`, cached-input price, effective date, and deprecation status. Pricing research is intentionally deferred to the parent API-pricing track.
-
-## Suggested schema and implementation priorities
-
-Store `total_params` separately from `active_params`: memory sizing uses total weights, while MoE per-token compute uses active experts plus shared/trunk compute. Store both `context_config` and `context_advertised`, plus `release_date_type`, `source_checked_at`, `status`, and `source_confidence`. Must-have first: Qwen3 dense/MoE, DeepSeek V3/R1, Llama 4 Scout/Maverick, Gemma 3, Phi-4, Qwen2.5 Coder, Codestral, BGE-M3/E5, and Qwen2.5-VL. Nice: OLMo-2, Falcon3, Yi1.5, InternLM2.5, Pixtral, InternVL2, CogVLM2, Whisper Turbo, Parakeet. Future/status-only: Behemoth, unverified V3.5/R2, requested Qwen2.5-MoE alias, and official WizardLM-2 until accessible.
-
-
----
-
-# tokcalc Section C/E evidence ledger (snapshot 2026-09-20)
-
-## Conventions
-Official peak FLOPS values are theoretical. NVIDIA/AMD pages frequently publish 2:4 sparsity; dense = one-half of sparse only when footnote says so. Rows explicitly state per-chip vs system. Prices are dynamic, region/tier dependent; capture date 2026-09-20 IST. N/P means not publicly listed in the source, not zero.
-
-## Section C — accelerators
-
-![AMD Instinct MI355X product image](https://www.amd.com/content/dam/amd/en/images/products/data-centers/3366850-instinct-accelerator-mi350x-prod-front.jpg)
-
-Official visual: [NVIDIA RTX PRO 6000 Blackwell family](https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000-family/).
-
-
-| Vendor/product (status; year) | Memory / bandwidth | Dense FP16/BF16 | Interconnect | Benchmark evidence | Official source |
-|---|---|---|---|---|---|
-| NVIDIA H200 SXM / NVL (shipping; 2024) | 141 GB HBM3e, 4.8 TB/s | SXM 1,979 TFLOPS each FP16/BF16; NVL 1,671 TFLOPS | NVLink differs by form factor; use SKU-specific datasheet | N/P on official spec page | [H200 GPU](https://www.nvidia.com/en-us/data-center/h200), [H200 datasheet](https://resources.nvidia.com/en-us-gpu-resources/hpc-datasheet-sc23) |
-| NVIDIA B200 SXM (shipping in HGX/DGX systems; 2025) | 180 GB HBM3e, ~7.7–8.0 TB/s (official DGX gives 1,440 GB/64 TB/s for 8 GPUs) | Current official page exposes FP4/FP8 system metrics but not a clean FP16/BF16 dense row; store N/P until SKU datasheet is verified. Do not use AI TOPS or FP8 as FP16. | 5th-gen NVLink; DGX B200 14.4 TB/s aggregate NVLink | N/P in official page | [NVIDIA B200/HGX table](https://www.nvidia.com/en-us/data-center/b200), [DGX B200](https://www.nvidia.com/en-us/data-center/dgx-b200/) |
-| NVIDIA B300 SXM (shipping/available in 2026 cloud listings; 2025 announcement) | 288 GB HBM3e, 8 TB/s | DGX B300 official page gives FP4 144 sparse/108 dense and FP8 72 sparse; it does not expose FP16/BF16 dense. Store N/P rather than infer from FP8. | 5th-gen NVLink; DGX B300 14.4 TB/s aggregate | N/P | [DGX B300](https://www.nvidia.com/en-us/data-center/dgx-b300), [B300/HGX table](https://www.nvidia.com/en-us/data-center/b200) |
-| NVIDIA GB200 NVL72 system (shipping announced systems; 2025) | System: 13.4 TB HBM3e GPU memory, 576 TB/s; one GB200 superchip: 372 GB HBM3e, 16 TB/s | 360 PFLOPS FP16/BF16 shown for NVL72 in sparse format => ~180 PFLOPS dense; superchip 10 PFLOPS shown, verify footnote before storing | NVLink 130 TB/s system; 3.6 TB/s/superchip | N/P official | [GB200 NVL72](https://www.nvidia.com/en-us/data-center/gb200-nvl72/) |
-| NVIDIA GB300 NVL72 (announced/availability status SKU-dependent; 2025/2026) | N/P in official source reviewed; do not borrow B300 or GB200 values | N/P | N/P | N/P | [DGX B300](https://www.nvidia.com/en-us/data-center/dgx-b300), [Together GPU pricing](https://www.together.ai/pricing) (GB300 row shown —, not a spec source) |
-| NVIDIA RTX PRO 6000 Blackwell Workstation/Server/Max-Q (shipping; 2025) | 96 GB GDDR7 ECC; server product lists 1,597 GB/s | FP16/BF16 dense N/P in official page | PCIe Gen5 x16 | N/P | [RTX PRO 6000 family](https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000-family/), [server specs](https://www.nvidia.com/en-us/data-center/rtx-pro-6000-blackwell-server-edition/) |
-| NVIDIA RTX 5000 Ada workstation (shipping; 2023) | 32 GB GDDR6 ECC; official page does not expose bandwidth in extracted text | N/P (page gives 65.3 FP32 and 1,044.4 Tensor performance, not dense FP16/BF16) | PCIe Gen4 x16 | N/P | [RTX 5000 Ada](https://www.nvidia.com/en-us/products/workstations/rtx-5000/), [datasheet](https://resources.nvidia.com/en-us-briefcase-for-datasheets/rtx-5000-datashe?ncid=no-ncid) |
-| NVIDIA RTX 4500 Ada workstation (shipping; 2023) | 24 GB GDDR6 ECC; bandwidth N/P official page | N/P (634.0 Tensor listed, not FP16/BF16 dense) | PCIe Gen4 x16 | N/P | [RTX 4500 Ada](https://www.nvidia.com/en-us/products/workstations/rtx-4500/), [datasheet](https://www.nvidia.com/content/dam/en-zz/Solutions/products/workstations/nvidia-rtx-4500-datasheet.pdf) |
-| NVIDIA RTX 5090 consumer (shipping; 2025) | 32 GB GDDR7, 1,792 GB/s | N/P official FP16/BF16 (AI TOPS is not interchangeable) | N/P | N/P | [RTX 5090](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/) |
-| NVIDIA RTX 5080 consumer (shipping; 2025) | 16 GB GDDR7, 960 GB/s | N/P official FP16/BF16 | N/P | N/P | [RTX 5080](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5080) |
-| NVIDIA RTX 4080 Super consumer (shipping; 2024) | 16 GB GDDR6X, 736 GB/s | N/P official FP16/BF16 | N/P | N/P | [NVIDIA GeForce 40 series](https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/) |
-| NVIDIA RTX 4070 Ti Super consumer (shipping; 2024) | 16 GB GDDR6X, 672 GB/s | N/P official FP16/BF16 | N/P | N/P | [NVIDIA GeForce 40 series](https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/) |
-| AMD Instinct MI300X (shipping; 2023/2024) | 192 GB HBM3, 5.325 TB/s | 1,307.4 TFLOPS FP16 and BF16 dense | 7 x 128 GB/s Infinity Fabric links; PCIe Gen5 x16 | N/P official | [MI300X product](https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html), [datasheet PDF](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/data-sheets/amd-instinct-mi300x-data-sheet.pdf) |
-| AMD Instinct MI325X (shipping; 2024) | 256 GB HBM3E, 6 TB/s | 1,307.4 TFLOPS FP16/BF16 dense (official page) | N/P in extracted product page | N/P | [MI325X](https://www.amd.com/en/products/accelerators/instinct/mi300/mi325x.html), [datasheet](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/product-briefs/instinct-mi325x-datasheet.pdf) |
-| AMD Instinct MI350X (shipping/launch 2025) | 288 GB HBM3E, 8 TB/s | Product page shows 4.6 PFLOPS FP16/BF16 with sparsity; dense ~2.3 PFLOPS (only if applying AMD sparse footnote; store both) | N/P product page; platform data required | N/P | [MI350X](https://www.amd.com/en/products/accelerators/instinct/mi350/mi350x.html), [AMD 2025 blog](https://www.amd.com/en/blogs/2025/amd-instinct-mi350-series-and-beyond-accelerating-the-future-of-ai-and-hpc.html) |
-| AMD Instinct MI355X (shipping/launch 2025) | 288 GB HBM3E, 8 TB/s | Official page shows 5.0 PFLOPS FP16/BF16 with sparsity; dense ~2.5 PFLOPS (derived half) | Platform: 8-GPU 2.3 TB total; 1,194.8 GB/s aggregate bidirectional peer-to-peer | N/P | [MI355X](https://www.amd.com/en/products/accelerators/instinct/mi350/mi355x.html), [platform](https://www.amd.com/en/products/accelerators/instinct/mi350/mi355x/platform.html) |
-| Intel Gaudi 2 (shipping; 2022) | 96 GB HBM2e, 2.45 TB/s; 48 MB SRAM | Official architecture page gives memory but not clean dense BF16/FP16 number in extracted text; keep N/P rather than infer | 24 x 100GbE RoCE RDMA NICs | Intel published comparative benchmarks; capture separately | [Gaudi architecture](https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Architecture.html), [Gaudi2 white paper](https://cdrdv2-public.intel.com/839363/Intel-Gaudi2-AI-Accelerators-whitepaper.pdf) |
-| Intel Gaudi 3 (shipping/PCIe available; 2024; expanded availability 2025) | 128 GB HBM2e, 3.7 TB/s; 96 MB SRAM | 1.678 PFLOPS BF16 matrix (official Intel brief) | 12 x 200GbE; PCIe Gen5 x16 128 GB/s bidirectional; 1,200 GB/s accelerator interconnect in Intel comparison | Intel performance/positioning PDF includes Llama 3 8B/70B/405B test matrix; numbers need SKU/config extraction | [Gaudi 3 architecture](https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Architecture.html), [PCIe brief](https://cdrdv2-public.intel.com/817488/Gaudi%203%20PCIe%20Product%20Brief_RB_1_V6.pdf), [availability](https://newsroom.intel.com/artificial-intelligence/intel-gaudi-3-expands-availability-drive-ai-innovation-scale) |
-| Google TPU v5e (shipping/GA; 2023) | 16 GB HBM, 800 GiB/s | 197 TFLOPS BF16/chip | 400 GB/s bidirectional ICI/chip; 256-chip pod; 50.63 PFLOPS/pod | N/P official LLM tok/s | [v5e docs](https://docs.cloud.google.com/tpu/docs/v5e) |
-| Google TPU v5p (shipping/GA; 2023) | 95 GiB HBM, 2,765 GB/s | 459 TFLOPS BF16/chip | 1,200 GB/s bidirectional ICI/chip; 8,960-chip pod | N/P official LLM tok/s | [v5p docs](https://docs.cloud.google.com/tpu/docs/v5p), [launch](https://cloud.google.com/blog/products/ai-machine-learning/introducing-cloud-tpu-v5p-and-ai-hypercomputer) |
-| Google TPU v6e Trillium (GA Dec 2024; shipping) | 32 GB HBM; ~1,638 GB/s (Google page gives doubled v5e bandwidth; v6e docs expose 1,638 GB/s) | 918 TFLOPS BF16/chip | 800 GB/s bidirectional ICI/chip; up to 256 chips/pod | Google says up to 4.7x peak compute/chip vs v5e | [v6e docs](https://docs.cloud.google.com/tpu/docs/v6e), [GA release](https://cloud.google.com/blog/products/compute/trillium-tpu-is-ga) |
-| Google TPU v7x Ironwood (GA/available in Google docs by 2026; 2025/2026) | 192 GiB HBM, 7,380 GB/s | 2,307 TFLOPS BF16/chip | 1,200 GB/s bidirectional ICI/chip; 9,216 chips/pod | Google TPU page says 42.5 exaflops/pod; system-level | [TPU7x docs](https://docs.cloud.google.com/tpu/docs/tpu7x), [TPU landing page](https://cloud.google.com/tpu) |
-| Groq LPU (shipping cloud; architecture generation/SKU varies) | Groq says on-chip SRAM bandwidth >80 TB/s; exact per-chip capacity/FP16 FLOPS not disclosed on reviewed page | N/P; do not substitute FP8 rack number | Platform page: 256 LPUs/rack, 40 PB/s SRAM bandwidth, 315 PFLOPS FP8/rack (system) | Artificial Analysis: Llama 3 70B 284 tok/s; Groq later docs show Llama 3.3 70B ~280 tok/s; speculative endpoint 1,665 tok/s (different model/config) | [Groq Llama 3 benchmark](https://groq.com/blog/12-hours-later-groq-is-running-llama-3-instruct-8-70b-by-meta-ai-on-its-lpu-inference-enginge), [LPU](https://groq.com/blog/the-groq-lpu-explained), [platform](https://groq.com/platform), odel docs](https://console.groq.com/docs/model/llama-3.3-70b-versatile) |
-| Cerebras WSE-3 / CS-3 (shipping; 2024) | CS-3 system: 44 GB on-chip SRAM, 21 PB/s memory bandwidth | WSE-3/CS-3: 125 PFLOPS peak AI performance is system/chip marketing metric, not directly dense FP16/BF16; mark N/P for exact dense | CS-3: 214 Pb/s interconnect bandwidth | Cerebras Inference: 1,800 tok/s Llama 3.1 8B and 450 tok/s Llama 3.1 70B; benchmark is service/system-level | [CS-3 system](https://www.cerebras.ai/system), [WSE-3 release](https://www.cerebras.ai/press-release/cerebras-announces-third-generation-wafer-scale-engine), [inference benchmark](https://www.cerebras.ai/blog/introducing-cerebras-inference-ai-at-instant-speed) |