@tokcalc/mcp-server 0.1.1 → 0.1.3

package/tool-results/bash_1789888203128_6cca13c71b47.txt

@@ -0,0 +1,199 @@
+![AMD Instinct MI355X product image](https://www.amd.com/content/dam/amd/en/images/products/data-centers/3366850-instinct-accelerator-mi350x-prod-front.jpg)
+
+Official visual: [NVIDIA RTX PRO 6000 Blackwell family](https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000-family/).
+
+
+| Vendor/product (status; year) | Memory / bandwidth | Dense FP16/BF16 | Interconnect | Benchmark evidence | Official source |
+|---|---|---|---|---|---|
+| NVIDIA H200 SXM / NVL (shipping; 2024) | 141 GB HBM3e, 4.8 TB/s | SXM 1,979 TFLOPS each FP16/BF16; NVL 1,671 TFLOPS | NVLink differs by form factor; use SKU-specific datasheet | N/P on official spec page | [H200 GPU](https://www.nvidia.com/en-us/data-center/h200), [H200 datasheet](https://resources.nvidia.com/en-us-gpu-resources/hpc-datasheet-sc23) |
+| NVIDIA B200 SXM (shipping in HGX/DGX systems; 2025) | 180 GB HBM3e, ~7.7–8.0 TB/s (official DGX gives 1,440 GB/64 TB/s for 8 GPUs) | Current official page exposes FP4/FP8 system metrics but not a clean FP16/BF16 dense row; store N/P until SKU datasheet is verified. Do not use AI TOPS or FP8 as FP16. | 5th-gen NVLink; DGX B200 14.4 TB/s aggregate NVLink | N/P in official page | [NVIDIA B200/HGX table](https://www.nvidia.com/en-us/data-center/b200), [DGX B200](https://www.nvidia.com/en-us/data-center/dgx-b200/) |
+| NVIDIA B300 SXM (shipping/available in 2026 cloud listings; 2025 announcement) | 288 GB HBM3e, 8 TB/s | DGX B300 official page gives FP4 144 sparse/108 dense and FP8 72 sparse; it does not expose FP16/BF16 dense. Store N/P rather than infer from FP8. | 5th-gen NVLink; DGX B300 14.4 TB/s aggregate | N/P | [DGX B300](https://www.nvidia.com/en-us/data-center/dgx-b300), [B300/HGX table](https://www.nvidia.com/en-us/data-center/b200) |
+| NVIDIA GB200 NVL72 system (shipping announced systems; 2025) | System: 13.4 TB HBM3e GPU memory, 576 TB/s; one GB200 superchip: 372 GB HBM3e, 16 TB/s | 360 PFLOPS FP16/BF16 shown for NVL72 in sparse format => ~180 PFLOPS dense; superchip 10 PFLOPS shown, verify footnote before storing | NVLink 130 TB/s system; 3.6 TB/s/superchip | N/P official | [GB200 NVL72](https://www.nvidia.com/en-us/data-center/gb200-nvl72/) |
+| NVIDIA GB300 NVL72 (announced/availability status SKU-dependent; 2025/2026) | N/P in official source reviewed; do not borrow B300 or GB200 values | N/P | N/P | N/P | [DGX B300](https://www.nvidia.com/en-us/data-center/dgx-b300), [Together GPU pricing](https://www.together.ai/pricing) (GB300 row shown —, not a spec source) |
+| NVIDIA RTX PRO 6000 Blackwell Workstation/Server/Max-Q (shipping; 2025) | 96 GB GDDR7 ECC; server product lists 1,597 GB/s | FP16/BF16 dense N/P in official page | PCIe Gen5 x16 | N/P | [RTX PRO 6000 family](https://www.nvidia.com/en-us/products/workstations/professional-desktop-gpus/rtx-pro-6000-family/), [server specs](https://www.nvidia.com/en-us/data-center/rtx-pro-6000-blackwell-server-edition/) |
+| NVIDIA RTX 5000 Ada workstation (shipping; 2023) | 32 GB GDDR6 ECC; official page does not expose bandwidth in extracted text | N/P (page gives 65.3 FP32 and 1,044.4 Tensor performance, not dense FP16/BF16) | PCIe Gen4 x16 | N/P | [RTX 5000 Ada](https://www.nvidia.com/en-us/products/workstations/rtx-5000/), [datasheet](https://resources.nvidia.com/en-us-briefcase-for-datasheets/rtx-5000-datashe?ncid=no-ncid) |
+| NVIDIA RTX 4500 Ada workstation (shipping; 2023) | 24 GB GDDR6 ECC; bandwidth N/P official page | N/P (634.0 Tensor listed, not FP16/BF16 dense) | PCIe Gen4 x16 | N/P | [RTX 4500 Ada](https://www.nvidia.com/en-us/products/workstations/rtx-4500/), [datasheet](https://www.nvidia.com/content/dam/en-zz/Solutions/products/workstations/nvidia-rtx-4500-datasheet.pdf) |
+| NVIDIA RTX 5090 consumer (shipping; 2025) | 32 GB GDDR7, 1,792 GB/s | N/P official FP16/BF16 (AI TOPS is not interchangeable) | N/P | N/P | [RTX 5090](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5090/) |
+| NVIDIA RTX 5080 consumer (shipping; 2025) | 16 GB GDDR7, 960 GB/s | N/P official FP16/BF16 | N/P | N/P | [RTX 5080](https://www.nvidia.com/en-us/geforce/graphics-cards/50-series/rtx-5080) |
+| NVIDIA RTX 4080 Super consumer (shipping; 2024) | 16 GB GDDR6X, 736 GB/s | N/P official FP16/BF16 | N/P | N/P | [NVIDIA GeForce 40 series](https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/) |
+| NVIDIA RTX 4070 Ti Super consumer (shipping; 2024) | 16 GB GDDR6X, 672 GB/s | N/P official FP16/BF16 | N/P | N/P | [NVIDIA GeForce 40 series](https://www.nvidia.com/en-us/geforce/graphics-cards/40-series/) |
+| AMD Instinct MI300X (shipping; 2023/2024) | 192 GB HBM3, 5.325 TB/s | 1,307.4 TFLOPS FP16 and BF16 dense | 7 x 128 GB/s Infinity Fabric links; PCIe Gen5 x16 | N/P official | [MI300X product](https://www.amd.com/en/products/accelerators/instinct/mi300/mi300x.html), [datasheet PDF](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/data-sheets/amd-instinct-mi300x-data-sheet.pdf) |
+| AMD Instinct MI325X (shipping; 2024) | 256 GB HBM3E, 6 TB/s | 1,307.4 TFLOPS FP16/BF16 dense (official page) | N/P in extracted product page | N/P | [MI325X](https://www.amd.com/en/products/accelerators/instinct/mi300/mi325x.html), [datasheet](https://www.amd.com/content/dam/amd/en/documents/instinct-tech-docs/product-briefs/instinct-mi325x-datasheet.pdf) |
+| AMD Instinct MI350X (shipping/launch 2025) | 288 GB HBM3E, 8 TB/s | Product page shows 4.6 PFLOPS FP16/BF16 with sparsity; dense ~2.3 PFLOPS (only if applying AMD sparse footnote; store both) | N/P product page; platform data required | N/P | [MI350X](https://www.amd.com/en/products/accelerators/instinct/mi350/mi350x.html), [AMD 2025 blog](https://www.amd.com/en/blogs/2025/amd-instinct-mi350-series-and-beyond-accelerating-the-future-of-ai-and-hpc.html) |
+| AMD Instinct MI355X (shipping/launch 2025) | 288 GB HBM3E, 8 TB/s | Official page shows 5.0 PFLOPS FP16/BF16 with sparsity; dense ~2.5 PFLOPS (derived half) | Platform: 8-GPU 2.3 TB total; 1,194.8 GB/s aggregate bidirectional peer-to-peer | N/P | [MI355X](https://www.amd.com/en/products/accelerators/instinct/mi350/mi355x.html), [platform](https://www.amd.com/en/products/accelerators/instinct/mi350/mi355x/platform.html) |
+| Intel Gaudi 2 (shipping; 2022) | 96 GB HBM2e, 2.45 TB/s; 48 MB SRAM | Official architecture page gives memory but not clean dense BF16/FP16 number in extracted text; keep N/P rather than infer | 24 x 100GbE RoCE RDMA NICs | Intel published comparative benchmarks; capture separately | [Gaudi architecture](https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Architecture.html), [Gaudi2 white paper](https://cdrdv2-public.intel.com/839363/Intel-Gaudi2-AI-Accelerators-whitepaper.pdf) |
+| Intel Gaudi 3 (shipping/PCIe available; 2024; expanded availability 2025) | 128 GB HBM2e, 3.7 TB/s; 96 MB SRAM | 1.678 PFLOPS BF16 matrix (official Intel brief) | 12 x 200GbE; PCIe Gen5 x16 128 GB/s bidirectional; 1,200 GB/s accelerator interconnect in Intel comparison | Intel performance/positioning PDF includes Llama 3 8B/70B/405B test matrix; numbers need SKU/config extraction | [Gaudi 3 architecture](https://docs.habana.ai/en/latest/Gaudi_Overview/Gaudi_Architecture.html), [PCIe brief](https://cdrdv2-public.intel.com/817488/Gaudi%203%20PCIe%20Product%20Brief_RB_1_V6.pdf), [availability](https://newsroom.intel.com/artificial-intelligence/intel-gaudi-3-expands-availability-drive-ai-innovation-scale) |
+| Google TPU v5e (shipping/GA; 2023) | 16 GB HBM, 800 GiB/s | 197 TFLOPS BF16/chip | 400 GB/s bidirectional ICI/chip; 256-chip pod; 50.63 PFLOPS/pod | N/P official LLM tok/s | [v5e docs](https://docs.cloud.google.com/tpu/docs/v5e) |
+| Google TPU v5p (shipping/GA; 2023) | 95 GiB HBM, 2,765 GB/s | 459 TFLOPS BF16/chip | 1,200 GB/s bidirectional ICI/chip; 8,960-chip pod | N/P official LLM tok/s | [v5p docs](https://docs.cloud.google.com/tpu/docs/v5p), [launch](https://cloud.google.com/blog/products/ai-machine-learning/introducing-cloud-tpu-v5p-and-ai-hypercomputer) |
+| Google TPU v6e Trillium (GA Dec 2024; shipping) | 32 GB HBM; ~1,638 GB/s (Google page gives doubled v5e bandwidth; v6e docs expose 1,638 GB/s) | 918 TFLOPS BF16/chip | 800 GB/s bidirectional ICI/chip; up to 256 chips/pod | Google says up to 4.7x peak compute/chip vs v5e | [v6e docs](https://docs.cloud.google.com/tpu/docs/v6e), [GA release](https://cloud.google.com/blog/products/compute/trillium-tpu-is-ga) |
+| Google TPU v7x Ironwood (GA/available in Google docs by 2026; 2025/2026) | 192 GiB HBM, 7,380 GB/s | 2,307 TFLOPS BF16/chip | 1,200 GB/s bidirectional ICI/chip; 9,216 chips/pod | Google TPU page says 42.5 exaflops/pod; system-level | [TPU7x docs](https://docs.cloud.google.com/tpu/docs/tpu7x), [TPU landing page](https://cloud.google.com/tpu) |
+| Groq LPU (shipping cloud; architecture generation/SKU varies) | Groq says on-chip SRAM bandwidth >80 TB/s; exact per-chip capacity/FP16 FLOPS not disclosed on reviewed page | N/P; do not substitute FP8 rack number | Platform page: 256 LPUs/rack, 40 PB/s SRAM bandwidth, 315 PFLOPS FP8/rack (system) | Artificial Analysis: Llama 3 70B 284 tok/s; Groq later docs show Llama 3.3 70B ~280 tok/s; speculative endpoint 1,665 tok/s (different model/config) | [Groq Llama 3 benchmark](https://groq.com/blog/12-hours-later-groq-is-running-llama-3-instruct-8-70b-by-meta-ai-on-its-lpu-inference-enginge), [LPU](https://groq.com/blog/the-groq-lpu-explained), [platform](https://groq.com/platform), odel docs](https://console.groq.com/docs/model/llama-3.3-70b-versatile) |
+| Cerebras WSE-3 / CS-3 (shipping; 2024) | CS-3 system: 44 GB on-chip SRAM, 21 PB/s memory bandwidth | WSE-3/CS-3: 125 PFLOPS peak AI performance is system/chip marketing metric, not directly dense FP16/BF16; mark N/P for exact dense | CS-3: 214 Pb/s interconnect bandwidth | Cerebras Inference: 1,800 tok/s Llama 3.1 8B and 450 tok/s Llama 3.1 70B; benchmark is service/system-level | [CS-3 system](https://www.cerebras.ai/system), [WSE-3 release](https://www.cerebras.ai/press-release/cerebras-announces-third-generation-wafer-scale-engine), [inference benchmark](https://www.cerebras.ai/blog/introducing-cerebras-inference-ai-at-instant-speed) |
+| SambaNova RDU/SN40L (shipping cloud; 2024 generation) | Exact capacity/bandwidth and clean dense FP16/BF16 not verified from an official accessible source in this run | N/P | N/P | No official Llama 3 throughput row verified; do not fabricate | [SambaNova platform](https://sambanova.ai/technology) (spec gap; revisit with browser/tab if needed) |
+| Apple M2 Ultra (shipping; 2023) | Unified memory 192 GB max; >800 GB/s bandwidth (Apple Mac Studio page) | Apple does not publish FP16/BF16 dense TFLOPS in reviewed sources | Unified memory, no NVLink; UltraFusion interconnect details not required for single chip | N/P official LLM tok/s | [M2 Ultra](https://www.apple.com/newsroom/2023/06/apple-unveils-m2-ultra/), [Mac Studio](https://www.apple.com/newsroom/2023/06/apple-unveils-new-mac-studio-and-mac-pro/) |
+| Apple M3 Ultra (shipping; 2025) | Unified memory 96–512 GB; >800 GB/s unified bandwidth (Mac Studio page) | Apple does not publish FP16/BF16 dense TFLOPS | Unified memory; no NVLink | N/P | [M3 Ultra](https://www.apple.com/newsroom/2025/03/apple-reveals-m3-ultra-taking-apple-silicon-to-a-new-extreme), [Mac Studio](https://www.apple.com/newsroom/2025/03/apple-unveils-new-mac-studio-the-most-powerful-mac-ever/) |
+| Apple M4 Ultra | No official Apple M4 Ultra shipping/announcement evidence verified in this run; mark unavailable rather than assume | N/P | N/P | N/P | Apple Newsroom search found M4 consumer products, not M4 Ultra; keep status unresolved |
+| NVIDIA Tesla V100 (still installed/cloud; launched 2017) | 16/32 GB HBM2, 900 GB/s PCIe or 1,134 GB/s V100S; NVLink 300 GB/s bidirectional | Tensor FP16 112 TFLOPS PCIe / 125 TFLOPS SXM2 (marketing tensor metric; document as tensor, not scalar dense) | PCIe 32 GB/s; NVLink 300 GB/s | NVIDIA app guide: frameworks up to 50x vs P100 (system/framework benchmark, not LLM) | [V100 datasheet](https://images.nvidia.com/content/technologies/volta/pdf/tesla-volta-v100-datasheet-letter-fnl-web.pdf), [V100 page](https://www.nvidia.com/en-gb/data-center/tesla-v100) |
+| NVIDIA Tesla P100 (legacy/hobbyist; launched 2016) | 12/16 GB HBM2, 549/732 GB/s | 18.7 TFLOPS PCIe or >21 TFLOPS native half (variant-dependent) | PCIe Gen3; no NVLink on PCIe card; SXM2 NVLink context differs | NVIDIA guide: frameworks up to 3x vs K80 | [P100 PCIe datasheet](https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/tesla-p100/pdf/nvidia-tesla-p100-PCIe-datasheet.pdf), [P100 page](https://www.nvidia.com/en-au/data-center/tesla-p100/) |
+| NVIDIA Tesla K80 (legacy/hobbyist; launched 2014) | 24 GB GDDR5 aggregate (12 GB/GPU), 480 GB/s aggregate | N/P official FP16; 8.73 TFLOPS FP32 aggregate/board | PCIe Gen3; dual-GPU board | NVIDIA says up to 2x application throughput with two onboard GPUs | [K80 page](https://www.nvidia.com/en-gb/data-center/tesla-k80), [K80 board spec](https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/tesla-product-literature/Tesla-K80-BoardSpec-07317-001-v05.pdf) |
+
+### Section C implementation notes
+- Add `dense_fp16_tflops`, `dense_bf16_tflops`, `sparse_fp16_bf16_tflops`, `metric_scope` (`chip`, `board`, `system`), `memory_capacity_gb`, `memory_type`, `memory_bandwidth_gbps`, `interconnect_type`, `interconnect_bandwidth_gbps`, `release_year`, `shipping_status`, `benchmark_tokens_s`, `benchmark_model`, `benchmark_config`, `benchmark_source`, `as_of`.
+- Never put AI TOPS, FP8, FP4, sparse, or system PFLOPS into the dense FP16/BF16 field. For AMD/NVIDIA sparse rows, retain sparse value and derive dense only when vendor footnote explicitly says 2x.
+- Apple unified memory is not VRAM; use `memory_scope=unified` and note CPU/GPU shared pool.
+
+## Section E — cloud pricing snapshot (USD; capture 2026-09-20; rates volatile)
+
+Prices below are exact search/extraction outputs from direct provider pages; provider pages can show different tiers/regions at different times. `—` means no current public rate verified, not that capacity never exists.
+
+| Provider + direct page | H100 80GB | H200 | A100 80GB | B200 | L40S | RTX 4090 | MI300X | Mac M2 Ultra | Billing/caveat |
+|---|---:|---:|---:|---:|---:|---:|---:|---:|---|
+| RunPod [pricing](https://www.runpod.io/pricing), [GPU models](https://www.runpod.io/gpu-models), [comparison](https://www.runpod.io/articles/comparison/choosing-gpus) | $4.79/hr serverless listing (dedicated/community varies) | $3.59/hr GPU-model snapshot; $4.31/hr clusters | $1.59/hr PCIe comparison; cluster page $1.79 SXM | $5.98/hr GPU-model snapshot; comparison $6.79 | $1.09/hr comparison | $0.34/hr Community article; product page says from $0.34 (another page headline $0.74) | — | — | Per-second; Community/Secure/Serverless/Cluster tiers and region supply change rate |
+| Lambda [instances](https://lambda.ai/instances) | $3.99/GPU-hr SXM | — | $2.79/GPU-hr 80GB SXM in 8-GPU plan | $6.69/GPU-hr | — | — | — | — | Pay by minute; A100 80GB price is per GPU within 8-GPU instance |
+| Modal [pricing](https://modal.com/pricing) | $0.001097/s = $3.9492/hr | $0.001261/s = $4.5396/hr | $0.000694/s = $2.4984/hr | $0.001736/s = $6.2496/hr | $0.000542/s = $1.9512/hr | — | — | — | Per-second compute; exact page is dynamic |
+| Vast.ai [pricing](https://vast.ai/pricing) | H100 SXM start $1.73, median $2.67/hr | H200 start $1.98, median $4.69/hr | A100 SXM start $0.20, median $0.80; PCIe start $0.33, median $0.78 | B200 start $6.25, median $8.13/hr | live guide ~$0.47 start/$0.53 median | live guide ~$0.13 start (median not captured) | — | — | Marketplace; on-demand per-second, interruptible 50%+ cheaper, reserved up to 50% off; location/host quality matters |
+| Together AI [pricing](https://www.together.ai/pricing), [GPU clusters](https://www.together.ai/gpu-clusters) | $3.99/GPU-hr on-demand; $3.19 reserved | $5.99/GPU-hr | — | HGX B200 $8.19/GPU-hr | — | — | — | — | Cluster service, typically 8–256 GPU scale; reserved terms differ |
+| Replicate [pricing](https://replicate.com/pricing), [L40S announcement](https://replicate.com/blog/nvidia-l40s-gpus-are-here) | $0.006100/s = $21.96/hr (single H100 listing) | pricing page lists H200 but amount not captured in extraction | — | — | $3.51/hr historical public-model hardware price | — | — | — | Per-second/request execution; multi-GPU H100 listed $0.0122/s = $43.92/hr; public/private model rates can differ |
+| CoreWeave [pricing](https://www.coreweave.com/pricing) | Official current page dynamic; no exact H100 amount extracted; third-party normalization $6.16/GPU-hr from $49.24 8-GPU node | third-party normalization $6.31/GPU-hr from $50.44 8-GPU node | third-party normalized $1.19/GPU-hr | $68.80/hr on-demand / $34.87 spot (node scope must be confirmed) | — | — | — | — | 8-GPU HGX minimum is common; use provider page to verify current region/node |
+| TensorDock [cloud GPU table](https://www.tensordock.com/cloud-gpus.html) | $2.25/hr H100 SXM5 | — | $1.80/hr SXM4; $1.50/hr PCIe | — | $0.95/hr L40 (not L40S; do not relabel) | $0.35/hr (site has a $0.37 product headline) | — | — | Typical hourly, varies by host; resources not included |
+| Hugging Face Endpoints [pricing](https://huggingface.co/docs/inference-endpoints/en/pricing), [pricing page](https://huggingface.co/pricing) | $10/hr (GCP H100 x1 endpoint table) | — | $2.50/hr A100 large (HF pricing page; endpoint table may vary) | — | — | — | — | — | Endpoint compute; provider/region and scale-to-zero settings matter |
+| AWS EC2 [P5](https://aws.amazon.com/ec2/instance-types/p5/), [on-demand](https://aws.amazon.com/ec2/pricing/on-demand), [capacity blocks](https://aws.amazon.com/ec2/capacityblocks/pricing/) | P5.48xlarge 8x H100: $55.04/hr on-demand => $6.88/GPU-hr (third-party link to official); Capacity Blocks examples $41.528 instance/$5.191 GPU Atlanta, $37.76/$4.720 Sydney/London | P5e/P5en supports 8x H200 (1128 GB total) but exact public rate not extracted | P4d page exists but exact current 80GB rate not extracted | — | G5/G6 prices not extracted | — | — | — | On-demand vs Capacity Blocks differs; region, OS, tenancy and egress excluded |
+| Google Cloud [GPU pricing](https://cloud.google.com/products/compute/gpus-pricing), [TPU pricing](https://cloud.google.com/tpu/pricing) | A3/H100 exact current rate not extracted from official table | A3 Ultra/H200 exact rate not extracted | — | A4/B200 exact rate not extracted | — | — | — | — | Public calculator/table is region and VM-shape dependent; do not use nonofficial estimate |
+| Azure [VM pricing](https://azure.microsoft.com/en-us/pricing/details/virtual-machines/linux/) | ND H100 v5 exact current rate not extracted | ND H200 v5 exact current rate not extracted | — | — | — | — | — | — | Region, Linux/Windows, reservation/spot matter; mark unavailable until calculator query |
+| Oracle Cloud [GPU compute](https://www.oracle.com/cloud/compute/gpu/) | Official exact public rate not extracted (third-party reports $10/hr; do not treat as official) | — | third-party reports $4/hr; not verified official | — | — | — | 8x MI300X node third-party $48/hr = $6/GPU-hr, not official Oracle page | — | OCI shape pricing, region and tenancy; verify in OCI cost estimator |
+
+### Google TPU prices (useful for calculator)
+- v5e: $1.20/chip-hour on-demand, us-central1; v5p: $4.20/chip-hour on-demand, us-east1/us-east5; Trillium v6e: $2.70/chip-hour on-demand in us-east1/us-east5; Ironwood table shows $12.00/chip-hour us-central1. Prices are from [Google Cloud TPU pricing](https://cloud.google.com/tpu/pricing), with reservation/commitment columns lower than on-demand. v5e/v5p/v6e rates are chip-hour, not pod-hour.
+
+### Pricing interpretation
+- Treat rates as timestamped observations, not stable constants. Store `provider`, `gpu_sku`, `scope` (per GPU/per node/per endpoint), `billing_unit`, `tier`, `region`, `captured_at`, `source_url`, `price_usd`, `availability`, and `notes`.
+- Directly missing providers/GPUs must remain null. Do not copy a H100 rate to H200, MI300X or Mac M2 Ultra.
+
+## Access gaps / unsuccessful attempts
+- Direct extraction timed out for several dynamic provider pages (RunPod, Lambda, Modal, Vast, Replicate, CoreWeave, TensorDock, AWS, Google, Azure, Oracle); search indexing exposed current table snippets but dynamic pages require browser/tab or provider calculators for a fully reproducible snapshot.
+- NVIDIA H200 and Google TPU pages intermittently timed out; official indexed datasheet snippets supplied H200 compute and TPU specs.
+- SambaNova official RDU exact numeric spec/LLM benchmark and M4 Ultra shipping status were not verified; mark unavailable.
+- No official exact current rates were found for Together A100/L40S/4090/MI300X/Mac, HF H200/B200 etc.; preserve nulls.
+- Apple does not publish FP16/BF16 dense TFLOPS in reviewed pages; do not invent from GPU-core counts.
+
+
+---
+
+# Section D — More Quantization Formats (tokcalc)
+
+Research date: 2026-09-20. Status: best effort, source-backed within the primary/official sources listed below. Quantization quality and speed are highly model-, calibration-, engine-, GPU-, batch-, context-, and kernel-dependent. Do not present any single benchmark as a universal factor.
+
+## 1. GGUF / llama.cpp variants
+
+GGUF (GPT-Generated Unified Format, as named by the official specification) is the file/container format used by llama.cpp; these are block-quantized weight schemes, usually WnA16 at inference (integer/FP weights with dequantization into FP16/BF16 compute). Source: [GGUF specification](https://github.com/ggml-org/ggml/blob/master/docs/gguf.md). The official llama.cpp quantizer reports effective bits/weight (including scale/metadata overhead), model size, and benchmark speeds. Source: [llama.cpp quantize README](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/quantize/README.md). The legacy per-variant PPL deltas below come from llama.cpp's `quantize.cpp` table for Llama-3-8B; they are not universal. Source: [quantize.cpp](https://raw.githubusercontent.com/ggml-org/llama.cpp/master/tools/quantize/quantize.cpp).
+
+### Official Llama-3.1-8B GGUF benchmark table
+
+The current README's FP16 reference is 14.96 GiB, 16.0005 bpw, 29.17 generated tokens/s; the quantized rows are the same benchmark run (prompt 512, generation 128). Size reduction and decode speedup are calculated from those published values: `1 - size/14.96` and `gen_tps/29.17`. They are a model-dependent llama.cpp reference, not a generic promise.
+
+| Variant | Effective bpw | Size GiB | Approx reduction vs FP16 | Published gen t/s | Derived decode speedup vs FP16 | Accuracy/perplexity evidence and caveat |
+|---|---:|---:|---:|---:|---:|---|
+| Q2_K | 3.1593 | 2.95 | 80.3% | 79.85 | 2.74x (+174%) | Legacy Llama-3-8B table: +3.5199 PPL; severe quality loss at this level. |
+| Q3_K_S | 3.6429 | 3.41 | 77.2% | 69.84 | 2.39x (+139%) | Legacy Llama-3-8B: +1.6321 PPL; low-bit degradation can be task-visible. |
+| Q3_K_M | 3.9960 | 3.74 | 75.0% | 71.68 | 2.46x (+146%) | Legacy Llama-3-8B: +0.6569 PPL. |
+| Q4_0 | approximately 4 (legacy effective bpw varies by tensor/metadata) | 4.34 | 71.0% | not published in current README row | not quantified | Legacy Llama-3-8B: +0.4685 PPL; older/simple scheme. |
+| Q4_1 | approximately 4 | 4.78 | 68.0% | not published in current README row | not quantified | Legacy Llama-3-8B: +0.4511 PPL. |
+| Q4_K_S | 4.6672 | 4.36 | 70.9% | 76.71 | 2.63x (+163%) | Legacy Llama-3-8B: +0.2689 PPL. |
+| Q4_K_M | 4.8944 | 4.58 | 69.4% | 71.93 | 2.47x (+147%) | Legacy Llama-3-8B: +0.1754 PPL; common quality/size compromise. |
+| Q5_0 | approximately 5 | 5.21 | 65.2% | not published in current README row | not quantified | Legacy Llama-3-8B: +0.1316 PPL. |
+| Q5_1 | approximately 5 | 5.65 | 62.2% | not published in current README row | not quantified | Legacy Llama-3-8B: +0.1062 PPL. |
+| Q5_K_S | 5.5704 | 5.21 | 65.2% | 69.53 | 2.38x (+138%) | Legacy Llama-3-8B: +0.1049 PPL. |
+| Q5_K_M | 5.7036 | 5.33 | 64.4% | 67.23 | 2.30x (+130%) | Legacy Llama-3-8B: +0.0569 PPL; high-fidelity compromise. |
+| Q6_K | 6.5633 | 6.14 | 59.0% | 58.67 | 2.01x (+101%) | Legacy Llama-3-8B: +0.0217 PPL; usually near-FP16 quality, but still model/task dependent. |
+| Q8_0 | 8.5008 | 7.95 | 46.9% | 50.93 | 1.75x (+75%) | Legacy Llama-3-8B: +0.0026 PPL; near-FP16 PPL in that test. |
+| FP16 reference | 16.0005 | 14.96 | — | 29.17 | 1.00x | Baseline in current README. |
+
+A second, independent 2026 evaluation of Llama-3.1-8B-Instruct reports FP16 GGUF 15,317 MiB and PPL 7.32; Q3_K_S 3,487 MiB/PPL 8.96, Q3_K_M 3,825 MiB/7.96, Q4_0 4,438 MiB/7.74, Q4_1 4,885 MiB/7.72, Q4_K_S 4,468 MiB/7.62, Q4_K_M 4,685 MiB/7.56, Q5_0 5,332 MiB/7.43, Q5_1 5,780 MiB/7.43, Q5_K_S 5,332 MiB/7.43, Q5_K_M 5,460 MiB/7.40, Q6_K 6,283 MiB/7.35, and Q8_0 8,138 MiB/7.33. Source: [2026 GGUF evaluation](https://arxiv.org/html/2601.14277v1). This demonstrates why tokcalc should store benchmark provenance and not convert PPL deltas to a universal “accuracy loss %”.
+
+![GGUF compression versus quality-loss plot from the 2026 evaluation](https://arxiv.org/html/2601.14277v1/output.png)
+
+The plot is useful as a maintainer-facing visual, but its points are still one Llama-3.1-8B-Instruct evaluation—not a universal quality curve.
+
+Engine support: llama.cpp is the reference engine and supports the family of 1.5/2/3/4/5/6/8-bit integer quantization. Source: [llama.cpp README](https://github.com/ggml-org/llama.cpp). vLLM has GGUF loading, but its own documentation calls it “highly experimental and under-optimized,” with single-file limitations in current docs. Source: [vLLM GGUF docs](https://docs.vllm.ai/en/stable/features/quantization/gguf). TGI, TensorRT-LLM and ExLlamaV2 are not GGUF-native runtimes; do not mark them as direct GGUF support without conversion.
+
+Dequant overhead: no format-wide constant. GGUF WnA16 generally dequantizes blocks during matmul; the published llama.cpp decode ratios above are the most useful end-to-end evidence. At larger batches/compute-bound workloads, a low-bit weight-only format may not speed up because dequantization and kernel utilization dominate.
+
+## 2. Weight-only PTQ formats and runtimes
+
+| Format / full name | Effective bits and approximate size vs FP16 | Accuracy/perplexity caveat | Dequant/kernel overhead and measured speed | Engine support (verified) | Primary/official sources |
+|---|---|---|---|---|---|
+| GPTQ — GPT Quantization / “GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers” | Common W3/W4, also W2 and W8 implementations. Ideal raw weight storage is about 25% of FP16 at 4-bit, 18.75% at 3-bit, 12.5% at 2-bit; group scales/zeros add metadata, so use actual checkpoint size. | Paper reports 3–4 bits with negligible degradation for tested GPT models, but low-bit quality is model/calibration dependent; use measured PPL/task metrics. | GPTQ paper reports end-to-end inference speedups around 3.25x on A100 and 4.5x on A6000 in its tested setup; this is not a general 2026 multiplier. W4A16 requires dequantization/packing; high batch can become compute-bound. | vLLM supports GPTQ across Volta/Turing/Ampere/Ada/Hopper in its support table; TGI supports pre-quantized GPTQ; TensorRT-LLM supports W4A16/W4A8 GPTQ; ExLlamaV2 supports GPTQ. Sources: [vLLM quantization](https://docs.vllm.ai/en/latest/features/quantization/), [TGI quantization](https://huggingface.co/docs/text-generation-inference/en/conceptual/quantization), [TRT-LLM quantization](https://nvidia.github.io/TensorRT-LLM/latest/features/quantization.html), [ExLlamaV2](https://github.com/turboderp-org/exllamav2). | [GPTQ paper](https://arxiv.org/abs/2210.17323), [GPTQ code](https://github.com/IST-DASLab/gptq) |
+| AWQ — Activation-aware Weight Quantization | Usually W4A16; protected salient channels are scaled, not stored as FP16 mixed-precision. Approx raw weight storage ~25% of FP16 plus scales/metadata. | AWQ protects approximately 1% salient weights based on activation distribution; paper reports strong 4-bit quality, but quality depends on calibration/data/model. | Paper reports >3x over Hugging Face FP16 in TinyChat on tested desktop/mobile settings. AutoAWQ README reports up to 3x and fused-module 1.6–2.5x; it also warns W4A16 can lose speedup in compute-bound regimes due to dequant overhead. | vLLM supports AWQ; TGI supports pre-quantized AWQ; TensorRT-LLM supports W4A16/W4A8 AWQ; Transformers/AutoAWQ ecosystem supports it. | [AWQ paper](https://arxiv.org/abs/2306.00978), [llm-awq code](https://github.com/mit-han-lab/llm-awq), [AutoAWQ](https://github.com/casper-hansen/autoawq), [TRT-LLM matrix](https://nvidia.github.io/TensorRT-LLM/latest/features/quantization.html) |
+| EXL2 — ExLlamaV2 mixed-bitrate format | Mixed 2/3/4/5/6/8-bit layers; target average bitrate can be any value roughly 2–8 bpw. 4 bpw is ~25% raw weight bytes vs FP16, 3 bpw ~18.75%; actual file includes scales/metadata. Typical Llama-3 8B practical config: 4.0 bpw is a conservative quality/VRAM point; 3.0–3.5 bpw saves more but requires validation. | Layer-wise bit allocation minimizes calibration error at target bitrate. Do not map “4.0 bpw” to one universal accuracy number. | ExLlamaV2 README gives RTX 4090 decode examples: Llama2 7B EXL2 3.0 bpw 257 t/s, 4.0 bpw 211, 5.0 bpw 179; CodeLlama 34B 4.0 bpw 50 t/s. No FP16 matched baseline in the same table, so report these as absolute measured speeds, not claimed universal speedups. Remapping is designed for little/no performance impact, but kernels and GPU dominate. | ExLlamaV2/ExLlamaV3 and TabbyAPI are primary; TGI explicitly supports pre-quantized EXL2; vLLM/TensorRT-LLM are not general EXL2-native formats in their primary support tables. | [ExLlamaV2](https://github.com/turboderp-org/exllamav2), [TGI quantization](https://huggingface.co/docs/text-generation-inference/en/conceptual/quantization) |
+| SmoothQuant — SmoothQuant: Accurate and Efficient Post-Training Quantization for LLMs | W8A8 (8-bit weights + 8-bit activations), not merely a file format; ~50% raw tensor storage vs FP16 when both weights/activations are 8-bit. It migrates activation outliers into weights using per-channel scales. | Paper reports negligible loss for tested models; calibration and model sensitivity matter. W8A8 is generally safer than very-low-bit W4 but can differ by layer. | Paper reports up to 1.56x speedup and 2x memory reduction; one A100 OPT benchmark reports up to 1.51x. These are paper-specific. Dequant/requant/scaling overhead is replaced by INT8 kernels; do not model a generic factor. | Commonly implemented in TensorRT-LLM/Model Optimizer and related HF/vLLM compression stacks; not a native GGUF/ExL2 file. TensorRT-LLM documents INT8 SQ and W8A8 recipes. | [SmoothQuant paper](https://arxiv.org/abs/2211.10438), [TensorRT-LLM quantization blog](https://nvidia.github.io/TensorRT-LLM/1.2.0/blogs/quantization-in-TRT-LLM.html) |
+| QuaRot — Outlier-Free 4-Bit Inference in Rotated LLMs | End-to-end W4A4KV4 in the paper, plus 6/8-bit variants. Raw 4-bit weights are ~25% of FP16; activations/KV also become 4-bit, so total runtime savings depend heavily on KV/context. | Llama-2 WikiText-2 PPL: 7B FP baseline 5.47 vs QuaRot 4-bit GPTQ 6.10 / QuaRot-128G 5.93; 13B 4.88 vs 5.40/5.26; 70B 3.32 vs 3.79/3.61. Llama-2-70B average zero-shot 75.98% vs FP16 77.07%. | Online Hadamard transforms add at most 7% forward-pass overhead; paper reports up to 2.16x prefill speedup, 3.71x decode-memory reduction, and 3.2–4.3x linear-layer speedup in tested setups. Do not treat as end-to-end universal. | Research PyTorch/HF implementation with CUTLASS INT4, FlashInfer KV quantization, FlashAttention; no broad direct vLLM/TGI/ExLlamaV2/TensorRT-LLM turnkey format documented. | [QuaRot paper](https://arxiv.org/html/2404.00456v1), [QuaRot code](https://github.com/spcl/QuaRot) |
+| SpinQuant — LLM quantization with learned rotations | 4-bit weights, activations, and KV cache in the paper (W4A4KV4); raw 4-bit weight storage ~25% of FP16 before metadata. Learned rotations are optimized to make the model more quantization-friendly. | On Llama-2 7B, the paper reports a 2.9-point zero-shot gap to full precision, outperforming LLM-QAT by 19.1 points and SmoothQuant by 25.0 points; on hard-to-quantize Llama-3 8B it reduces the gap to FP by up to 45.1% relative to QuaRot. These are paper-specific task results, not a generic accuracy-loss percentage. | No matched end-to-end speedup range was verified from the primary source; rotation application and low-bit kernels are implementation-dependent. Store overhead as unknown unless measured for the chosen engine. | Official PyTorch/CUDA research implementation and model/checkpoint code; no verified turnkey support in llama.cpp, vLLM, TGI, TensorRT-LLM, or ExLlamaV2 primary matrices. | [SpinQuant paper](https://arxiv.org/abs/2405.16406), [official code](https://github.com/facebookresearch/SpinQuant) |
+| QuIP# — QuIP#: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks | Primarily 2-bit weights using E8 lattice codebooks; model zoo also 3/4-bit. 2-bit raw weights ~12.5% FP16 before codebook/scales. | Authors report near-FP16 performance at 2 bits for tested Llama 1/2 models; specialized calibration/rotations and model architecture matter. No universal PPL. | Custom CUDA kernels; third-party vLLM benchmark reports ~159 t/s Llama-7B batch 1 on RTX 4090, but no matched FP16 baseline in that source, so do not convert to generic speedup. | Official CUDA code/model zoo; niche integrations (third-party vLLM/QuIP-for-all). No verified mainstream vLLM/TGI/TensorRT-LLM/ExLlamaV2 direct support in primary docs. | [QuIP# paper](https://arxiv.org/abs/2402.04396), [official code](https://github.com/Cornell-RelaxML/quip-sharp), [QuIP-for-all benchmark](https://github.com/chu-tianxiang/QuIP-for-all) |
+| HQQ — Half-Quadratic Quantization | Calibration-free 8/4/3/2/1-bit; actual storage includes scales/zeros. 4-bit raw ~25% FP16, 2-bit ~12.5%; no single canonical checkpoint overhead. | No calibration data required. Official README says axis=0 generally has better quality than axis=1 at low bits, while optimized inference backends often use axis=1; validate PPL/task quality. | Official repo reports ~158 t/s Llama-3-8B 4-bit on RTX 4090 with optimized backend; no FP16 matched baseline, so no universal speedup. On-the-fly quantization trades setup/dequant overhead for convenience. | HQQ/PyTorch, torch.compile, ATen/CUDA, GemLite, torchao int4, HF Transformers/PEFT, and vLLM via GemLite/on-the-fly path. | [HQQ code](https://github.com/dropbox/hqq), [HF HQQ docs](https://huggingface.co/docs/transformers/quantization/hqq), [HQQ technical blog](https://dropbox.github.io/hqq_blog/) |
+| EETQ — Easy and Efficient Quantization for Transformers | INT8 weight-only per-channel (W8A16); raw weights ~50% FP16 plus per-channel scales. | Usually high fidelity relative to lower-bit W4, but accuracy varies by model; no universal PPL/accuracy percentage found in official docs. | Uses FasterTransformer/TensorRT-LLM GEMM/GEMV kernels and FlashAttention2; no stable official cross-model speedup range found. On-the-fly TGI quantization adds startup cost; runtime dequantization/kernel behavior is engine-dependent. | HF Transformers and TGI; TGI supports `--quantize eetq` on the fly. Not a direct GGUF/EXL2 format. vLLM current primary support table does not establish EETQ as a universal backend. | [EETQ repo](https://github.com/NetEase-FuXi/EETQ), [HF EETQ docs](https://github.com/huggingface/transformers/blob/main/docs/source/en/quantization/eetq.md), [TGI quantization](https://huggingface.co/docs/text-generation-inference/en/conceptual/quantization) |
+| BitBLAS — mixed-precision BLAS kernel library, not a model file format | Supports many W/A combinations (e.g., W4A16 GPTQ, INT2/INT8 BitNet, FP4/FP8/INT4/INT2/NF4); effective size is inherited from chosen weight format. | Accuracy is determined by the source quantizer/checkpoint (GPTQ/AWQ/BitNet etc.), not BitBLAS itself. | BitBLAS README reports BitNet WINT2AINT8 GEMV/GEMM 8x/2x vs cuBLAS FP16 on A100; kernel microbenchmark, not a universal end-to-end LLM speedup. | PyTorch, GPTQModel/AutoGPTQ, vLLM integration, BitNet integration; hardware-specific A100/A6000/V100/RTX 4090 matrices. | [BitBLAS repo](https://github.com/microsoft/BitBLAS), [Ladder paper](https://www.usenix.org/conference/osdi24/presentation/wang-lei) |
+
+## 3. Native low-precision formats
+
+| Format | Definition / effective bits and size | Accuracy caveat | Dequant/compute overhead and measured speed | Engine support/status | Primary/official sources |
+|---|---|---|---|---|---|
+| BitNet b1.58 | Native ternary weights in {-1,0,+1}; paper names 1.58 bits because ternary entropy is log2(3), but practical packed formats may use ~2 bits/weight plus scales/metadata. Compared with FP16, ideal raw weight storage is ~9.9% at 1.58 bits or 12.5% at 2-bit packing. | Requires a model trained or specifically converted for ternary weights; ordinary FP16 model PTQ is not equivalent. Original paper and 2B4T report parity/competitive quality against similarly sized baselines in tested tasks, but not broad universal accuracy. | Official bitnet.cpp reports CPU speedups: roughly 1.37–5.07x on ARM and 2.37–6.17x on x86, with energy reductions in its tested setups; these are runtime/model/platform-specific. No general GPU speed claim should be inserted. | Microsoft bitnet.cpp is official CPU-focused inference (x86/ARM, I2_S/TL1/TL2 kernels; GPU/NPU support status should be checked per release). Not a drop-in GGUF/GPTQ quant of arbitrary models. | [BitNet foundational paper](https://arxiv.org/abs/2402.17764), [CPU inference report](https://arxiv.org/abs/2410.16144), [official BitNet runtime](https://github.com/microsoft/BitNet), [2B4T model](https://huggingface.co/microsoft/BitNet-b1.58-2B-4T) |
+| FP8 — 8-bit floating point (E4M3/E5M2) | W8A8 or W8A16 depending recipe; raw tensor storage ~50% FP16. E4M3/E5M2 range/precision differ; scale metadata may be per-tensor, rowwise, or blockwise. | Usually near-FP16 on supported models with calibration; TensorRT-LLM’s example reports MMLU losses 0.14–0.89% for FP8 across Falcon/Llama/MPT rows, but this is not a universal accuracy percentage. | TensorRT-LLM Llama-2-7B H100 benchmark: FP8 1.40–1.51x at BS 1–8, and 2.3x at latency-constrained BS16. A tuned Llama-3.3-70B H100 case reports 6049 vs 2474 token/s = 2.44x (+144.5%) vs tuned FP16, with TTFT -40.4%; use only with conditions. | Production-grade on H100/H200/Ada/Blackwell paths in TensorRT-LLM; vLLM W8A8 officially Hopper/Ada and some AMD MI300X paths; TGI supports FP8 on the fly. | [FP8 paper](https://arxiv.org/abs/2209.05433), [TRT-LLM benchmark](https://nvidia.github.io/TensorRT-LLM/1.2.0/blogs/quantization-in-TRT-LLM.html), [TRT-LLM FP8 tuning guide](https://nvidia.github.io/TensorRT-LLM/0.19.0/performance/performance-tuning-guide/fp8-quantization.html), [vLLM FP8 docs](https://docs.vllm.ai/en/v0.11.2/features/quantization/fp8) |
+| NVFP4 — NVIDIA FP4 (E2M1) with hierarchical scaling | Each value is 1 sign + 2 exponent + 1 mantissa = 4 bits; NVIDIA’s 2025 technical blog specifies an E4M3 FP8 scale per 16-value microblock plus a tensor-level FP32 scalar. Approx effective weight storage ~4.5 bits/value before tensor-scale amortization (4 + 8/16), about 3.56x smaller than FP16; do not call it “exactly 4 bits/weight.” | NVIDIA’s Jun-2025 blog reports 1% or less accuracy degradation vs FP8 on seven DeepSeek-R1-0528 evaluations (AIME 2024 was 2% better) for that PTQ recipe; general quality remains model/recipe dependent. NVFP4 cannot be assumed safe for ordinary FP16 PTQ without validation. | Blackwell tensor cores execute native FP4; NVIDIA blog reports memory/compute benefits but not a portable end-to-end tokens/s multiplier. The official TensorRT-LLM support matrix marks NVFP4 on Blackwell, not Hopper. Avoid a universal speedup unless a benchmark supplies model, GPU, batch, and baseline. | TensorRT-LLM/Model Optimizer/Transformer Engine on Blackwell; TRT-LLM docs list NVFP4, MXFP4, FP8 and KV-cache support by SM generation. | [NVIDIA NVFP4 blog (Jun 2025)](https://developer.nvidia.com/blog/introducing-nvfp4-for-efficient-and-accurate-low-precision-inference), [Transformer Engine NVFP4 docs](https://docs.nvidia.com/deeplearning/transformer-engine-releases/release-2.13/user-guide/features/low_precision_training/nvfp4/nvfp4.html), [TRT-LLM matrix](https://nvidia.github.io/TensorRT-LLM/latest/features/quantization.html), [Model Optimizer](https://github.com/NVIDIA/Model-Optimizer) |
+| mxFP8 / OCP MX — microscaling FP8 | OCP MX groups 32 elements with a shared E8M0 scale and 8-bit FP8 elements (E4M3 or E5M2). Effective storage ≈8 + 8/32 = 8.25 bits/value, ~48.4% of FP16 raw bytes (1.94x reduction), plus tensor/layout metadata. MXFP4/MXFP6 also exist; user specifically asked mxFP8. | MX paper reports MXFP8 near-lossless in broad evaluated settings; exact LLM PTQ quality varies by format, scale granularity, and hardware. No universal PPL loss. | Scale metadata and block-scale operations add implementation complexity; speedups are hardware/kernel-specific. Do not promise FP8’s 2x without a matched benchmark. | OCP spec is open; NVIDIA Blackwell/TRT-LLM supports MXFP8-related block scaling paths, and vLLM lists MXFP8/MXFP4 in current project support. Broad cross-vendor production support is still emerging; vLLM issue/docs show ongoing work. | [OCP MX v1.0 spec](https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf), [MX paper](https://arxiv.org/abs/2310.10537), [Microsoft microxcaling](https://github.com/microsoft/microxcaling), [vLLM project](https://github.com/vllm-project/vllm) |
+
+## 4. What tokcalc should store
+
+Do not model “quantization” as one scalar percentage. Store a normalized format/checkpoint record with:
+
+```ts
+type QuantizationRecord = {
+  id: string;                    // gguf:q4_k_m, gptq:w4a16:g128, fp8:e4m3, nvfp4, ...
+  family: 'gguf'|'gptq'|'awq'|'exl2'|'smoothquant'|'rotation'|'hqq'|'eetq'|'kernel'|'native_fp'|'mx';
+  displayName: string;
+  fullName?: string;
+  weightBits: number;            // nominal element bits (4, 8, 1.58, ...)
+  effectiveBitsPerWeight?: number;// includes scales/zeros/metadata if measured
+  activationBits?: number;
+  kvCacheBits?: number;
+  granularity?: 'tensor'|'channel'|'row'|'group'|'block'|'microblock'|'mixed_layer';
+  groupSize?: number;            // e.g. 128; NVFP4 microblock 16; OCP MX 32
+  scaleBits?: number;
+  zeroPoint?: boolean;
+  accumulationType?: 'fp16'|'bf16'|'fp32'|'int32'|string;
+  sizeBytesMeasured?: number;    // prefer actual checkpoint size
+  sizeBytesEstimated?: number;   // estimate from params*effectiveBits/8 + nonquantized tensors
+  nonQuantizedFraction?: number;// embeddings, norms, lm_head, etc.
+  calibration?: 'none'|'data_free'|'calibration_required'|'qat'|'unknown';
+  modelCompatibility?: string[];
+  engineSupport?: Record<string, 'native'|'plugin'|'experimental'|'conversion'|'unsupported'|'unknown'>;
+  benchmarkRefs?: Array<{url:string; model:string; gpu:string; batch?:number; context?:number; metric:string; value:number; baseline?:number;}>;
+  qualityRefs?: Array<{url:string; model:string; dataset:string; metric:'ppl'|'accuracy'|'loss'; baseline:number; quantized:number;}>;
+  statusAsOf: string;
+};
+```
+
+Recommended formulas (show assumptions):
+
+- Raw weight estimate: `params * effectiveBitsPerWeight / 8`.
+- Better checkpoint estimate: `quantized tensors + non-quantized tensors + scales/zeros + metadata`; use an actual file size whenever possible.
+- KV memory: `tokens * layers * 2 * kv_heads * head_dim * bytes_per_kv_element`, then multiply by replication/parallelism factors; weight quantization does not automatically quantize KV.
+- Throughput: store separate prefill and decode values, not one “tokens/sec”. Store batch, prompt/output lengths, context, GPU, engine version, kernel, tensor parallelism, and whether CUDA graphs/FlashAttention/speculative decoding were enabled.
+- Accuracy: store raw metric and baseline/quantized pair (e.g., PPL 7.32→7.56 or MMLU 69.1→68.5), not “accuracy loss = 0.87%” unless source defines that calculation. PPL is not accuracy; a PPL percentage change cannot be reported as an accuracy percentage.
+- Speed: represent `speedup = quant_tps / baseline_tps` only when source gives matched baseline conditions. Keep measured range by workload category (prefill/decode, batch 1/high batch) and label it “benchmark-specific”.
+
+## 5. Implementation recommendation for tokcalc
+
+1. Must-have: GGUF variants with actual effective bpw, measured size, and a low/medium/high quality note. Default user-facing presets: Q4_K_M, Q5_K_M, Q6_K, Q8_0. Include Q2/Q3 only with warning badges.
+2. Must-have: GPTQ/AWQ/EXL2 W4A16 as separate engine-aware records; do not merge them into “INT4.”
+3. Must-have: FP8 W8A8 for H100/H200/Ada/Blackwell with engine-specific benchmark ranges and a warning that speed is workload dependent.
+4. Nice: NVFP4 and MXFP8 as hardware-gated Blackwell/OCP records, with storage estimate including scales.
+5. Future/research: QuaRot, SpinQuant, QuIP#, HQQ, BitBLAS, BitNet, SmoothQuant, EETQ as an “experimental / specialized” tab. They need checkpoint/runtime compatibility metadata, not merely an extra quant dropdown.
+
+## 6. Evidence gaps / blockers
+
+- No primary source gave a universal accuracy-loss percentage, dequantization factor, or real-world speedup for every requested format. Such values are not scientifically portable; use null/“benchmark required” rather than fabricated defaults.
+- Current llama.cpp README does not publish matched decode speeds for Q4_0/Q4_1/Q5_0/Q5_1, so those rows deliberately leave speedup unquantified.
+- HQQ, QuIP#, EETQ, and NVFP4 official pages expose absolute or selected benchmark numbers but not matched FP16 end-to-end baselines in all cases; do not convert them to speedup without a baseline.
+- vLLM/TGI support changes quickly. TGI documentation says it is in maintenance mode and recommends vLLM/SGLang/local engines for new work; tokcalc should timestamp engine support and version-pin it.
+- Search/extraction timed out on several dynamic GitHub/NVIDIA/Hugging Face pages; where possible I used raw GitHub files, official docs, papers, and search-confirmed page titles. No claim above relies solely on a search snippet when a primary source was available.

package/tool-results/bash_1789929256963_2a52aff0d0a8.txt

@@ -0,0 +1,160 @@
+     1→# Research memo: tokcalc growth and roadmap
+     2→
+     3→Tokcalc’s strongest opportunity is to become the **engineering-grade “pre-deployment decision layer” for LLM inference**: workload-aware capacity, latency, KV-cache, topology, and cost planning—not another static GPU or API-price calculator. The evidence supports prioritizing open-source distribution, high-intent comparison SEO, and integrations into the serving ecosystem over broad paid acquisition.
+     4→
+     5→**Important limitations (as of 20 September 2026):** several requested competitor domains, alleged future hardware/model releases, social subscriber counts, influencer follower counts, and third-party SEO metrics cannot be independently verified from reliable public sources in this research pass. I do not invent those values. For volatile pricing and unreleased-product claims, tokcalc should label data with source, region, retrieval date, and confidence.
+     6→
+     7→## A. Acquisition channels
+     8→
+     9→### Channel priority
+    10→
+    11→| Rank | Channel | Reach / size signal | Engagement cost | Evidence of dev-tool success | Specific tokcalc launch tactic | Feasibility | Effort | Expected ROI |
+    12→|---:|---|---|---|---|---|---|---|---|
+    13→| 1 | GitHub + README + releases | GitHub is a primary discovery layer for infrastructure; unusually strong open-source projects can compound rapidly. For example, OpenClaw reportedly crossed 100K stars in under two weeks, while Graphiti reached 20K stars within roughly a year. These are exceptional, not baseline expectations.  [developersdigest](https://www.developersdigest.tech/blog/openclaw-github-star-chart-vertical-line) | Low | GitHub-native open source can turn users into contributors, backlinks, and downstream docs references | Make the repository the product’s canonical source: interactive screenshots/GIF, “capacity plan in 30 seconds,” benchmark-source methodology, calculators as shareable URLs, one-click issue templates for models/GPUs, and a public data-quality roadmap | High | Med | Very high |
+    14→| 2 | High-intent SEO | Queries around GPU comparison, model inference speed, long context, quantization, and API-vs-self-hosting express an imminent engineering or buying decision | Low–Med | Engineering comparison pages can earn links when they include reproducible assumptions and calculators—not merely prose | Generate programmatic but editorially reviewed landing pages such as `/llama-3-70b/h100/vllm`, with live calculator presets, sensitivity tables, assumptions, citations, and shareable scenario URLs | High | High | Very high |
+    15→| 3 | Hacker News | Small relative to mass social networks but highly concentrated among technical builders and decision-makers. A 2024–25 study of 138 AI/LLM repo launches found a mean 121 GitHub stars in 24 hours, 189 in 48 hours, and 289 in a week after HN exposure; timing mattered more than merely using “Show HN.”  [arxiv](https://arxiv.org/abs/2511.04453) | Low | Engineering depth and direct interaction consistently matter more than growth marketing language | Post a **Show HN: Tokcalc — open-source capacity planner for serving LLMs** only when the tool has a genuinely surprising technical artifact: e.g., “Given traffic, P95 TTFT, context, and cache hit rate, find the cheapest feasible topology.” Publish formulas, limitations, and a public dataset | High | Med | High |
+    16→| 4 | Reddit: r/LocalLLaMA first | Strong practitioner concentration around local inference, quantization, VRAM constraints, hardware, and benchmarks; exact subscriber figures must be captured manually on launch day because they change continuously | Low | Utility-first benchmark posts and transparent build logs tend to fit better than product announcements | Do not post “I made a calculator.” Post an original analysis: “How many concurrent 32K-context Llama/Qwen users fit on 4090, H100, H200, and B200 under vLLM?” Include methodology, raw data, calculator link, and invite corrections | High | Med | High |
+    17→| 5 | Serving-engine communities | vLLM and SGLang are actively releasing and expanding into hardware plugins and multimodal serving; their users already have the deployment problem tokcalc solves. vLLM’s repository listed v0.29.0 on 9 September 2026; SGLang listed v0.5.20 on 18 September 2026.  [github](https://github.com/vllm-project/vllm?utm_cta=website) | Low | Open-source users reward tools that reduce configuration uncertainty | Offer engine-specific presets and contribute documentation examples: “capacity-planning workflow for vLLM/SGLang.” Seek maintainer approval before requesting a docs link | High | Med | High |
+    18→| 6 | X / “LLM Twitter” | Fast discovery and feedback loop, but reach is volatile and follower counts are not a durable acquisition metric | Med | Demos, visual benchmark charts, and credible technical threads travel more reliably than generic launch posts | Ship one visual “capacity card” per week: model, context length, latency SLO, GPU/topology, concurrency, and monthly cost. Quote-post benchmark authors with a reproducible tokcalc scenario | High | Med | Med–High |
+    19→| 7 | LinkedIn | Viable primarily for infra buyers, platform leads, and cloud/GPU operators—not for broad open-source virality. Zep reportedly ran 23 LinkedIn ads among 33 active ads in August 2026, suggesting it can be a targeted B2B distribution channel.  [systemaic](https://www.systemaic.com/teardowns/zep-ai) | Med–High | Decision-maker content benefits from concrete economics and case studies | Publish operator-oriented posts: “Why a 128K context limit turns your H100 fleet into a KV-cache capacity problem.” Gate nothing; use a scenario calculator and invite architecture reviews | Med | Med | Med |
+    20→| 8 | Product Hunt | Still useful for launch-day discovery, but should be a secondary distribution event, not the strategy. PH reported 633 Developer Tools launches in Q2 2026, up 136% year over year; Open Source grew 143%. AI averaged 252 upvotes, Dev Tools 201, and Open Source 192.  [producthunt](https://www.producthunt.com/p/general/product-hunt-s-state-of-tech-discovery-q2-2026) | Med | Can create social proof and backlinks, but category noise is severe | Launch as **Open Source + Developer Tools + AI**. Avoid a generic “AI calculator” pitch; show a real capacity-plan workflow and publish an accompanying GitHub release and HN post on different days | High | Low–Med | Med |
+    21→| 9 | YouTube | Useful when a creator can demonstrate a full deployment decision rather than “reviewing a website”; public sponsorship pricing varies too much to cite as a reliable market rate | High | Technical creators amplify tools that improve an actual build workflow | Target small-to-mid technical channels first with a reproducible challenge: “plan a 70B model serving stack under \$X/month and prove the math.” Offer no paid claim requirements | Med | Med–High | Med |
+    22→| 10 | Dev.to / Medium / Substack | Better for evergreen technical explainers than direct product launch traffic; no credible cross-platform “LLM engineer readership per article” benchmark or universal view counts is publicly available | Low–Med | Deep, original engineering explainers can become SEO assets and link sources | Publish canonical articles on your domain first, then syndicate with canonical links: “Continuous batching math,” “KV cache per concurrent user,” and “When API pricing beats self-hosting” | High | Med | Med–High |
+    23→| 11 | Conferences | Strong for partnerships and design partners, weak for low-cost mass acquisition | High | Best for obtaining customer language, validation data, and integrations | Attend before sponsoring. Prioritize AI Engineer Summit for practitioner discovery; MLSys for serving credibility; GTC only if pursuing NVIDIA/cloud partnerships | Med | High | Med |
+    24→
+    25→### Reddit operating rules
+    26→
+    27→| Subreddit | Likely fit for tokcalc | What tends to perform | What risks removal or backlash | Recommended post |
+    28→|---|---|---|---|---|
+    29→| r/LocalLLaMA | Excellent | Local hardware tests, quantization comparisons, VRAM math, reproducible benchmarks, model-serving guides | Link-only promotion, unverifiable token/s claims, affiliate framing, reposting | “I modeled 4K/32K/128K concurrency for Llama/Qwen across 4090, Mac, H100, H200, B200—here are the formulas and calculator.” |
+    30→| r/MachineLearning | Moderate | Research contribution, methodology, benchmark analysis, open datasets | Product marketing, shallow demos, claims without methodology | Post only with a technical artifact: public workload trace schema, benchmark normalization method, or reproducible capacity-planning dataset |
+    31→| r/MLscaling | Excellent | Cluster economics, systems bottlenecks, serving architecture, topology | Simplistic GPU calculator framing | “A workload-driven model for prefill/decode disaggregation and topology selection” |
+    32→| r/LLMDevs | Good | API implementation, tool launches with code, practical deployment advice | Link dumping, broad “AI tool” content | “Open-source planner that exports a vLLM/SGLang configuration checklist” |
+    33→| r/ArtificialIntelligence | Low–Moderate | Accessible explainers, visual comparisons | Overly technical product pitches, spam | Use only after technical community validation; publish a simple “self-host vs API” explainer |
+    34→| r/SideProject | Good for founder feedback | Build stories, metrics, design feedback, launch retrospectives | Aggressive promotion without a story | “I built a free open-source LLM capacity planner—what assumptions would make you distrust it?” |
+    35→
+    36→**Recommendation:** Treat Reddit as an engineering peer-review channel, not an ad platform. Lead with a useful answer, disclose authorship, put methodology above CTA, and follow each community’s current rules on the day of posting. The public record in this pass did not provide reliable current subscriber counts or full moderation-rule text for all six communities, so scrape/capture these manually immediately before launch rather than embedding stale figures.
+    37→
+    38→### Hacker News: format that fits
+    39→
+    40→| Element | What to do | Why |
+    41→|---|---|---|
+    42→| Title | “Show HN: Tokcalc – open-source LLM serving capacity planner” | Clear, literal, technically scoped |
+    43→| First paragraph | State what it calculates and what it does **not** know: model, engine, GPU, context, traffic profile, SLO, cache behavior, topology assumptions | HN responds better to precision than aspiration |
+    44→| Proof | Include equations, sources, benchmark provenance, a worked example, and GitHub source | Makes claims inspectable |
+    45→| Differentiator | “Not a speed chart: it separates prefill/decode, KV-cache limits, continuous batching, prompt caching, and topology.” | Matches the actual product wedge |
+    46→| Founder participation | Be present for 8–12 hours and answer criticism with changes/links rather than defensiveness | Discussion is part of the launch |
+    47→| Avoid | “The first,” “revolutionary,” inflated accuracy claims, tracking-heavy signup gates, and vague “AI infra platform” language | These are common credibility failures |
+    48→
+    49→A defensible benchmark is the HN study: HN exposure correlated with average gains of 121 stars in 24 hours and 289 over a week among the sampled AI/LLM launches, but a “Show HN” tag itself had no independent statistical advantage once other factors were controlled. Build the artifact first; title format alone is not a growth hack. [arxiv](https://arxiv.org/abs/2511.04453)
+    50→
+    51→**Requested three specific recent successful AI-infra Show HN launches:** the available source results did not reliably provide three verified Show HN URLs, launch dates, and outcome data in the requested window. Do not cite unaudited listicles as evidence. Before posting, use the Algolia HN API or HN search to assemble a current, source-linked comp set by searching “Show HN” plus `inference`, `LLM`, `GPU`, `vLLM`, and `benchmark`.
+    52→
+    53→### X: practical target list
+    54→
+    55→Follower counts are volatile, often partly inauthentic, and require live profile/API checks; I cannot responsibly provide “current top 20” counts without a verifiable real-time social data source. Instead, build a target list around the people whose audiences match the product:
+    56→
+    57→| Cluster | Example accounts to validate before outreach | What to offer | Outreach angle |
+    58→|---|---|---|---|
+    59→| Open-model / local LLM | @ggerganov, @TheBlokeAI, @CohereForAI, @huggingface | Hardware/model scenario cards, quantization math | “We added a transparent planner for your model formats; can you audit the assumptions?” |
+    60→| Serving engines | @vllm_project, @sgl_project, NVIDIA TensorRT-LLM team accounts | Engine presets, benchmark ingestion, docs example | “Could tokcalc export a reproducible sizing plan for your deployment guide?” |
+    61→| Inference systems | @tri_dao, @ClementDelangue, @swyx, @hamelhusain | Architecture and cost visualizations | “Here is an interactive version of a capacity-planning question your audience asks repeatedly.” |
+    62→| GPU clouds / infrastructure | @runpod_io, @LambdaAPI, @modal_labs, @replicate, @togethercompute | Current-price adapter or embedded planner | “Let users estimate capacity on your SKUs with citations and a clear price timestamp.” |
+    63→| AI engineering community | @aiengineer, @LatentSpacePod, @Cognition, @LangChainAI | Shareable benchmark/cost cards | “We turned serving economics into a scenario link people can fork.” |
+    64→
+    65→Do **not** mass-DM twenty people with an identical launch request. Make one relevant capability or dataset contribution per account class, obtain feedback publicly where appropriate, and make the output useful even if nobody reposts it.
+    66→
+    67→### Community and directory integrations
+    68→
+    69→| Surface | Integration process | Recommendation | Feasibility | Effort | Expected ROI |
+    70→|---|---|---|---|---|---|
+    71→| Hugging Face model cards | Submit PRs only where project maintainers welcome it; use model-card metadata and linked Spaces where appropriate | Build a HF Space/Gradio wrapper or static “capacity-plan” link generator. Ask model owners to add a **Community resources** link; do not edit third-party cards without approval | High | Med | High |
+    72→| Hugging Face Spaces | Create a Space that calls a versioned tokcalc calculation API or bundles the open-source calculator | Use as discovery/demo, but keep canonical pages on tokcalc.dev for SEO and source governance | High | Low–Med | Med |
+    73→| Replicate | Model pages and APIs are provider-controlled | Propose a “capacity / self-host alternative” co-marketing widget; avoid assuming they will link externally | Med | Med–High | Med |
+    74→| Artificial Analysis | Its product is a benchmark and model-intelligence platform | Offer a transparent “capacity planner” outbound link/API where benchmark data feeds tokcalc assumptions; negotiate data licensing/provenance | Med | High | High |
+    75→| Papers with Code | Contribution requires a relevant paper/task/repository association | Publish an open benchmark dataset or methodology paper, not a promotional listing | Low–Med | High | Low–Med |
+    76→| GitHub Marketplace | Publish a GitHub Action with an action.yml, marketplace metadata, version tags, and documented permissions | `tokcalc/plan`: read a YAML traffic profile, calculate recommended capacity and estimated cost, post a PR comment/check | High | Med | High |
+    77→| VS Code Marketplace | Build a TypeScript extension | “Estimate serving impact” from model config, deployment YAML, or prompt/context profile; deep-link to full plans | High | Med | Med |
+    78→| MCP registry/ecosystem | MCP is now a large distribution layer; one report cited the TypeScript SDK at 52.6M weekly npm downloads in August 2026, although this should be independently rechecked before using it in marketing.  [buttondown](https://buttondown.com/Builder-Radar/archive/builder-radar-week-of-august-2-2026/) | Release a read-only tokcalc MCP server: `estimate_capacity`, `compare_gpu`, `estimate_api_vs_self_host`, `get_assumptions` | High | Med | Very high |
+    79→
+    80→## B. SEO opportunity analysis
+    81→
+    82→### Measurement caveat
+    83→
+    84→Ahrefs and Semrush do not provide a complete public, reproducible keyword-volume and difficulty dataset for the 20+ requested exact variants. Therefore, I will not fabricate monthly volumes or keyword-difficulty scores. Build a repeatable keyword-data job using an Ahrefs/Semrush account, Google Search Console, and Google Ads Keyword Planner; tag each number with country, device, match type, retrieval date, and tool.
+    85→
+    86→The most important SEO insight is that the requested terms are **not one keyword set**. They break into three very different intent classes:
+    87→
+    88→1. **Product/model lookup** — e.g., “Llama 4 Scout vs Maverick,” “DeepSeek R1 cost per million tokens.” Fast-changing, high freshness requirement.
+    89→2. **Comparison / deployment selection** — e.g., “H100 vs H200 LLM inference,” “vLLM vs TGI vs SGLang throughput.” Strong commercial and integration intent.
+    90→3. **Capacity-planning problem** — e.g., “how many concurrent users LLM GPU,” “long context LLM cost,” “continuous batching multiplier.” Likely lower raw volume but unusually high product fit and weaker competition.
+    91→
+    92→### Keyword cluster recommendations
+    93→
+    94→| Cluster / target query family | Volume / KD status | Likely ranking incumbents | Can tokcalc reach top 10 in 6 months? | Page to build | Feasibility | Effort | Expected ROI |
+    95→|---|---|---|---|---|---|---|---|
+    96→| “tokens per second” + Llama 3 / Qwen / DeepSeek / Mistral / Mixtral variants | **Estimated; obtain paid-tool export.** Exact model queries are fragmented and freshness-sensitive | Model docs, benchmark databases, Reddit, provider blogs, GitHub benchmark repos | Yes for long-tail combinations if pages add measured assumptions and calculator presets; no guarantee for head terms | One hub plus real pages for each model-family × hardware/engine combination. Show prefill, decode, batch, TTFT, ITL, context, quantization, methodology | High | High | Very high |
+    97→| H100 vs H200 / H100 vs A100 / B200 vs H100 for LLM inference | **Estimated; commercial comparison cluster** | Cloud provider docs, NVIDIA, benchmark vendors, cloud-pricing blogs | Yes for long-tail workload pages; head terms are competitive | “H100 vs H200 for 70B serving at 8K/32K/128K” with cost per useful output token, not raw FLOPS | High | Med | High |
+    98→| RTX 4090 / 3090 / 5090 LLM inference | **Estimated; enthusiast plus developer intent** | Local-LLM communities, hardware review sites, Ollama/llama.cpp content | Yes, especially with memory/quant/context constraints | “Can odel] run on RTX [GPU]?” pages with VRAM, quantization, expected single-user and batched serving scenarios | High | Med | High |
+    99→| Mac M2 Ultra / M3 Max / M4 Max LLM | **Estimated; persistent hardware intent** | Apple/MLX/llama.cpp guides, local-AI blogs | Moderate–High | Mac local inference planner, including unified-memory budget, context, MLX/llama.cpp engine presets | High | Med | Med–High |
+   100→| GGUF Q4_K_M vs Q5_K_M; GGUF vs GPTQ vs AWQ | **Estimated; educational long-tail** | Hugging Face, TheBloke-era guides, GitHub docs, community posts | High if technically rigorous and maintained | Quantization decision tree: quality, model weight size, kernels/engine support, VRAM, throughput—not a generic comparison | High | Med | High |
+   101→| vLLM vs TGI vs SGLang throughput | **Estimated; high-value engineering comparison** | Official docs, GitHub, vendor benchmarks, independent analyses | Moderate; high competition but strong differentiator possible | Workload matrix: identical model, request mix, shared-prefix rate, context, TTFT/ITL, topology; link directly to planner | High | High | Very high |
+   102→| FP8 vs FP16 H100 | **Estimated; engineering intent** | NVIDIA, TensorRT-LLM, framework docs | Moderate–High | Precision calculator with model support, memory impact, expected accuracy caveats, and engine/hardware compatibility | High | Med | High |
+   103→| self host vs OpenAI API | **Estimated; broad and competitive** | OpenAI, cloud vendors, cost-calculator blogs | Moderate for generic head term; High for “at [traffic/model/context]” long-tail | Interactive break-even calculator with utilization, reserved/spot/on-demand, labor/ops, prompt caching, reasoning tokens | High | High | Very high |
+   104→| how many concurrent users LLM GPU | **Estimated; likely modest but ideal product intent** | Forums, docs, sporadic blogs | High | Canonical explainer + calculator: concurrency is a workload/SLO/KV-cache question, not a GPU constant | Very high | Med | Very high |
+   105→| long context LLM cost | **Estimated; emerging, growing commercial intent** | API pricing pages, model provider blogs | High | Long-context cost planner: prefill, KV memory, cache hit rate, context distribution, batching, output ratio | Very high | Med | Very high |
+   106→| DeepSeek R1 cost per million tokens | **Estimated; freshness-heavy** | Official provider/API pages, aggregators | Moderate; dependent on rapid pricing updates and authority | Date-stamped source table plus “effective cost including reasoning/prompt cache”; no evergreen unsupported claims | Med | Low–Med | Med |
+   107→| Llama 4 Scout vs Maverick | **Estimated; model-comparison freshness query** | Meta, Hugging Face, reviews/benchmarks | Moderate | Capability + serving-cost comparison with explicit release/version dates and workload presets | Med | Med | Med |
+   108→| AMD MI300X vs NVIDIA H100 | **Estimated; high buyer intent** | AMD/NVIDIA, cloud providers, benchmark sites | Moderate–High | Model-specific serving comparison, ROCm maturity and engine support, not merely hardware specs | High | Med | High |
+   109→| 1M context window model comparison | **Estimated; emerging high-intent topic** | Provider docs, model review sites | High | Context economics table: max context, attention behavior, supported serving engines, price/KV implications | High | Med | High |
+   110→| continuous batching multiplier | **Estimated; low-volume expert query** | vLLM docs, papers, technical blogs | High | Explain why no universal multiplier exists; interactive arrival-rate, request-size, SLO, batch, cache simulation | Very high | Med | High |
+   111→| Anthropic prompt caching savings | **Estimated; strong branded topic, fast-changing pricing** | Anthropic official docs | Low–Moderate for head term; High for calculators/examples | Source-cited calculator that implements current rules and clearly shows timestamp/version | Med | Low | Med |
+   112→| reasoning tokens cost o1 / o3-mini | **Estimated; branded and mutable** | OpenAI pricing/docs | Low–Moderate for head query; High for explanatory long tails | “Visible vs billed token economics” page with model/version citations and usage scenarios | Med | Low–Med | Med |
+   113→
+   114→### Six-month SEO execution plan
+   115→
+   116→| Month | Deliverable | Success metric |
+   117→|---:|---|---|
+   118→| 1 | Establish data schema, canonical URL structure, calculator deep links, JSON-LD, citation/provenance component, sitemap | Every page has model/version/GPU/engine/date assumptions; no duplicate programmatic pages |
+   119→| 1–2 | Publish 10 “capacity problem” pages: concurrent users, long context, KV cache, prompt caching, continuous batching, TTFT/ITL | Impressions and backlinks from technical communities |
+   120→| 2–3 | Publish 15 engine × hardware × model workload comparisons | Ranked long-tail queries; calculator scenario starts |
+   121→| 3–4 | Publish self-host vs API and provider/GPU cost pages with regional timestamped price feeds | Organic conversion into saved/shared plans |
+   122→| 4–6 | Refresh price/model pages automatically; turn validated benchmark inputs into study pages | Click-through rate, referring domains, GitHub stars, recurring usage |
+   123→| Continuous | Use Search Console to identify actual query variants and create only pages with distinct user intent | Avoid low-value scaled-content risk |
+   124→
+   125→### SEO data system to build
+   126→
+   127→| Field | Why it matters |
+   128→|---|---|
+   129→| Exact keyword + locale + language + device | “H100 vs H200” demand varies materially by country and device |
+   130→| Tool/source + retrieval date | Keyword volumes and difficulty are estimates, not facts |
+   131→| Intent label | Informational, comparison, purchase-adjacent, calculator |
+   132→| SERP composition | Official docs vs forums vs commercial tools determines realistic competitiveness |
+   133→| Target page + scenario | Every keyword should map to an actual unique calculator use case |
+   134→| Freshness SLA | Pricing: daily/weekly; model releases: on release; hardware specs: on official update |
+   135→| Conversion event | Copy scenario, export report, GitHub star, mailing-list opt-in, integration request |
+   136→
+   137→## C. Trends to prepare for
+   138→
+   139→### High-confidence product trends
+   140→
+   141→| Trend | What is happening | Timing | Tokcalc feature | Priority | Feasibility | Effort | Expected ROI |
+   142→|---|---|---|---|---|---|---|---|
+   143→| Prefill/decode disaggregation | Production serving is moving beyond a single monolithic server. A 2026 serving analysis described SGLang production-scale P/D on 96 H100s, the vLLM llm-d project’s Kubernetes-native disaggregation, and NVIDIA Dynamo integration across major engines.  [fish](https://fish.audio/blog/open-source-llm-inference-engines-2026/) | Now; accelerating through 2026 | Add a **P/D architecture planner**: separate prefill and decode pools, interconnect requirement, KV-transfer overhead, queueing, topology, and cost model | Must | High | High | Very high |
+   144→| KV-cache / prefix-cache economics | Serving advantage is shifting toward cache reuse, routing, memory layout, and long-context scheduler behavior; the ecosystem is explicitly emphasizing prefix reuse and cache routing.  [linkedin](https://www.linkedin.com/pulse/kv-cache-eats-scheduler-sanchit-monga-mfpkc) | Now | Add **cache-aware capacity**: prefix-sharing distribution, cache hit rate, cache retention, multi-turn/agent trace models, cache storage cost, warm/cold TTFT | Must | High | Med–High | Very high |
+   145→| Speculative decoding | Analyses of the 2026 ecosystem describe speculative decoding as widespread and report latency reductions, but exact multipliers depend heavily on model/draft model/workload and must not be universalized.  [baeseokjae.github](https://baeseokjae.github.io/posts/local-ai-model-serving-frameworks-2026/) | Now; default-like in many optimized stacks | Add draft-model selector, acceptance-rate sensitivity, speculative overhead, TTFT vs ITL effects, and engine support matrix | Must | High | Med | High |
+   146→| Engine divergence by workload | vLLM and SGLang remain central; comparisons emphasize that throughput leadership varies with workload, structured output, prefix reuse, and operational maturity rather than a single winner.  [devopsbeast](https://devopsbeast.com/blog/vllm-vs-sglang-production-2026) | Now | Add **engine selector** driven by workload shape: shared prefixes, JSON constraint, agent turns, batch profile, hardware, and operational maturity | Must | High | Med | Very high |
+   147→| Multimodal serving | vLLM-Omni and SGLang-Omni show serving scope expanding from text into audio, vision, video, TTS, and diffusion. vLLM-Omni’s August 2026 notes describe broader production-ready coverage across TTS, speech, diffusion, image/video, and robot-policy serving; SGLang-Omni had a v0.1.6 release on 17 September 2026.  [github](https://github.com/vllm-project/vllm-omni) | Now through 12 months | Extend workload schema beyond text tokens: image patches, audio seconds, video frames, modality-specific prefill, streaming output, and KV/memory accounting | Nice | Med | High | High |
+   148→| Blackwell pricing / availability tracking | Public B200 price claims vary dramatically by provider and packaging. One June comparison reported \$3.70–\$14/GPU-hour, with AWS p6-B200 shown at \$14.24/GPU-hour in an eight-GPU shape; another source showed different values. Treat public-price data as region/SKU/date-specific.  [beam](https://www.beam.cloud/blog/nvidia-b200-pricing) | Available in major-cloud forms; price discovery remains fragmented | Build a **versioned cloud SKU price tracker** with region, VM shape, min GPU count, on-demand/spot/reserved, retrieval timestamp, and source URL | Must | Med | High | Very high |
+   149→| AMD MI400 / MI455X | Public sources conflict on architecture naming and launch specifics. One report says MI400 launched July 2026 and describes MI455X/Helios; earlier reporting cited 432GB HBM4 and 19.6TB/s. Confirm only against AMD primary sources before publishing product specs.  [whatledto](https://whatledto.com/events/amd-instinct-ai-gpus) | 2026 generation; exact cloud availability/pricing must be source-verified | Add MI400-series schema now, mark data as “pending official cloud SKU”; include ROCm/engine maturity and topology considerations | Nice | Med | Med | Med–High |
+   150→| Apple serving / Metal | vLLM Metal is active: its repository notes M5 NAX tensor-unit acceleration for MHA/GQA/MQA prefill in August 2026 and a default unified paged varlen Metal attention backend.  [github](https://github.com/vllm-project/vllm-metal) | Now | Add Apple Silicon engine support, unified-memory planning, local deployment vs cloud cost, MLX/llama.cpp/vLLM-Metal selection | Nice | High | Med | Med |
+   151→| MCP distribution | MCP is becoming a broadly distributed interface layer; a 2026 report cited large npm activity for the SDK, though exact figures must be refreshed before use externally.  [buttondown](https://buttondown.com/Builder-Radar/archive/builder-radar-week-of-august-2-2026/) | Now | Ship a read-only MCP server plus an embeddable widget/API, so coding agents and internal engineering assistants can call tokcalc during design reviews | Must | Med | Med | Very high |
+   152→| Energy / carbon disclosure | Regulatory requirements should be handled with legal review. The user’s named EU AI Act and California SB 53 are relevant policy areas, but exact applicability, dates, thresholds, and obligations must be verified against primary legal/regulatory materials before making compliance claims | 2026 onward; jurisdiction-specific | Add an **optional carbon / energy estimator** with clear assumptions, region electricity intensity source, utilization, and a “not legal advice / not compliance determination” label | Future | Med | High | Med |
+   153→
+   154→### Low-confidence or unverified roadmap items
+   155→
+   156→| Requested item | Research status as of 20 Sep 2026 | Tokcalc response | Priority |
+   157→|---|---|---|---|
+   158→| NVIDIA B100 / B300 cloud availability | The public results retrieved support B200 cloud availability/pricing discussion but do not provide a reliable, primary-source cross-cloud date/pricing matrix for B100/B300 | Do not publish unsupported dates. Create hardware records only after official AWS/GCP/Azure product/pricing pages exist; track “announced,” “limited,” “GA,” and “price available” separately | Nice |
+   159→| DeepSeek V4 / R2 schedule | No reliable primary-source schedule retrieved | Do not create prediction pages. Add a “rumor/unconfirmed” exclusion rule to catalog governance | Future |
+   160→| Llama 4 Behemoth GA / Scout 2 | No reliable release schedule retrieved | Maintain release watchlist; publish only released model variants and official configurations | Future |