Skip to content

Tools / LLM GPU calculator / H200

What LLMs fit on an NVIDIA H200?

Which of 22 open LLMs fit one H200, and how many it takes for the rest, at FP16, FP8 and 4-bit.

Size your own scenario

Short answer

One NVIDIA H200 (141 GB) runs 17 of the 22 open LLMs sized here at 4-bit, 17 at FP8 and 12 at FP16, for one user at an 8K context. At FP8: every model except Llama 3.1 405B, Llama 4 Maverick, Qwen3-235B-A22B, DeepSeek-V3 / R1 and GLM-5.3. The largest that fits one GPU is gpt-oss-120b (116.8B parameters), at FP8. Within one 8-GPU server, tensor parallelism runs all of them at FP8. Llama 3.3 70B takes 2 GPUs (tensor parallel 2) at FP16 and 1 GPU at FP8.

Updated . One user, 8K context, KV cache at FP16, plus 10% headroom for activations and the runtime.

Every model on the H200

ModelFP16/BF16FP84-bit (INT4)
Llama 3.1 8B1 GPU19 GB1 GPU10 GB1 GPU6.1 GB
Qwen3 8B1 GPU20 GB1 GPU10 GB1 GPU6.3 GB
Qwen3 14B1 GPU34 GB1 GPU18 GB1 GPU11 GB
Phi-4 14B1 GPU35 GB1 GPU18 GB1 GPU11 GB
gpt-oss-20b1 GPU47 GB1 GPU23 GB1 GPU13 GB
LFM2.5-24B-A2B1 GPU54 GB1 GPU27 GB1 GPU15 GB
Mistral Small 3.2 24B1 GPU55 GB1 GPU28 GB1 GPU16 GB
Gemma 3 27B1 GPU62 GB1 GPU32 GB1 GPU18 GB
Qwen3-30B-A3B1 GPU69 GB1 GPU35 GB1 GPU20 GB
Qwen3 32B1 GPU75 GB1 GPU39 GB1 GPU22 GB
Qwen2.5 32B1 GPU75 GB1 GPU39 GB1 GPU22 GB
Mixtral 8x7B1 GPU105 GB1 GPU53 GB1 GPU30 GB
Llama 3.3 70B2 GPUs · TP2160 GB1 GPU81 GB1 GPU46 GB
Qwen2.5 72B2 GPUs · TP2165 GB1 GPU84 GB1 GPU47 GB
Kolibri-12 GPUs · TP2174 GB1 GPU87 GB1 GPU48 GB
Llama 4 Scout2 GPUs · TP2243 GB1 GPU122 GB1 GPU68 GB
gpt-oss-120b2 GPUs · TP2260 GB1 GPU130 GB1 GPU72 GB
Qwen3-235B-A22B4 GPUs · TP4524 GB2 GPUs · TP2263 GB2 GPUs · TP2145 GB
Llama 4 Maverick8 GPUs · TP8894 GB4 GPUs · TP4448 GB2 GPUs · TP2247 GB
Llama 3.1 405B8 GPUs · TP8907 GB4 GPUs · TP4456 GB2 GPUs · TP2253 GB
DeepSeek-V3 / R1Multi-server1,492 GB8 GPUs · TP8746 GB4 GPUs · TP4411 GB
GLM-5.3Multi-server1,654 GB8 GPUs · TP8827 GB4 GPUs · TP4455 GB

Counts use a tensor-parallel size (TP) that divides the model's attention heads, within one 8-GPU server. Below each count is the memory needed: (weights + KV cache) ÷ 0.9. Capacity only: speed depends on memory bandwidth and the serving stack.

Memory is step one. Throughput, latency, power and the network decide the real build.

Size my actual workload

H200 specifications

Memory
141 GB HBM3e
Memory bandwidth
4,800 GB/s
Typical system
8-GPU HGX or PCIe (NVL) server
Counted as
Tensor parallel within one 8-GPU server (vLLM style)

Configurations and pricing: NVIDIA H200 in the catalog.

Questions about the H200

What LLMs can run on an NVIDIA H200?

For one user at an 8K context, these fit one H200. At 4-bit: every model except Llama 3.1 405B, Llama 4 Maverick, Qwen3-235B-A22B, DeepSeek-V3 / R1 and GLM-5.3. At FP8: every model except Llama 3.1 405B, Llama 4 Maverick, Qwen3-235B-A22B, DeepSeek-V3 / R1 and GLM-5.3. At FP16/BF16: every model except Llama 3.3 70B, Llama 3.1 405B, Llama 4 Scout, Llama 4 Maverick, Qwen3-235B-A22B, Qwen2.5 72B, DeepSeek-V3 / R1, gpt-oss-120b, GLM-5.3 and Kolibri-1.

What is the largest LLM an H200 can run?

Of the models sized here, for one user at an 8K context: at 4-bit, gpt-oss-120b (116.8B parameters, 72 GB); at FP8, gpt-oss-120b (116.8B parameters, 130 GB); at FP16, Mixtral 8x7B (46.7B parameters, 105 GB). Mixture-of-experts models count every expert, because all of them must be in memory.

How many NVIDIA H200 GPUs does Llama 3.3 70B need?

For one user at an 8K context it needs 160 GB at FP16/BF16, 81 GB at FP8 and 46 GB at 4-bit: 2 GPUs (tensor parallel 2) at FP16 and 1 GPU at FP8 or 4-bit.

Can an NVIDIA H200 run DeepSeek-V3 / R1?

Not on one GPU: with all 671B parameters resident it needs 746 GB at FP8 and 411 GB at 4-bit for one user at an 8K context. That is 8 GPUs (tensor parallel 8) at FP8 and 4 GPUs (tensor parallel 4) at 4-bit.

Other GPUs

Different context length, more users, or a model that isn’t listed? The LLM GPU calculator sizes any Hugging Face model for any of these GPUs.

Need to deploy private AI?

We design it, validate it, source it, deploy it and help run it.

Get my 48-hour blueprint

Give us your workload, users, budget and timeline. Within 48 hours we'll give you a validated cloud/on-prem architecture, BOM, expected costs and sourcing options.