Tools / LLM GPU calculator / Llama 4 Maverick
Llama 4 Maverick VRAM requirements
GPU memory for the weights and KV cache at FP16, FP8 and 4-bit, and how many GPUs that takes. Figures from the model's config.json.
Short answer
Llama 4 Maverick needs about 894 GB of GPU memory at FP16/BF16, 448 GB at FP8 and 247 GB at 4-bit to serve one user at an 8K-token context. Its 401.6B parameters take 803.2 GB at 16-bit, and that conversation adds 1.6 GB of KV cache. It is a mixture-of-experts model: 17B parameters are active per token, but all 401.6B must be in GPU memory. On NVIDIA H100 80 GB GPUs that is more than one 8-GPU server (at least 12 GPUs) at FP16 and 8 GPUs (tensor parallel 8) at FP8; for a desktop, at 4-bit it needs one 512 GB Mac Studio.
Updated . KV cache at FP16, plus 10% headroom for activations and the runtime.
GPU memory by precision
| Weights | Weight memory | 1 user, 8K context | 32 users, 8K context | 1 user, 128K context |
|---|---|---|---|---|
| FP16/BF16 | 803.2 GB | 894 GB | 950 GB | 901 GB |
| FP8 | 401.6 GB | 448 GB | 503 GB | 455 GB |
| 4-bit (INT4) | 220.9 GB | 247 GB | 303 GB | 254 GB |
Required memory = (weights + KV cache) ÷ 0.9. An FP8 KV cache halves the cache share; set it in the calculator.
How many GPUs for one user at 8K context
| GPU | Memory | FP16/BF16 | FP8 | 4-bit (INT4) |
|---|---|---|---|---|
| Data center · tensor parallel | ||||
| NVIDIA L40S | 48 GB | Multi-server | Multi-server | 8 GPUs · TP8 |
| NVIDIA RTX PRO 6000 Blackwell Server Edition | 96 GB | Multi-server | 8 GPUs · TP8 | 4 GPUs · TP4 |
| NVIDIA H100 SXM | 80 GB | Multi-server | 8 GPUs · TP8 | 4 GPUs · TP4 |
| NVIDIA H100 NVL | 94 GB | Multi-server | 8 GPUs · TP8 | 4 GPUs · TP4 |
| NVIDIA H200 | 141 GB | 8 GPUs · TP8 | 4 GPUs · TP4 | 2 GPUs · TP2 |
| NVIDIA B200 | 180 GB | 8 GPUs · TP8 | 4 GPUs · TP4 | 2 GPUs · TP2 |
| NVIDIA B300 | 288 GB | 4 GPUs · TP4 | 2 GPUs · TP2 | 1 GPU |
| AMD Instinct MI300X | 192 GB | 8 GPUs · TP8 | 4 GPUs · TP4 | 2 GPUs · TP2 |
| AMD Instinct MI355X | 288 GB | 4 GPUs · TP4 | 2 GPUs · TP2 | 1 GPU |
| Workstation & consumer · layer split | ||||
| NVIDIA RTX PRO 6000 Blackwell Workstation Edition | 96 GB | 10 cards | 5 cards | 3 cards |
| NVIDIA GeForce RTX 3090 | 24 GB | 38 cards | 19 cards | 11 cards |
| NVIDIA GeForce RTX 4090 | 24 GB | 38 cards | 19 cards | 11 cards |
| NVIDIA GeForce RTX 5090 | 32 GB | 28 cards | 15 cards | 8 cards |
| Apple Mac Studio M3 Ultra / M5 Ultra | 96 GB (72 usable) | 13 Macs | 7 Macs | 4 Macs |
| Apple Mac Studio M3 Ultra / M5 Ultra | 256 GB (192 usable) | 5 Macs | 3 Macs | 2 Macs |
| Apple Mac Studio M5 Ultra | 512 GB (384 usable) | 3 Macs | 2 Macs | 1 Mac |
Data center GPUs are counted at a tensor-parallel size (TP) that divides the model’s attention heads, within one 8-GPU server. Workstation and consumer cards are counted for a layer split, which works with any number of cards. Mac Studio counts about 75% of unified memory as usable by the GPU. Capacity only: speed depends on memory bandwidth and the serving stack.
Memory is step one. Throughput, latency, power and the network decide the real build.
Size my actual workloadLlama 4 Maverick architecture
- Parameters
- 401.6B total, 17B active per token (mixture of experts)
- Layers with a KV cache
- 48
- Attention heads / KV heads
- 40 / 8
- Head dimension
- 128
- Attention
- 36 chunked layers (8,192 tokens) + 12 global
- Maximum context
- 1,048,576 tokens
- KV cache per token (FP16)
- 2 × 12 global layers × 8 KV heads × 128 head dim × 2 bytes = 49,152 bytes (0.05 MB) per token, plus 4,096 bytes per token in each of the 36 chunked-attention layers, for at most 8,192 tokens
Source: meta-llama/Llama-4-Maverick-17B-128E-Instruct config.json on Hugging Face. Parameter counts are the Hub’s count of the repo’s safetensors files, or the model card’s figure.
Questions about Llama 4 Maverick
How much VRAM does Llama 4 Maverick need?
About 894 GB at FP16/BF16, 448 GB at FP8 and 247 GB at 4-bit, for one user at an 8K-token context. That is 803.2 GB, 401.6 GB or 220.9 GB of weights, plus 1.6 GB of KV cache, plus 10% headroom for activations and the runtime. Serving 32 users at 8K each takes 950 GB at FP16 or 503 GB at FP8, because every conversation keeps its own KV cache.
How many H100 GPUs does Llama 4 Maverick need?
At FP16/BF16 it takes more than one 8-GPU server (at least 12 GPUs), for one user at 8K or for 32 users. At FP8, 8 GPUs (tensor parallel 8), for one user at 8K or for 32 users. One user at FP8 takes 4 GPUs (tensor parallel 4) on H200 (141 GB) and 4 GPUs (tensor parallel 4) on B200 (180 GB).
Can Llama 4 Maverick run on an RTX 4090 or RTX 5090?
At 4-bit it needs about 247 GB for one user at 8K: 11 RTX 4090s or 8 RTX 5090s. At FP8 (448 GB) it takes 19 RTX 4090s or 15 RTX 5090s, more cards than one workstation holds. Several cards work with a layer split, as in llama.cpp, which divides the model's layers between them.
Can Llama 4 Maverick run on a Mac Studio?
macOS lets the GPU use about 75% of unified memory by default: about 72 GB of a 96 GB, 192 GB of a 256 GB and 384 GB of a 512 GB Mac Studio. At 4-bit Llama 4 Maverick needs 247 GB for one user at 8K, so it fits one 512 GB Mac Studio; at FP8 (448 GB) it needs two 512 GB Mac Studios with a layer split.
How large is Llama 4 Maverick's KV cache?
36 of its 48 layers use chunked attention and cache at most 8,192 tokens; only the other 12 keep the whole context. At FP16 that is 2 × 12 global layers × 8 KV heads × 128 head dim × 2 bytes = 49,152 bytes (0.05 MB) per token, plus 4,096 bytes per token in each of the 36 chunked-attention layers, for at most 8,192 tokens. An 8K-token conversation holds 1.6 GB at FP16, and a 128K-token one holds 7.7 GB.
Does Llama 4 Maverick need memory for all its experts?
Yes. Llama 4 Maverick is a mixture-of-experts model with 401.6B parameters, of which about 17B are active for each token. Any token can be routed to any expert, so all of them must be in GPU memory: memory follows total parameters, speed follows active ones.
What is Llama 4 Maverick's maximum context length?
1,048,576 tokens, according to its Hugging Face configuration. One user at that full length holds 53 GB of KV cache at FP16, for 505 GB in total with FP8 weights, or 476 GB with an FP8 KV cache as well.
Other models
- Llama 3.1 8B
- Llama 3.3 70B
- Llama 3.1 405B
- Llama 4 Scout
- Qwen3 8B
- Qwen3 14B
- Qwen3 32B
- Qwen3-30B-A3B
- Qwen3-235B-A22B
- Qwen2.5 32B
- Qwen2.5 72B
- DeepSeek-V3 / R1
- gpt-oss-20b
- gpt-oss-120b
- Gemma 3 27B
- Mistral Small 3.2 24B
- Mixtral 8x7B
- Phi-4 14B
- GLM-5.3
- LFM2.5-24B-A2B
- Kolibri-1
Not listed? The LLM GPU calculator sizes any Hugging Face model from its ID, link or config.json.
Need to deploy private AI?
We design it, validate it, source it, deploy it and help run it.
Give us your workload, users, budget and timeline. Within 48 hours we'll give you a validated cloud/on-prem architecture, BOM, expected costs and sourcing options.