Skip to content

Tools / LLM GPU calculator / Qwen3-235B-A22B

Qwen3-235B-A22B VRAM requirements

GPU memory for the weights and KV cache at FP16, FP8 and 4-bit, and how many GPUs that takes. Figures from the model's config.json.

Size your own scenario

Short answer

Qwen3-235B-A22B needs about 524 GB of GPU memory at FP16/BF16, 263 GB at FP8 and 145 GB at 4-bit to serve one user at an 8K-token context. Its 235.1B parameters take 470.2 GB at 16-bit, and that conversation adds 1.6 GB of KV cache. It is a mixture-of-experts model: 22B parameters are active per token, but all 235.1B must be in GPU memory. On NVIDIA H100 80 GB GPUs that is more than one server, since no tensor-parallel size within 8 GPUs divides its attention heads at FP16 and 4 GPUs (tensor parallel 4) at FP8; for a desktop, at 4-bit it needs one 256 GB Mac Studio.

Updated . KV cache at FP16, plus 10% headroom for activations and the runtime.

GPU memory by precision

WeightsWeight memory1 user, 8K context32 users, 8K context1 user, 40K context
FP16/BF16470.2 GB524 GB579 GB531 GB
FP8235.1 GB263 GB317 GB270 GB
4-bit (INT4)129.3 GB145 GB200 GB152 GB

Required memory = (weights + KV cache) ÷ 0.9. An FP8 KV cache halves the cache share; set it in the calculator.

How many GPUs for one user at 8K context

GPUMemoryFP16/BF16FP84-bit (INT4)
Data center · tensor parallel
NVIDIA L40S48 GBMulti-serverMulti-server4 GPUs · TP4
NVIDIA RTX PRO 6000 Blackwell Server Edition96 GBMulti-server4 GPUs · TP42 GPUs · TP2
NVIDIA H100 SXM80 GBMulti-server4 GPUs · TP42 GPUs · TP2
NVIDIA H100 NVL94 GBMulti-server4 GPUs · TP42 GPUs · TP2
NVIDIA H200141 GB4 GPUs · TP42 GPUs · TP22 GPUs · TP2
NVIDIA B200180 GB4 GPUs · TP42 GPUs · TP21 GPU
NVIDIA B300288 GB2 GPUs · TP21 GPU1 GPU
AMD Instinct MI300X192 GB4 GPUs · TP42 GPUs · TP21 GPU
AMD Instinct MI355X288 GB2 GPUs · TP21 GPU1 GPU
Workstation & consumer · layer split
NVIDIA RTX PRO 6000 Blackwell Workstation Edition96 GB6 cards3 cards2 cards
NVIDIA GeForce RTX 309024 GB22 cards11 cards7 cards
NVIDIA GeForce RTX 409024 GB22 cards11 cards7 cards
NVIDIA GeForce RTX 509032 GB17 cards9 cards5 cards
Apple Mac Studio M3 Ultra / M5 Ultra96 GB (72 usable)8 Macs4 Macs3 Macs
Apple Mac Studio M3 Ultra / M5 Ultra256 GB (192 usable)3 Macs2 Macs1 Mac
Apple Mac Studio M5 Ultra512 GB (384 usable)2 Macs1 Mac1 Mac

Data center GPUs are counted at a tensor-parallel size (TP) that divides the model’s attention heads, within one 8-GPU server. Workstation and consumer cards are counted for a layer split, which works with any number of cards. Mac Studio counts about 75% of unified memory as usable by the GPU. Capacity only: speed depends on memory bandwidth and the serving stack.

Memory is step one. Throughput, latency, power and the network decide the real build.

Size my actual workload

Qwen3-235B-A22B architecture

Parameters
235.1B total, 22B active per token (mixture of experts)
Layers with a KV cache
94
Attention heads / KV heads
64 / 4
Head dimension
128
Attention
Grouped-query attention (16 query heads per KV head)
Maximum context
40,960 tokens
KV cache per token (FP16)
2 × 94 layers × 4 KV heads × 128 head dim × 2 bytes = 192,512 bytes (0.19 MB) per token

Source: Qwen/Qwen3-235B-A22B config.json on Hugging Face. Parameter counts are the Hub’s count of the repo’s safetensors files, or the model card’s figure.

Questions about Qwen3-235B-A22B

How much VRAM does Qwen3-235B-A22B need?

About 524 GB at FP16/BF16, 263 GB at FP8 and 145 GB at 4-bit, for one user at an 8K-token context. That is 470.2 GB, 235.1 GB or 129.3 GB of weights, plus 1.6 GB of KV cache, plus 10% headroom for activations and the runtime. Serving 32 users at 8K each takes 579 GB at FP16 or 317 GB at FP8, because every conversation keeps its own KV cache.

How many H100 GPUs does Qwen3-235B-A22B need?

At FP16/BF16 it takes more than one server, since no tensor-parallel size within 8 GPUs divides its attention heads, for one user at 8K or for 32 users. At FP8, 4 GPUs (tensor parallel 4), for one user at 8K or for 32 users. One user at FP8 takes 2 GPUs (tensor parallel 2) on H200 (141 GB) and 2 GPUs (tensor parallel 2) on B200 (180 GB).

Can Qwen3-235B-A22B run on an RTX 4090 or RTX 5090?

At 4-bit it needs about 145 GB for one user at 8K: 7 RTX 4090s or 5 RTX 5090s. At FP8 (263 GB) it takes 11 RTX 4090s or 9 RTX 5090s, more cards than one workstation holds. Several cards work with a layer split, as in llama.cpp, which divides the model's layers between them.

Can Qwen3-235B-A22B run on a Mac Studio?

macOS lets the GPU use about 75% of unified memory by default: about 72 GB of a 96 GB, 192 GB of a 256 GB and 384 GB of a 512 GB Mac Studio. At 4-bit Qwen3-235B-A22B needs 145 GB for one user at 8K, so it fits one 256 GB Mac Studio; at FP8 (263 GB) it fits one 512 GB Mac Studio.

How large is Qwen3-235B-A22B's KV cache?

At FP16, 2 × 94 layers × 4 KV heads × 128 head dim × 2 bytes = 192,512 bytes (0.19 MB) per token. It uses grouped-query attention: 16 query heads share each KV head, which makes the cache 16× smaller than full multi-head attention. An 8K-token conversation holds 1.6 GB at FP16, and a 40K-token one holds 7.9 GB.

Does Qwen3-235B-A22B need memory for all its experts?

Yes. Qwen3-235B-A22B is a mixture-of-experts model with 235.1B parameters, of which about 22B are active for each token. Any token can be routed to any expert, so all of them must be in GPU memory: memory follows total parameters, speed follows active ones.

What is Qwen3-235B-A22B's maximum context length?

40,960 tokens, according to its Hugging Face configuration. One user at that full length holds 7.9 GB of KV cache at FP16, for 270 GB in total with FP8 weights, or 266 GB with an FP8 KV cache as well.

Other models

Not listed? The LLM GPU calculator sizes any Hugging Face model from its ID, link or config.json.

Need to deploy private AI?

We design it, validate it, source it, deploy it and help run it.

Get my 48-hour blueprint

Give us your workload, users, budget and timeline. Within 48 hours we'll give you a validated cloud/on-prem architecture, BOM, expected costs and sourcing options.