Skip to content

Tools / LLM GPU calculator / GLM-5.3

GLM-5.3 VRAM requirements

GPU memory for the weights and KV cache at FP16, FP8 and 4-bit, and how many GPUs that takes. Figures from the model's config.json.

Size your own scenario

Short answer

GLM-5.3 needs about 1,654 GB of GPU memory at FP16/BF16, 827 GB at FP8 and 455 GB at 4-bit to serve one user at an 8K-token context. Its 744B parameters take 1,488 GB at 16-bit, and that conversation adds 0.7 GB of KV cache. It is a mixture-of-experts model: 40B parameters are active per token, but all 744B must be in GPU memory. On NVIDIA H100 80 GB GPUs that is more than one 8-GPU server (at least 21 GPUs) at FP16 and more than one 8-GPU server (at least 11 GPUs) at FP8; for a desktop, at 4-bit it needs two 512 GB Mac Studios with a layer split.

Updated . KV cache at FP16, plus 10% headroom for activations and the runtime.

GPU memory by precision

WeightsWeight memory1 user, 8K context32 users, 8K context1 user, 128K context
FP16/BF161,488 GB1,654 GB1,680 GB1,666 GB
FP8744 GB827 GB853 GB840 GB
4-bit (INT4)409.2 GB455 GB481 GB468 GB

Required memory = (weights + KV cache) ÷ 0.9. An FP8 KV cache halves the cache share; set it in the calculator.

How many GPUs for one user at 8K context

GPUMemoryFP16/BF16FP84-bit (INT4)
Data center · tensor parallel
NVIDIA L40S48 GBMulti-serverMulti-serverMulti-server
NVIDIA RTX PRO 6000 Blackwell Server Edition96 GBMulti-serverMulti-server8 GPUs · TP8
NVIDIA H100 SXM80 GBMulti-serverMulti-server8 GPUs · TP8
NVIDIA H100 NVL94 GBMulti-serverMulti-server8 GPUs · TP8
NVIDIA H200141 GBMulti-server8 GPUs · TP84 GPUs · TP4
NVIDIA B200180 GBMulti-server8 GPUs · TP84 GPUs · TP4
NVIDIA B300288 GB8 GPUs · TP84 GPUs · TP42 GPUs · TP2
AMD Instinct MI300X192 GBMulti-server8 GPUs · TP84 GPUs · TP4
AMD Instinct MI355X288 GB8 GPUs · TP84 GPUs · TP42 GPUs · TP2
Workstation & consumer · layer split
NVIDIA RTX PRO 6000 Blackwell Workstation Edition96 GB18 cards9 cards5 cards
NVIDIA GeForce RTX 309024 GB69 cards35 cards19 cards
NVIDIA GeForce RTX 409024 GB69 cards35 cards19 cards
NVIDIA GeForce RTX 509032 GB52 cards26 cards15 cards
Apple Mac Studio M3 Ultra / M5 Ultra96 GB (72 usable)23 Macs12 Macs7 Macs
Apple Mac Studio M3 Ultra / M5 Ultra256 GB (192 usable)9 Macs5 Macs3 Macs
Apple Mac Studio M5 Ultra512 GB (384 usable)5 Macs3 Macs2 Macs

Data center GPUs are counted at a tensor-parallel size (TP) that divides the model’s attention heads, within one 8-GPU server. Workstation and consumer cards are counted for a layer split, which works with any number of cards. Mac Studio counts about 75% of unified memory as usable by the GPU. Capacity only: speed depends on memory bandwidth and the serving stack.

Memory is step one. Throughput, latency, power and the network decide the real build.

Size my actual workload

GLM-5.3 architecture

Parameters
744B total, 40B active per token (mixture of experts)
Layers with a KV cache
78
Attention heads / KV heads
64 / 64
Head dimension
192
Attention
Multi-head latent attention (latent 512 + RoPE key 64)
Maximum context
1,048,576 tokens
KV cache per token (FP16)
78 layers × (512 latent + 64 RoPE key) × 2 bytes = 89,856 bytes (0.09 MB) per token

Source: zai-org/GLM-5.3 config.json on Hugging Face. Parameter counts are the Hub’s count of the repo’s safetensors files, or the model card’s figure.

Questions about GLM-5.3

How much VRAM does GLM-5.3 need?

About 1,654 GB at FP16/BF16, 827 GB at FP8 and 455 GB at 4-bit, for one user at an 8K-token context. That is 1,488 GB, 744 GB or 409.2 GB of weights, plus 0.7 GB of KV cache, plus 10% headroom for activations and the runtime. Serving 32 users at 8K each takes 1,680 GB at FP16 or 853 GB at FP8, because every conversation keeps its own KV cache.

How many H100 GPUs does GLM-5.3 need?

At FP16/BF16 it takes more than one 8-GPU server (at least 21 GPUs), for one user at 8K or for 32 users. At FP8, more than one 8-GPU server (at least 11 GPUs), for one user at 8K or for 32 users. One user at FP8 takes 8 GPUs (tensor parallel 8) on H200 (141 GB) and 8 GPUs (tensor parallel 8) on B200 (180 GB).

Can GLM-5.3 run on an RTX 4090 or RTX 5090?

At 4-bit it needs about 455 GB for one user at 8K: 19 RTX 4090s or 15 RTX 5090s, more cards than one workstation holds. At FP8 (827 GB) it takes 35 RTX 4090s or 26 RTX 5090s, more cards than one workstation holds. Several cards work with a layer split, as in llama.cpp, which divides the model's layers between them.

Can GLM-5.3 run on a Mac Studio?

macOS lets the GPU use about 75% of unified memory by default: about 72 GB of a 96 GB, 192 GB of a 256 GB and 384 GB of a 512 GB Mac Studio. At 4-bit GLM-5.3 needs 455 GB for one user at 8K, so it needs two 512 GB Mac Studios with a layer split; at FP8 (827 GB) it needs three 512 GB Mac Studios with a layer split.

How large is GLM-5.3's KV cache?

At FP16, 78 layers × (512 latent + 64 RoPE key) × 2 bytes = 89,856 bytes (0.09 MB) per token. Multi-head latent attention caches one compressed latent per layer instead of keys and values for each of its 64 heads. An 8K-token conversation holds 0.7 GB at FP16, and a 128K-token one holds 12 GB.

Does GLM-5.3 need memory for all its experts?

Yes. GLM-5.3 is a mixture-of-experts model with 744B parameters, of which about 40B are active for each token. Any token can be routed to any expert, so all of them must be in GPU memory: memory follows total parameters, speed follows active ones.

What is GLM-5.3's maximum context length?

1,048,576 tokens, according to its Hugging Face configuration. One user at that full length holds 94 GB of KV cache at FP16, for 931 GB in total with FP8 weights, or 879 GB with an FP8 KV cache as well.

Other models

Not listed? The LLM GPU calculator sizes any Hugging Face model from its ID, link or config.json.

Need to deploy private AI?

We design it, validate it, source it, deploy it and help run it.

Get my 48-hour blueprint

Give us your workload, users, budget and timeline. Within 48 hours we'll give you a validated cloud/on-prem architecture, BOM, expected costs and sourcing options.