Tools / LLM GPU calculator / gpt-oss-120b

gpt-oss-120b VRAM requirements

GPU memory for the weights and KV cache at FP16, FP8 and 4-bit, and how many GPUs that takes. Figures from the model's config.json.

Size your own scenario

Short answer

gpt-oss-120b needs about 260 GB of GPU memory at FP16/BF16, 130 GB at FP8 and 72 GB at 4-bit to serve one user at an 8K-token context. Its 116.8B parameters take 233.6 GB at 16-bit, and that conversation adds 0.3 GB of KV cache. It is a mixture-of-experts model: 5.1B parameters are active per token, but all 116.8B must be in GPU memory. On NVIDIA H100 80 GB GPUs that is 4 GPUs (tensor parallel 4) at FP16 and 2 GPUs (tensor parallel 2) at FP8; for a desktop, at 4-bit it needs 3 RTX 5090s with a layer split, or one 96 GB Mac Studio.

Updated October 16, 2026. KV cache at FP16, plus 10% headroom for activations and the runtime.

GPU memory by precision

Weights	Weight memory	1 user, 8K context	32 users, 8K context	1 user, 128K context
FP16/BF16	233.6 GB	260 GB	270 GB	265 GB
FP8	116.8 GB	130 GB	141 GB	135 GB
4-bit (INT4)	64.2 GB	72 GB	82 GB	77 GB

Required memory = (weights + KV cache) ÷ 0.9. An FP8 KV cache halves the cache share; set it in the calculator.

How many GPUs for one user at 8K context

GPU	Memory	FP16/BF16	FP8	4-bit (INT4)
Data center · tensor parallel
NVIDIA L40S	48 GB	8 GPUs · TP8	4 GPUs · TP4	2 GPUs · TP2
NVIDIA RTX PRO 6000 Blackwell Server Edition	96 GB	4 GPUs · TP4	2 GPUs · TP2	1 GPU
NVIDIA H100 SXM	80 GB	4 GPUs · TP4	2 GPUs · TP2	1 GPU
NVIDIA H100 NVL	94 GB	4 GPUs · TP4	2 GPUs · TP2	1 GPU
NVIDIA H200	141 GB	2 GPUs · TP2	1 GPU	1 GPU
NVIDIA B200	180 GB	2 GPUs · TP2	1 GPU	1 GPU
NVIDIA B300	288 GB	1 GPU	1 GPU	1 GPU
AMD Instinct MI300X	192 GB	2 GPUs · TP2	1 GPU	1 GPU
AMD Instinct MI355X	288 GB	1 GPU	1 GPU	1 GPU
Workstation & consumer · layer split
NVIDIA RTX PRO 6000 Blackwell Workstation Edition	96 GB	3 cards	2 cards	1 card
NVIDIA GeForce RTX 3090	24 GB	11 cards	6 cards	3 cards
NVIDIA GeForce RTX 4090	24 GB	11 cards	6 cards	3 cards
NVIDIA GeForce RTX 5090	32 GB	9 cards	5 cards	3 cards
Apple Mac Studio M3 Ultra / M5 Ultra	96 GB (72 usable)	4 Macs	2 Macs	1 Mac
Apple Mac Studio M3 Ultra / M5 Ultra	256 GB (192 usable)	2 Macs	1 Mac	1 Mac
Apple Mac Studio M5 Ultra	512 GB (384 usable)	1 Mac	1 Mac	1 Mac

Data center GPUs are counted at a tensor-parallel size (TP) that divides the model’s attention heads, within one 8-GPU server. Workstation and consumer cards are counted for a layer split, which works with any number of cards. Mac Studio counts about 75% of unified memory as usable by the GPU. Capacity only: speed depends on memory bandwidth and the serving stack.

Memory is step one. Throughput, latency, power and the network decide the real build.

Size my actual workload

gpt-oss-120b architecture

Parameters: 116.8B total, 5.1B active per token (mixture of experts)
Layers with a KV cache: 36
Attention heads / KV heads: 64 / 8
Head dimension: 64
Attention: 18 sliding-window layers (128 tokens) + 18 global
Maximum context: 131,072 tokens
KV cache per token (FP16): 2 × 18 global layers × 8 KV heads × 64 head dim × 2 bytes = 36,864 bytes (0.04 MB) per token, plus 2,048 bytes per token in each of the 18 sliding-window layers, for at most 128 tokens

Source: openai/gpt-oss-120b config.json on Hugging Face. Parameter counts are the Hub’s count of the repo’s safetensors files, or the model card’s figure.

Questions about gpt-oss-120b

How much VRAM does gpt-oss-120b need?

About 260 GB at FP16/BF16, 130 GB at FP8 and 72 GB at 4-bit, for one user at an 8K-token context. That is 233.6 GB, 116.8 GB or 64.2 GB of weights, plus 0.3 GB of KV cache, plus 10% headroom for activations and the runtime. Serving 32 users at 8K each takes 270 GB at FP16 or 141 GB at FP8, because every conversation keeps its own KV cache.

How many H100 GPUs does gpt-oss-120b need?

At FP16/BF16 it takes 4 GPUs (tensor parallel 4), for one user at 8K or for 32 users. At FP8, 2 GPUs (tensor parallel 2), for one user at 8K or for 32 users. One user at FP8 takes 1 GPU on H200 (141 GB) and 1 GPU on B200 (180 GB).

Can gpt-oss-120b run on an RTX 4090 or RTX 5090?

At 4-bit it needs about 72 GB for one user at 8K: 3 RTX 4090s or 3 RTX 5090s. At FP8 (130 GB) it takes 6 RTX 4090s or 5 RTX 5090s. Several cards work with a layer split, as in llama.cpp, which divides the model's layers between them.

Can gpt-oss-120b run on a Mac Studio?

macOS lets the GPU use about 75% of unified memory by default: about 72 GB of a 96 GB, 192 GB of a 256 GB and 384 GB of a 512 GB Mac Studio. At 4-bit gpt-oss-120b needs 72 GB for one user at 8K, so it fits one 96 GB Mac Studio; at FP8 (130 GB) it fits one 256 GB Mac Studio.

How large is gpt-oss-120b's KV cache?

18 of its 36 layers use a sliding window and cache at most 128 tokens; only the other 18 keep the whole context. At FP16 that is 2 × 18 global layers × 8 KV heads × 64 head dim × 2 bytes = 36,864 bytes (0.04 MB) per token, plus 2,048 bytes per token in each of the 18 sliding-window layers, for at most 128 tokens. An 8K-token conversation holds 0.3 GB at FP16, and a 128K-token one holds 4.8 GB.

Does gpt-oss-120b need memory for all its experts?

Yes. gpt-oss-120b is a mixture-of-experts model with 116.8B parameters, of which about 5.1B are active for each token. Any token can be routed to any expert, so all of them must be in GPU memory: memory follows total parameters, speed follows active ones.

What is gpt-oss-120b's maximum context length?

131,072 tokens, according to its Hugging Face configuration. One user at that full length holds 4.8 GB of KV cache at FP16, for 135 GB in total with FP8 weights, or 132 GB with an FP8 KV cache as well.

Other models

Not listed? The LLM GPU calculator sizes any Hugging Face model from its ID, link or config.json.

Need to deploy private AI?

We design it, validate it, source it, deploy it and help run it.

Get my 48-hour blueprint

Give us your workload, users, budget and timeline. Within 48 hours we'll give you a validated cloud/on-prem architecture, BOM, expected costs and sourcing options.