Skip to content

Tools / LLM GPU calculator / RTX 5090

What LLMs can run on an RTX 5090?

Which of 22 open LLMs fit one RTX 5090, and how many it takes for the rest, at FP16, FP8 and 4-bit.

Size your own scenario

Short answer

One RTX 5090 (32 GB) runs 12 of the 22 open LLMs sized here at 4-bit, 8 at FP8 and 2 at FP16, for one user at an 8K context. At 4-bit: every model except Llama 3.3 70B, Llama 3.1 405B, Llama 4 Scout, Llama 4 Maverick, Qwen3-235B-A22B, Qwen2.5 72B, DeepSeek-V3 / R1, gpt-oss-120b, GLM-5.3 and Kolibri-1. The largest that fits one card is Mixtral 8x7B (46.7B parameters), at 4-bit. Bigger models split their layers across several cards: Llama 3.3 70B takes 2 at 4-bit.

Updated . One user, 8K context, KV cache at FP16, plus 10% headroom for activations and the runtime.

Every model on the RTX 5090

ModelFP16/BF16FP84-bit (INT4)
Llama 3.1 8B1 card19 GB1 card10 GB1 card6.1 GB
Qwen3 8B1 card20 GB1 card10 GB1 card6.3 GB
Qwen3 14B2 cards34 GB1 card18 GB1 card11 GB
Phi-4 14B2 cards35 GB1 card18 GB1 card11 GB
gpt-oss-20b2 cards47 GB1 card23 GB1 card13 GB
LFM2.5-24B-A2B2 cards54 GB1 card27 GB1 card15 GB
Mistral Small 3.2 24B2 cards55 GB1 card28 GB1 card16 GB
Gemma 3 27B2 cards62 GB1 card32 GB1 card18 GB
Qwen3-30B-A3B3 cards69 GB2 cards35 GB1 card20 GB
Qwen3 32B3 cards75 GB2 cards39 GB1 card22 GB
Qwen2.5 32B3 cards75 GB2 cards39 GB1 card22 GB
Mixtral 8x7B4 cards105 GB2 cards53 GB1 card30 GB
Llama 3.3 70B5 cards160 GB3 cards81 GB2 cards46 GB
Qwen2.5 72B6 cards165 GB3 cards84 GB2 cards47 GB
Kolibri-16 cards174 GB3 cards87 GB2 cards48 GB
Llama 4 Scout8 cards243 GB4 cards122 GB3 cards68 GB
gpt-oss-120b9 cards260 GB5 cards130 GB3 cards72 GB
Qwen3-235B-A22B17 cards524 GB9 cards263 GB5 cards145 GB
Llama 4 Maverick28 cards894 GB15 cards448 GB8 cards247 GB
Llama 3.1 405B29 cards907 GB15 cards456 GB8 cards253 GB
DeepSeek-V3 / R147 cards1,492 GB24 cards746 GB13 cards411 GB
GLM-5.352 cards1,654 GB26 cards827 GB15 cards455 GB

Counts are for a layer split, which works with any number of cards. Below each count is the memory needed: (weights + KV cache) ÷ 0.9. Capacity only: speed depends on memory bandwidth and the serving stack.

Memory is step one. Throughput, latency, power and the network decide the real build.

Size my actual workload

RTX 5090 specifications

Memory
32 GB GDDR7
Memory bandwidth
1,792 GB/s
Typical system
Desktop PC
Counted as
Layer split across cards (llama.cpp style)

Questions about the RTX 5090

What LLMs can run on an RTX 5090?

For one user at an 8K context, these fit one RTX 5090. At 4-bit: every model except Llama 3.3 70B, Llama 3.1 405B, Llama 4 Scout, Llama 4 Maverick, Qwen3-235B-A22B, Qwen2.5 72B, DeepSeek-V3 / R1, gpt-oss-120b, GLM-5.3 and Kolibri-1. At FP8: Llama 3.1 8B, Qwen3 8B, Qwen3 14B, Phi-4 14B, gpt-oss-20b, LFM2.5-24B-A2B, Mistral Small 3.2 24B and Gemma 3 27B. At FP16/BF16: Llama 3.1 8B and Qwen3 8B.

What is the largest LLM an RTX 5090 can run?

Of the models sized here, for one user at an 8K context: at 4-bit, Mixtral 8x7B (46.7B parameters, 30 GB); at FP8, Gemma 3 27B (27.4B parameters, 32 GB); at FP16, Qwen3 8B (8.19B parameters, 20 GB). Mixture-of-experts models count every expert, because all of them must be in memory.

How many RTX 5090 cards does Llama 3.3 70B need?

For one user at an 8K context it needs 160 GB at FP16/BF16, 81 GB at FP8 and 46 GB at 4-bit: 5 cards at FP16, 3 cards at FP8 and 2 cards at 4-bit, with a layer split.

Can an RTX 5090 run DeepSeek-V3 / R1?

Not on one card: with all 671B parameters resident it needs 746 GB at FP8 and 411 GB at 4-bit for one user at an 8K context. That is 24 cards at FP8 and 13 cards at 4-bit, with a layer split across more cards than one workstation holds.

Can I combine several RTX 5090 cards?

Yes, with a layer split: each card holds some of the model's layers, as llama.cpp does. Memory adds up across them; speed does not, because they take turns on each token. The counts on this page assume a layer split.

Other GPUs

Different context length, more users, or a model that isn’t listed? The LLM GPU calculator sizes any Hugging Face model for any of these GPUs.

Need to deploy private AI?

We design it, validate it, source it, deploy it and help run it.

Get my 48-hour blueprint

Give us your workload, users, budget and timeline. Within 48 hours we'll give you a validated cloud/on-prem architecture, BOM, expected costs and sourcing options.