Skip to content

Tools / LLM GPU calculator / RTX PRO 6000 Workstation

What LLMs can run on an RTX PRO 6000 Workstation?

Which of 22 open LLMs fit one RTX PRO 6000 Workstation, and how many it takes for the rest, at FP16, FP8 and 4-bit.

Size your own scenario

Short answer

One RTX PRO 6000 Workstation (96 GB) runs 17 of the 22 open LLMs sized here at 4-bit, 15 at FP8 and 11 at FP16, for one user at an 8K context. At 4-bit: every model except Llama 3.1 405B, Llama 4 Maverick, Qwen3-235B-A22B, DeepSeek-V3 / R1 and GLM-5.3. The largest that fits one card is gpt-oss-120b (116.8B parameters), at 4-bit. Bigger models split their layers across several cards: Qwen3-235B-A22B takes 2 at 4-bit.

Updated . One user, 8K context, KV cache at FP16, plus 10% headroom for activations and the runtime.

Every model on the RTX PRO 6000 Workstation

ModelFP16/BF16FP84-bit (INT4)
Llama 3.1 8B1 card19 GB1 card10 GB1 card6.1 GB
Qwen3 8B1 card20 GB1 card10 GB1 card6.3 GB
Qwen3 14B1 card34 GB1 card18 GB1 card11 GB
Phi-4 14B1 card35 GB1 card18 GB1 card11 GB
gpt-oss-20b1 card47 GB1 card23 GB1 card13 GB
LFM2.5-24B-A2B1 card54 GB1 card27 GB1 card15 GB
Mistral Small 3.2 24B1 card55 GB1 card28 GB1 card16 GB
Gemma 3 27B1 card62 GB1 card32 GB1 card18 GB
Qwen3-30B-A3B1 card69 GB1 card35 GB1 card20 GB
Qwen3 32B1 card75 GB1 card39 GB1 card22 GB
Qwen2.5 32B1 card75 GB1 card39 GB1 card22 GB
Mixtral 8x7B2 cards105 GB1 card53 GB1 card30 GB
Llama 3.3 70B2 cards160 GB1 card81 GB1 card46 GB
Qwen2.5 72B2 cards165 GB1 card84 GB1 card47 GB
Kolibri-12 cards174 GB1 card87 GB1 card48 GB
Llama 4 Scout3 cards243 GB2 cards122 GB1 card68 GB
gpt-oss-120b3 cards260 GB2 cards130 GB1 card72 GB
Qwen3-235B-A22B6 cards524 GB3 cards263 GB2 cards145 GB
Llama 4 Maverick10 cards894 GB5 cards448 GB3 cards247 GB
Llama 3.1 405B10 cards907 GB5 cards456 GB3 cards253 GB
DeepSeek-V3 / R116 cards1,492 GB8 cards746 GB5 cards411 GB
GLM-5.318 cards1,654 GB9 cards827 GB5 cards455 GB

Counts are for a layer split, which works with any number of cards. Below each count is the memory needed: (weights + KV cache) ÷ 0.9. Capacity only: speed depends on memory bandwidth and the serving stack.

Memory is step one. Throughput, latency, power and the network decide the real build.

Size my actual workload

RTX PRO 6000 Workstation specifications

Memory
96 GB GDDR7
Memory bandwidth
1,792 GB/s
Typical system
Workstation
Counted as
Layer split across cards (llama.cpp style)

Questions about the RTX PRO 6000 Workstation

What LLMs can run on an RTX PRO 6000 Workstation?

For one user at an 8K context, these fit one RTX PRO 6000 Workstation. At 4-bit: every model except Llama 3.1 405B, Llama 4 Maverick, Qwen3-235B-A22B, DeepSeek-V3 / R1 and GLM-5.3. At FP8: every model except Llama 3.1 405B, Llama 4 Scout, Llama 4 Maverick, Qwen3-235B-A22B, DeepSeek-V3 / R1, gpt-oss-120b and GLM-5.3. At FP16/BF16: Llama 3.1 8B, Qwen3 8B, Qwen3 14B, Phi-4 14B, gpt-oss-20b, LFM2.5-24B-A2B, Mistral Small 3.2 24B, Gemma 3 27B, Qwen3-30B-A3B, Qwen3 32B and Qwen2.5 32B.

What is the largest LLM an RTX PRO 6000 Workstation can run?

Of the models sized here, for one user at an 8K context: at 4-bit, gpt-oss-120b (116.8B parameters, 72 GB); at FP8, Kolibri-1 (78.1B parameters, 87 GB); at FP16, Qwen3 32B (32.8B parameters, 75 GB). Mixture-of-experts models count every expert, because all of them must be in memory.

How many RTX PRO 6000 Workstation cards does Llama 3.3 70B need?

For one user at an 8K context it needs 160 GB at FP16/BF16, 81 GB at FP8 and 46 GB at 4-bit: 2 cards at FP16 and one card at FP8 or 4-bit, with a layer split.

Can an RTX PRO 6000 Workstation run DeepSeek-V3 / R1?

Not on one card: with all 671B parameters resident it needs 746 GB at FP8 and 411 GB at 4-bit for one user at an 8K context. That is 8 cards at FP8 and 5 cards at 4-bit, with a layer split.

Can I combine several RTX PRO 6000 Workstation cards?

Yes, with a layer split: each card holds some of the model's layers, as llama.cpp does. Memory adds up across them; speed does not, because they take turns on each token. The counts on this page assume a layer split.

Other GPUs

Different context length, more users, or a model that isn’t listed? The LLM GPU calculator sizes any Hugging Face model for any of these GPUs.

Need to deploy private AI?

We design it, validate it, source it, deploy it and help run it.

Get my 48-hour blueprint

Give us your workload, users, budget and timeline. Within 48 hours we'll give you a validated cloud/on-prem architecture, BOM, expected costs and sourcing options.