Skip to content

Tools / LLM GPU calculator / L40S

What LLMs fit on an NVIDIA L40S?

Which of 22 open LLMs fit one L40S, and how many it takes for the rest, at FP16, FP8 and 4-bit.

Size your own scenario

Short answer

One NVIDIA L40S (48 GB) runs 15 of the 22 open LLMs sized here at 4-bit, 11 at FP8 and 5 at FP16, for one user at an 8K context. At FP8: Llama 3.1 8B, Qwen3 8B, Qwen3 14B, Phi-4 14B, gpt-oss-20b, LFM2.5-24B-A2B, Mistral Small 3.2 24B, Gemma 3 27B, Qwen3-30B-A3B, Qwen3 32B and Qwen2.5 32B. The largest that fits one GPU is Kolibri-1 (78.1B parameters), at 4-bit. Within one 8-GPU server, tensor parallelism runs every model except Llama 3.1 405B, Llama 4 Maverick, Qwen3-235B-A22B, DeepSeek-V3 / R1 and GLM-5.3 at FP8. Llama 3.3 70B takes 4 GPUs (tensor parallel 4) at FP16 and 2 GPUs (tensor parallel 2) at FP8.

Updated . One user, 8K context, KV cache at FP16, plus 10% headroom for activations and the runtime.

Every model on the L40S

ModelFP16/BF16FP84-bit (INT4)
Llama 3.1 8B1 GPU19 GB1 GPU10 GB1 GPU6.1 GB
Qwen3 8B1 GPU20 GB1 GPU10 GB1 GPU6.3 GB
Qwen3 14B1 GPU34 GB1 GPU18 GB1 GPU11 GB
Phi-4 14B1 GPU35 GB1 GPU18 GB1 GPU11 GB
gpt-oss-20b1 GPU47 GB1 GPU23 GB1 GPU13 GB
LFM2.5-24B-A2B2 GPUs · TP254 GB1 GPU27 GB1 GPU15 GB
Mistral Small 3.2 24B2 GPUs · TP255 GB1 GPU28 GB1 GPU16 GB
Gemma 3 27B2 GPUs · TP262 GB1 GPU32 GB1 GPU18 GB
Qwen3-30B-A3B2 GPUs · TP269 GB1 GPU35 GB1 GPU20 GB
Qwen3 32B2 GPUs · TP275 GB1 GPU39 GB1 GPU22 GB
Qwen2.5 32B2 GPUs · TP275 GB1 GPU39 GB1 GPU22 GB
Mixtral 8x7B4 GPUs · TP4105 GB2 GPUs · TP253 GB1 GPU30 GB
Llama 3.3 70B4 GPUs · TP4160 GB2 GPUs · TP281 GB1 GPU46 GB
Qwen2.5 72B4 GPUs · TP4165 GB2 GPUs · TP284 GB1 GPU47 GB
Kolibri-14 GPUs · TP4174 GB2 GPUs · TP287 GB1 GPU48 GB
Llama 4 Scout8 GPUs · TP8243 GB4 GPUs · TP4122 GB2 GPUs · TP268 GB
gpt-oss-120b8 GPUs · TP8260 GB4 GPUs · TP4130 GB2 GPUs · TP272 GB
Qwen3-235B-A22BMulti-server524 GBMulti-server263 GB4 GPUs · TP4145 GB
Llama 4 MaverickMulti-server894 GBMulti-server448 GB8 GPUs · TP8247 GB
Llama 3.1 405BMulti-server907 GBMulti-server456 GB8 GPUs · TP8253 GB
DeepSeek-V3 / R1Multi-server1,492 GBMulti-server746 GBMulti-server411 GB
GLM-5.3Multi-server1,654 GBMulti-server827 GBMulti-server455 GB

Counts use a tensor-parallel size (TP) that divides the model's attention heads, within one 8-GPU server. Below each count is the memory needed: (weights + KV cache) ÷ 0.9. Capacity only: speed depends on memory bandwidth and the serving stack.

Memory is step one. Throughput, latency, power and the network decide the real build.

Size my actual workload

L40S specifications

Memory
48 GB GDDR6
Memory bandwidth
864 GB/s
Typical system
PCIe server
Counted as
Tensor parallel within one 8-GPU server (vLLM style)

Configurations and pricing: NVIDIA L40S in the catalog.

Questions about the L40S

What LLMs can run on an NVIDIA L40S?

For one user at an 8K context, these fit one L40S. At 4-bit: every model except Llama 3.1 405B, Llama 4 Scout, Llama 4 Maverick, Qwen3-235B-A22B, DeepSeek-V3 / R1, gpt-oss-120b and GLM-5.3. At FP8: Llama 3.1 8B, Qwen3 8B, Qwen3 14B, Phi-4 14B, gpt-oss-20b, LFM2.5-24B-A2B, Mistral Small 3.2 24B, Gemma 3 27B, Qwen3-30B-A3B, Qwen3 32B and Qwen2.5 32B. At FP16/BF16: Llama 3.1 8B, Qwen3 8B, Qwen3 14B, Phi-4 14B and gpt-oss-20b.

What is the largest LLM an L40S can run?

Of the models sized here, for one user at an 8K context: at 4-bit, Kolibri-1 (78.1B parameters, 48 GB); at FP8, Qwen3 32B (32.8B parameters, 39 GB); at FP16, gpt-oss-20b (20.9B parameters, 47 GB). Mixture-of-experts models count every expert, because all of them must be in memory.

How many NVIDIA L40S GPUs does Llama 3.3 70B need?

For one user at an 8K context it needs 160 GB at FP16/BF16, 81 GB at FP8 and 46 GB at 4-bit: 4 GPUs (tensor parallel 4) at FP16, 2 GPUs (tensor parallel 2) at FP8 and 1 GPU at 4-bit.

Can an NVIDIA L40S run DeepSeek-V3 / R1?

Not on one GPU: with all 671B parameters resident it needs 746 GB at FP8 and 411 GB at 4-bit for one user at an 8K context. That is more than one 8-GPU server (at least 16 GPUs) at FP8 and more than one 8-GPU server (at least 9 GPUs) at 4-bit.

Other GPUs

Different context length, more users, or a model that isn’t listed? The LLM GPU calculator sizes any Hugging Face model for any of these GPUs.

Need to deploy private AI?

We design it, validate it, source it, deploy it and help run it.

Get my 48-hour blueprint

Give us your workload, users, budget and timeline. Within 48 hours we'll give you a validated cloud/on-prem architecture, BOM, expected costs and sourcing options.