// cloud vs on-prem ai
Rent GPUs or own them?
The answer is mostly one number: how many hours a month your GPUs are genuinely busy.
// the short answer
Utilisation decides it
Cloud GPUs charge for every hour you hold them; owned GPUs cost roughly the same whether they are busy or idle. So the more hours your workload keeps GPUs busy, the stronger the case for owning.
Experiments, pilots and spiky demand favour cloud. Steady production inference, data that must stay on site, and multi-year roadmaps favour on-prem or a dedicated private cloud. Many teams end up hybrid: a baseline they own, with bursts in the cloud.
The other half of the decision is physical. An 8-GPU server can draw 10 kW or more, which many offices and older colo cages can't power or cool. That has to be checked before any hardware is quoted.
// break-even calculator
Put in your quotes. See which wins.
Starting values are illustrative, not prices. Replace them with your cloud quote, your hardware quote and your power rate.
Cloud / month
$16,000
68% utilisation
On-prem / month
$12,560
incl. depreciation, power, ops
Verdict
On-prem is cheaper at 68% utilisation.
On-prem pays back its hardware in about 26 months versus staying in cloud.
Cloud is billed only for hours used; on-prem draws power around the clock. Excludes staff time, financing, egress and resale value; the assessment includes them.
// beyond price
What else changes between the two
| Factor | Cloud GPUs | Owned (on-prem or colo) |
|---|---|---|
| Time to start | Hours to days, if capacity is available | Weeks: quote, lead time, rack, power |
| Cost shape | Operating expense, per hour | Capital expense, depreciated |
| Idle cost | Zero if released | Depreciation and power continue |
| Data location | Provider region and tenancy | Your premises or your cage |
| Capacity risk | Popular GPUs can be scarce on demand | Fixed once installed |
| Upgrades | Switch instance types | Plan a refresh or resale |
// before you decide
Six numbers to have ready
- GPU hours you will genuinely use per month, at peak and on average
- A cloud quote for that capacity, on-demand and reserved
- A hardware quote including networking, storage and support
- Rack power and cooling available, in kW per rack
- Your electricity rate and facility PUE
- How long the hardware must stay useful before refresh
Not sure about the GPU count?
Size the memory first with the LLM GPU calculator, then bring both numbers to the assessment. We return cloud and on-prem costed side by side in 48 hours.
Need to deploy private AI?
We design it, validate it, source it, deploy it and help run it.
Give us your workload, users, budget and timeline. Within 48 hours we'll give you a validated cloud/on-prem architecture, BOM, expected costs and sourcing options.