AI servers: how to choose the right GPU
For training, fine-tuning or serving a model, the graphics card is the decision that matters most. This guide covers the real selection criteria and how the common cards differ.
What should an AI server have?
An AI server is a server with a graphics card suited to parallel maths. Three things decide whether your model runs on it: graphics memory, memory bandwidth, and disk space for datasets and checkpoints.
Graphics memory (VRAM)
The first and hardest limit. If the model and optimiser do not fit in memory, raw speed does not help.
Memory bandwidth
In inference and training alike, how fast data moves often matters more than core count.
Disk and network
Datasets and checkpoints are large. Slow disk leaves a fast card waiting.
Which card for which job?
The common cards compared by memory and workload, not by shop benchmarks.
| Card | Memory | Best for | Note |
|---|---|---|---|
| RTX 4090 | 24 GB | Fine-tuning mid-size models, rendering, serving 7–13B models | Best price-to-performance entry point |
| L40S | 48 GB | Larger models, professional rendering, mixed AI and graphics workloads | Twice the memory at lower power than data-centre cards |
| A100 80GB | 80 GB | Training large language models, daily or monthly reservations | The common choice for multi-user training |
| H100 80GB | 80 GB | Training at scale, multi-card clusters, larger models | The fastest option, for when training time really costs money |
Memory figures are the manufacturer specifications. Check the GPU page for what is available right now.
Dedicated card or virtual GPU?
Both work; the difference is consistency of performance and cost.
Dedicated card
- All the card’s memory and compute is yours
- Stable performance across long training runs
- Suited to production and multi-user loads
Shared card
- Lower cost per user
- Enough for experiments and small models
- Can slow down under heavy load
How to think about cost
The price of an AI server depends on the card, how many, and how long you use them.
Hourly, for experiments
For trying a model or a short run, take an hourly server and remove it when you are done.
Daily, for short projects
When you have several days of training or rendering ahead, daily billing is more predictable than hourly.
Monthly and yearly, for production
For a service that is always on, a long-term plan is cheaper and reserves the capacity.
Frequent questions
Memory is the main criterion. Fine-tuning 7–13B models fits on a 24 GB card with optimisation; full training and larger models want 48 or 80 GB.
Pick the card that fits your workload
From RTX 4090 to start, up to H100 for training at scale, all billed hourly, daily, monthly or yearly.
See GPU servers