Memory
What fits in 32 GB of GPU memory
Answer in brief
The arithmetic for weights
Weights take about two bytes per parameter at 16-bit precision, which is roughly 2 GB for every billion parameters. At 8-bit precision it is about 1 GB per billion. At 4-bit it is about half a gigabyte.
The weights are not the whole load. Running a model needs room on top for context and activations, and that grows with prompt length, batch size, image size or the number of video frames. Keep several gigabytes free for it.
By model size
The weights of a model of 7 billion parameters take about 14 GB at 16-bit precision. That is the weights alone, and it leaves about 18 GB on top for context and activations.
The weights of a model of 13 billion take about 26 GB at 16-bit, again the weights alone. About 6 GB is left on top for context and activations, which is little. At 8-bit the weights take about 13 GB.
The weights of a model of 34 billion take about 34 GB at 8-bit, over the limit before anything else is counted. At 4-bit the weights alone take about 17 GB, with room on top for context and activations.
The weights of a model of 70 billion take about 35 GB even at 4-bit. The weights alone do not fit on one RTX 5090.
When one card is not enough
There are two ways up. Two RTX 5090s hold 64 GB in total, but only software that splits a model across cards can use it, and the split costs speed and setup.
One RTX Pro 6000 holds 96 GB on a single card. Weights of 35 GB sit whole on it, with room on top for context and activations, and nothing has to be split. Compare the hourly total of each route, not the per-card figures.
How to test a fit
Rent the card for a few minutes. Load the model at the precision you plan to use, send the largest request you expect, and watch memory use while it runs.
Write down the peak. If it holds with room left, rent with confidence. If it fails, you have lost minutes, and you know which card to try next.
Where Runcrate fits
Runcrate bills by the second on most GPUs, and booting is never billed, so a load test costs what it ran. Its list has 26 GPU models from $0.23/hr. Runcrate can start an RTX 5090 now from $0.72 per GPU-hour.
Questions and answers
Sources for this entry
- Runcrate: Runcrate’s official site: what the platform is, how it bills and the range of GPUs it offers. Read 12 Nov 2026.
- Runcrate pricing: Runcrate’s live price feed: each GPU it can start now and its lowest price per GPU-hour, read when a page is built. Read 12 Nov 2026.
- Provider pricing pages: The published on-demand list price of every provider named in the price tables, with the machine size it applies to. Read 12 Nov 2026.
- Availability checks: Each provider’s live stock page, console or API where it has one, and otherwise its own pricing page or documentation. Read 12 Nov 2026.
- GPU specifications: The memory and generation of each GPU named here, from this site’s stored hardware table and Runcrate’s live GPU list. Read 12 Nov 2026.
Checklists are original editorial guidance. Published prices do not establish availability.
Continue this reading path
Cite this entry
RTX 5090 Rental. “What fits in 32 GB of GPU memory.” Content reviewed 12 Nov 2026. https://rtx5090rental.com/topics/what-fits-in-32-gb