How much VRAM for a given model

This is the question everyone asks first, and the answer is a rule of thumb with two parts: the model size, then the context.

The rule of thumb

A model's memory need is roughly:

weights + KV cache + a little overhead

Weights are the model itself. The KV cache grows with your context length and is the part people forget. The short version:

8 GB runs 7–9B models at Q4, or a MoE at low quant, with a modest context.
12 GB is the comfortable home of 7–9B at high quant and of the ~30B MoE models, which stay fast because only part of the weights activate per token.
16 GB runs 14B at Q4, or a 27B that is tight and needs a small context or partial offload.
24 GB is the classic sweet spot: a 27B at Q4 with room for a large context, or a 70B squeezed in at low quant.
32 GB and up opens 70B comfortably and the workstation class.
96 GB+ is the multi-GPU and unified-memory territory where you stop counting and start planning.

These are starting points, not guarantees. Quantization, context and the exact runtime move every number.

Why quantization changes everything

A 7B model is about 14 GB in FP16. At Q4 it is roughly 4–5 GB. At IQ3 it is closer to 3 GB. So the same card can run a completely different class of model depending on the quant. That is why you will see quantization get its own lesson: it is the single biggest lever in the VRAM question.

The part everyone forgets: the KV cache

The KV cache holds the conversation's or the document's state and grows linearly with context. At 32k context a 27B can add several GB of cache on top of the weights. At 128k it can be larger than the model itself. Two practical consequences:

Always budget cache, not just weights.
Use a quantized KV cache (usually q8_0) when the runtime offers it. It usually cuts the cache roughly in half for almost no quality loss.

A worked example

You have a 24 GB card and want to run a 27B at Q4 for coding with a 64k context:

weights, Q4: about 16 GB,
KV cache at 64k, q8_0: roughly 6 GB,
overhead: about 1 GB.

That is about 23 GB, it fits, but there is nothing left over. Drop the context to 32k and it is comfortable; try 128k and it will offload or swap and feel slow. That is the trade-off triangle from the first lesson, made concrete.

Check against real setups

Rather than trust the arithmetic, check the catalog. On vram.wiki you can filter by exact reported VRAM and see what people actually run at each size, with their real tokens-per-second and their honest limitations. Start from your card, not from the marketing.