VRAM, speed, quality

Every local decision is the same three-way pull: VRAM (how big a model fits), speed (tokens per second) and quality (how close to the cloud the output is). You can usually have two, rarely all three. Understanding this triangle is the whole game.

The three corners

• VRAM is the hard limit. If a model does not fit, nothing else matters. It is the least negotiable corner because it is physical.
• Speed is usability. A model that answers in five minutes is a different tool from one that answers in five seconds, even if the text is identical.
• Quality is the reason you are here. A fast model that is wrong is worse than no model.

What moves each corner

Every lever trades one corner for another:

• A bigger model buys quality, costs VRAM and speed.
• Quantization (Q4, Q3, IQ3…) buys VRAM and speed, costs a little quality. At Q4 the loss is usually small; at Q2 it starts to be visible.
• Longer context buys usefulness, costs VRAM (the KV cache grows) and a little speed.
• Offload to CPU buys VRAM, costs a lot of speed.
• A better runtime or batching buys speed, usually without costing the others.

The useful consequence

The triangle explains almost every setup you will read. "Why does this person run a 27B at Q4 instead of a 70B at Q2?" Because they chose quality and speed over raw size. "Why do they offload half the layers to CPU?" Because they had RAM but not VRAM.

When you read a setup on vram.wiki, ask which two corners the author chose. That is the real story, and it is what the hardware and the numbers are actually telling you.