Multi-GPU and servers

Two cards do not always mean twice the model. How (and whether) a second GPU helps depends on what you are trying to do, and most people overbuy here.

The three things a second GPU can do

More total VRAM, with a catch. You can split a model across two cards, but unless the runtime does real tensor parallelism, the two cards talk over the PCIe bus and you lose much of the speed to the transfer.
KV cache or offload. The second card holds the KV cache, or the overflow layers, so the first card runs the model and the second catches the extra. This is usually the honest reason a second card helps.
Concurrent users. Each card can serve a separate request or model in parallel. This is where multi-GPU really pays, on a server with several clients.

The honest rule

For one person running one model, a second identical card rarely doubles your speed. It mostly gives you room (bigger context, a bigger model) at roughly the same, or slightly better, speed. If that room is what you need, it is worth it. If you expected twice the tokens-per-second, you will be disappointed.

What actually matters in a server

Bandwidth and PCIe lanes. Splitting a model across cards is only fast when the interconnect is fast (NVLink) or the runtime batches well.
Cooling and power. Several cards means several hundred watts. The case, the PSU and the noise are part of the design.
The runtime choice. vLLM and SGLang are built for this; a single-user runtime is not.

Before you buy a second card

Ask what problem it solves. "I want a bigger context" is a good reason. "I want it faster" usually is not, on its own. Then check the catalog: filter by the multi-GPU setups and read what people actually report, including the honest "it did not scale" notes.

Multi-GPU is a real technique, not a cheat code. It extends what fits; it does not multiply what you already have.