The runtimes, at a glance

A runtime is the engine that loads a model and turns it into tokens. The same model file, served by different runtimes, gives you different speed, different concurrency and different levels of control. Here is the landscape in one pass.

The foundation

• llama.cpp — the C/C++ engine that started it all. Runs GGUF quants on CPU and GPU, exposes every knob as a flag, and is the base most other tools build on. Maximum control, more work.

The friendly layers

• Ollama — a wrapper over llama.cpp with a one-command workflow and a clean API. The default for getting started, scripting and self-hosted apps.
• LM Studio — a desktop app with a visual UI, model browser and local server. For people who want to click, not type flags.
• Jan, GPT4All — similar desktop conveniences, each with its own ecosystem.

The server engines

• vLLM — built for throughput and concurrency. Overkill for one user, the right answer when many clients or agents share one model.
• SGLang — a newer engine in the same class, strong on fast structured output and advanced serving.
• TensorRT-LLM, ExLlamaV2 — GPU specialists that squeeze a specific card hard, at the cost of more setup.

The Apple lane

• MLX — Apple Silicon native. Uses unified memory well and is the fast, modern choice on Mac.

How to read the choice

The runtime decision is rarely about "best," it is about your situation:

• Getting started → Ollama or LM Studio.
• Want every knob → llama.cpp.
• Apple → MLX.
• Multiple users / agents → vLLM or SGLang.

Every one of these can run the same Qwen or Llama file. The model is not the differentiator; the runtime is what decides how that model behaves under your workload. The next lesson turns this into a concrete decision.