What a local model can (and can't) do

Local LLMs have stopped being a party trick. People run entire coding workflows, document pipelines and small businesses on them. But the honest answer to "what can I do locally?" is a list with a sharp edge on both ends, and the whole point of this wiki is to show you the real version, not the hype.

What local is genuinely good at

• Repetitive work on your own data. Summarizing, extracting, classifying, drafting from documents you already have. The model does not need to be the smartest; it needs to be consistent.
• Work that must not leave the machine. Client data, health records, legal documents, credentials. For regulated or confidential work, local is often not a preference, it is a requirement.
• Long-running, cheap automation. A cron job that processes mail overnight, a helper that stays available without per-token anxiety.
• Control. Nobody upgrades the model under you, nobody changes the behaviour of your system, nobody retires your endpoint.

What local still loses at

• Raw intelligence on hard, novel reasoning. Frontier cloud models are still ahead, and a 27B is not a trillion-parameter model. For the hardest one-shot problems, cloud usually wins.
• Speed on a budget. A small card gives you usable, not instant, generation.
• Convenience. There is setup, drivers, quant choices and a learning curve. The convenience tax is real and it is why many people stay on cloud.

The three trade-offs

Almost every local decision is a triangle with three corners and you get to pick about two:

1. VRAM (how big a model fits),
2. Speed (tokens per second),
3. Quality (how close to the cloud the output is).

A bigger model costs VRAM and speed. A smaller one is fast but weaker. A heavily quantized model fits in less VRAM but loses a little quality. You will see this triangle again in every chapter.

The missing variable: context

One thing separates local from the demo era more than anything: context. A model that can hold 100k tokens of your project, your corpus or your conversation is dramatically more useful than the same model at 4k. Context is where local setups live or die, so it gets its own lesson.

Where this guide fits

This is the "understand" path. Once you have the map, the catalog on vram.wiki is the evidence: every claim here is backed by a real setup someone published, with real hardware, real numbers and honest limitations. Wherever a lesson needs a concrete example, it links one.