vram.wiki
My space Add your setup
Directory
LEARN LOCAL AI

Go from curious
to competent.

A guided path through everything you need to actually use local models: choosing hardware and models, running them well, building the harness, and proving the result. Each step links to real setups from the catalog as evidence, not theory.

48 lessons ready48 planned
0 of 48 complete
TRACK A

Understand and run

The craft, step by step, for every level.

01What local LLMs are

  1. What a local model can (and can't) doThe honest map of local capability, and the three trade-offs that decide everything.Beginner
  2. VRAM, speed, qualityWhy you always pick two, and how that shapes every setup.Beginner
  3. Context, explainedThe window that decides how much a model remembers, and why it matters more than size.Beginner
  4. The words behind the numbersThe glossary as a reading path, from VRAM to quantization to runtimes.Beginner

02Choosing hardware

  1. Reading a GPU specVRAM, bandwidth, and the CUDA/ROCm/Apple split.Beginner
  2. How much VRAM for a given modelThe rule of thumb for sizing, with real numbers.Beginner
  3. Unified memory: Apple and Strix HaloWhy bandwidth and a shared pool change the maths.Intermediate
  4. Multi-GPU and serversWhen two cards help, and when they just hold overflow.Advanced

03Choosing a model

  1. Sizes, families and MoEFrom 7B to 70B and the mixture-of-experts middle ground.Beginner
  2. Quantization, explainedQ4_K_M, IQ3, FP16 and 1-bit, without the marketing.Beginner
  3. Licenses and what you may doWhat you can legally build on top of each model.Intermediate
  4. Downloading and checking integrityTrusted sources and the checksum habit.Beginner

04Running models

  1. The runtimes, at a glancellama.cpp, Ollama, LM Studio, vLLM, SGLang, MLX.Beginner
  2. Which runtime for whomThe decision that depends on users, hardware and control.Beginner
  3. The KV cache, and why it is the real bottleneckThe invisible cost that eats VRAM as context grows.Intermediate
  4. CPU/GPU offloadWhat partial offload buys you, and what it costs in speed.Intermediate

05Configuring

  1. Sampling: temperature, top-p, top-kWhat these knobs actually change in the output.Intermediate
  2. Context limits and silent truncationThe failure mode that makes a fine model look dumb.Intermediate
  3. Threads, batch and parallelismSqueezing throughput without hurting latency.Intermediate
  4. Reading prefill vs decodeThe two speeds that matter, and how to read logs.Intermediate

06Building a harness

  1. What a harness isWhy the wrapper matters more than the model.Intermediate
  2. Agents, tools and MCPChaining a model into real work, safely.Intermediate
  3. Scoping, approval and isolationThe guardrails before you hand an agent the keys.Advanced

07RAG and data

  1. Embeddings and indexingTurning documents into things a model can search.Intermediate
  2. Chunking and retrievalHow you cut the corpus decides how good the answers are.Intermediate
  3. A local RAG, end to endThe whole pipeline on your own hardware.Advanced
  4. RAG pitfallsSourced hallucinations and sensitive data, and how to limit both.Advanced

08Fine-tuning

  1. When fine-tuning is worth itThe rare cases where it beats prompting and RAG.Advanced
  2. The fine-tuning workflowData, tools and the training loop at small scale.Advanced
  3. Fine-tuning pitfallsCatastrophic forgetting and overfitting, and how to spot them.Advanced
TRACK B

The method

Rigour and value: design, optimise, evaluate, prove.

09Design before you build

  1. Start from the objectiveWhat to achieve and why, before any tool choice.Intermediate
  2. Model the use case firstData, constraints and schema, like designing an app.Intermediate
  3. The local vs cloud decisionA decision you make on purpose, not by default.Intermediate
  4. Schema before promptShape the data, then the prompt writes itself.Advanced

10Optimising

  1. How far to quantizeChoosing the bit depth that keeps quality you can prove.Intermediate
  2. Tuning the KV cacheCache types and sizes, measured, not guessed.Advanced
  3. Memory and offload, againThe deep version: where the bytes actually go.Advanced
  4. Profiling prefill vs decodeFind the real bottleneck before you change anything.Advanced

11Build your own evaluation

  1. A real, anonymized datasetStart from the work you actually do, not a toy.Intermediate
  2. Version the dataset and pipelineReproducibility is the whole point.Intermediate
  3. Annotate what 'good' meansDefine the answer you are measuring before you measure it.Intermediate
  4. Measure the outputStats you can trust, and how to record them.Intermediate
  5. Replay and re-evaluateChange one thing, run again, compare the numbers.Advanced
  6. Keep the traceThe log that turns experiments into expertise.Advanced

12Demonstrating value

  1. Quantify the gainsTime, cost and quality, in numbers a client reads.Advanced
  2. Compare honestly to the cloudWhen local wins, when it does not, and saying so.Advanced
  3. Present to a clientExpertise by the numbers, not a demo.Advanced
  4. Regulated and confidential workThe cases where local is a requirement, not a preference.Advanced
This path is a wiki. The facts are explained here; the evidence lives in the catalog. Contribute a lesson.